Memory allocation method, related device and computer-readable storage medium

Abstract

The present application provides a memory allocation method, related equipment, and computer-readable storage medium, the method comprising: obtaining a computational graph corresponding to a neural network; allocating memory space to the M tensor data in sequence based on the sorting results of M tensor data, wherein, if one of the M tensor data can reuse at least a portion of the allocated memory space, at least a portion of the memory space that can be reused by the tensor data is allocated to the tensor data, the allocated memory space being the memory space that has been allocated to the M tensor data before the tensor data, the sorting result indicating the order in which the memory space is allocated to the M tensor data, and the sorting result being related to the information of each tensor data in the M tensor data. Implementation of the present application can avoid unreasonable memory allocation.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a memory allocation method, related equipment, and computer-readable storage medium. Background Art

[0002] In the current field of computer deep learning, to achieve better algorithm accuracy, deep learning neural networks are becoming increasingly complex. Hardware capabilities limit the development of neural networks in deeper directions, and memory optimization is necessary. To achieve memory optimization, the industry generally adopts the following memory allocation strategies:

[0003] Run the entire neural network, and then allocate memory for it in the order in which it runs. For example, during its execution, the neural network requires 100MB of memory, 10MB of memory, and then 50MB of memory. When the neural network requests 100MB of memory, it can allocate 100MB of memory to it. Then, when the neural network requests 10MB of memory, it determines whether the previously allocated 10MB of memory can be reused. If so, it reuses the previously allocated 10MB of memory without allocating new memory for the requested 10MB. Similarly, when the neural network requests 50MB of memory, it first determines whether the 50MB of memory can be reused within the previously allocated 100MB of memory. If so, it does not allocate new memory for the requested 50MB of memory.

[0004] From the above description, we can know that in the prior art, when a neural network applies for a memory space, it first needs to determine whether the memory space applied for can reuse the allocated memory space. If it can, the memory space applied for will be directly allocated to reuse the allocated memory space. If not, new memory space will be allocated for the memory space application. However, if the applied 10M memory space and the applied 50M memory space can both reuse the allocated 100M memory space, the applied 10M memory space will reuse the allocated 100M memory space, and an additional 50M memory space will be allocated to the neural network, so that the entire neural network needs to occupy 150M of memory space, resulting in a large amount of memory occupied by the entire neural network and unreasonable memory allocation. Summary of the Invention

[0005] This application provides a memory allocation method, related devices, and computer-readable storage medium that can avoid irrational memory allocation. For example, irrational memory allocation can be manifested in: the entire neural network occupies a large amount of memory.

[0006] In a first aspect, a memory allocation method is provided, which may include the following steps: first, obtaining a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the nodes are used to indicate a computational logic in the neural network, and the directed edges are used to indicate the flow direction of tensor data in the computational logic; the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1; second, based on the sorting results of the M tensor data, memory space is allocated to the M tensor data in sequence, wherein if one of the M tensor data can reuse at least a portion of the allocated memory space, Then at least a portion of the memory space that can be reused by the tensor data is allocated to the tensor data. The allocated memory space is the memory space that has been allocated to M tensor data before the tensor data. The sorting result indicates the order in which the memory space is allocated to the M tensor data. The sorting result is related to the information of each tensor data in the M tensor data. The information of each tensor data indicates at least one of the following information: the constraint relationship corresponding to each tensor data and the number of nodes to which each tensor data flows. The constraint relationship indicates the relationship between the available memory space of one tensor data in the M tensor data and the available memory space of other tensor data in the M tensor data. Among them, the phrase "memory space that has been allocated to M tensor data" takes the M tensor data as a whole and describes the memory space that has been allocated to this whole. There may be some tensor data in this whole that have not yet been allocated to memory space. Specifically, it means that before allocating memory space to the tensor data described in the above method, the memory space of one or more tensor data in the M tensor data has been allocated memory space. For example, if the memory space is allocated to the mth tensor data in the sorting result in order, then the memory space allocated to the M tensor data is the memory space allocated to the first m-1 allocated tensor data, where m is less than M and greater than 1.

[0007] Of course, for the first tensor in the sorting result, since there is no memory space allocated to M tensor data, the memory space can be directly allocated. This is the existing technology and will not be described in detail.

[0008] In layman's terms, the node to which tensor data flows is called a consumer node, and the node from which tensor data flows out is called a producer node.

[0009] It should be noted that a tensor data can be carried on different directed edges, and a tensor data can also be carried on a directed edge.

[0010] In implementing the embodiment of the present application, the memory allocation device allocates memory space of corresponding size to each tensor data in sequence based on the sorting results of M tensor data. Compared with the existing technology, the memory space is allocated and reused according to the order of the operation of the entire neural network, which can avoid the occurrence of unreasonable memory allocation, thereby saving the memory required by the entire neural network and optimizing the memory allocation of the neural network.

[0011] In a possible implementation, the method may further include the following steps: if the tensor data cannot reuse the allocated memory space, allocating other memory space for the tensor data, where the other memory space is different from the allocated memory space.

[0012] In one possible implementation, the constraint relationship indicates at least one of the following relationships: the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable, and the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable and continuous. In actual applications, the above constraint relationships have different priorities. In other words, when the relationship between the available memory spaces of two tensors is non-reusable and continuous, it must also be non-reusable. Then the relationship between the available memory spaces of the two tensors is indicated as non-reusable and continuous in the constraint relationship. In other words, it can be understood that non-reusable and continuous has a higher priority than non-reusable.

[0013] In one possible implementation, the constraint relationship is carried in a constraint relationship table, which includes the identifiers of M data tensors. In the constraint relationship table, the first value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the second value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and the third value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous. Specifically, the first value, the second value, and the third value can be mutually distinguishable values. For example, the first value can be "0", the second value can be "1", and the third value can be "2". Combined with the above, in one implementation, if a relationship is not reusable and continuous, this relationship in the constraint relationship table will only be marked as "2", rather than "1" and "2". This implementation facilitates the subsequent combination of the constraint relationship to obtain the sorting results of the M tensor data. Furthermore, when the memory space of corresponding sizes is allocated to the tensor data in sequence based on the sorting result, memory irrationality can be avoided.

[0014] In some possible implementations, when all the consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all the consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all the consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all the consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows. Specifically, node A is the upstream node of node B, which means that in the computational graph, one or more directed edges can be passed from node A to node B, and node A and node B can be connected by a directed edge from node A to node B, or by multiple directed edges and the nodes through which these directed edges pass. First, through this implementation, the constraint relationship of each tensor data can be determined, which provides a basis for obtaining the sorting results of the M tensor data in the future. When the memory space of corresponding size is allocated to the tensor data in sequence based on the sorting results, the unreasonable memory phenomenon can be avoided.

[0015] In one possible implementation, a computational graph includes multiple computational subtasks, each of which indicates a computational function through a set of nodes and edges associated with the set of nodes, and the execution relationship between the multiple computational subtasks is parallel execution; the method may further include the following steps: in a computational subtask, if there is no directed edge between two adjacent nodes, a directed edge is added between the two adjacent nodes to update the computational graph; wherein each added directed edge carries corresponding tensor data; the two adjacent nodes are two nodes in an execution order adjacent to the computational subtask; here, the execution order refers to an order with a temporal relationship. Based on the updated computational graph, information about each tensor data is obtained. In implementing an embodiment of the present application, when the execution relationships between the computational subtasks are all parallel, in a computational subtask in the computational graph, if there is no directed edge between two adjacent nodes, a directed edge is added between the two adjacent nodes to update the computational graph, providing a basis for subsequent analysis of the ancestral relationship of each node (e.g., upstream node) based on the computational graph and determination of the constraint relationship corresponding to each node. It should be noted that the parallel execution relationship between multiple computing subtasks refers to the fact that the time periods required to execute the multiple computing subtasks overlap under the same time base, and does not emphasize that the computing subtasks start and / or end at the same time. In actual applications, the computing subtasks with such a parallel execution relationship can be executed in parallel by different processor cores.

[0016] In one possible implementation, the computation graph also includes a first computation subtask and a second computation subtask whose execution relationship is serial, with the first computation subtask executing before the second computation subtask. The implementation process of updating the computation graph may further include the following step: if there is no directed edge between the last node of the first computation subtask and the first node of the second computation subtask, then adding a directed edge between the last node of the first computation subtask and the first node of the second computation subtask. When the execution relationship between computation subtasks includes both serial and parallel execution, this implementation method can update the computation graph, providing a basis for subsequent analysis of the ancestry relationships of each node and determination of the constraints corresponding to each node based on the computation graph. For example, if the execution relationship between computation subtask 1 and computation subtask 2 is serial, computation subtask 2 will be executed only after the processor completes execution of computation subtask 1. For another example, if the execution relationship between computation subtask 1 and computation subtask 2 is serial, and the execution relationship between computation subtask 2 and computation subtask 3 is parallel, computation subtask 1 and computation subtask 2 can be treated as a whole, with computation subtask 1 and computation subtask 2 executed by processor core 1, and computation subtask 3 executed by processor core 2. The time periods required by processor core 1 and processor core 2 to execute the above computing subtasks overlap when the time base is the same.

[0017] In one possible implementation, in a computational graph, the identifier of a node producing tensor data is smaller than the identifier of a node consuming the same tensor data; the producing node and the consuming node are adjacent nodes. This approach allows for the identification of each node, providing a foundation for subsequent analysis of ancestry relationships and determination of constraints based on the identifiers of each node.

[0018] In one possible implementation, the identifier of each node in the computation graph is used to determine the information of each tensor data in the M tensor data. Taking the example of the information of each tensor data indicating the constraint relationship of each tensor data, the ancestry relationship of each node can be analyzed based on the identifier of each node (the ancestry relationship can reflect which nodes are production nodes and which nodes are consumption nodes), and then the constraint relationship of each tensor data can be obtained based on the ancestry relationship.

[0019] In one possible implementation, the information of each tensor data indicates the constraint relationship corresponding to each tensor data. The method may further include the following steps: obtaining the constraint amount corresponding to each of the M tensor data according to the constraint relationship corresponding to each tensor data; the constraint amount is the number of tensor data in other tensor data that cannot reuse the same memory space with the tensor data; sorting the M tensor data according to the constraint amount corresponding to each of the M tensor data to obtain the sorting result of the M tensor data.

[0020] In one possible implementation, the information of each tensor data indicates the number of nodes to which each tensor data flows, and the method further includes: sorting the M tensor data according to the number of consumer nodes corresponding to each of the M tensor data to obtain a sorting result of the M tensor data.

[0021] It should be noted that in some possible implementations, the M tensor data may be sorted from large to small based on at least two of the information of each tensor data, to obtain a sorting result for the M tensor data. For example, the M tensor data may be sorted from large to small based on the constraint amount corresponding to each of the M tensor data and the memory space size corresponding to each of the tensor data, to obtain a sorting result for the M tensor data. For another example, the M tensor data may be sorted from large to small based on the constraint amount corresponding to each of the M tensor data and the number of consumer nodes corresponding to each of the tensor data, to obtain a sorting result for the M tensor data.

[0022] In some possible implementations, the method may further include the following steps: sorting the M tensor data using a heuristic algorithm based on the information of each tensor data to obtain the sorting results of the M tensor data within a preset time period. In one possible implementation, the sorting result is an optimized sorting result, wherein the maximum memory size required to be occupied by the neural network corresponding to the optimized sorting result is smaller than the maximum memory size required to be occupied by the neural network determined according to the sorting result before optimization. Through this implementation, since the maximum memory size required to be occupied by the neural network determined by the optimized sorting result is smaller than the maximum memory size required to be occupied by the neural network determined according to the sorting result, memory space can be saved.

[0023] In the second aspect, an embodiment of the present application also provides a memory allocation method, which may include the following steps: first, obtaining a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the nodes are used to indicate a computational logic in the neural network, and the directed edges are used to indicate the flow direction of tensor data in the computational logic; the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1; secondly, based on the constraint relationship corresponding to each tensor data, memory space is allocated to the M tensor data in sequence according to the execution order of the M tensor data in the neural network, wherein, if one of the M tensor data can reuse at least a part of the allocated memory space, at least a part of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data, and the constraint relationship indicates the relationship between the available memory space of one tensor data among the M tensor data and the available memory space of other tensor data among the M tensor data. By implementing the embodiments of the present application, the terminal device can allocate memory space to the M tensor data in sequence based on the constraint relationship corresponding to each tensor data and in the execution order of the M tensor data. This can avoid the situation in which, in a parallel scenario, the operator calculation results in different execution flows are erroneous due to the operators reusing the same memory space, and can ensure the accuracy of the calculation results of the neural network.

[0024] In a possible implementation, the method may further include the following steps: if the tensor data cannot reuse the allocated memory space, allocating other memory space for the tensor data, where the other memory space is different from the allocated memory space.

[0025] In general, the method proposed in this application can solve the problem of unreasonable memory allocation. For example, unreasonable memory allocation can be reflected in: avoiding excessive memory allocation for neural networks, avoiding errors in operator calculation results caused by operators reusing the same memory space in different execution flows in parallel scenarios, and ensuring the accuracy of the calculation results of the neural network.

[0026] In a third aspect, an embodiment of the present application provides a memory allocation device, which may include: an acquisition calculation graph unit for acquiring a calculation graph corresponding to a neural network; wherein the calculation graph includes N nodes and directed edges connecting different nodes, and the directed edges of the calculation graph carry tensor data, and the calculation graph includes M tensor data, where M is an integer greater than 1; an allocation unit for allocating memory space to the M tensor data in sequence based on the sorting results of the M tensor data, wherein if one of the M tensor data can reuse at least a part of the allocated memory space, at least one of the reusable tensor data is allocated. A portion of memory space is allocated to tensor data. The allocated memory space is the memory space that has been allocated to M tensor data before the tensor data. The sorting result indicates the order in which memory space is allocated to the M tensor data. The sorting result is related to the information of each tensor data in the M tensor data. The information of each tensor data indicates at least one of the following information: the constraint relationship corresponding to each tensor data and the number of nodes to which each tensor data flows. The constraint relationship indicates the relationship between the available memory space of one tensor data in the M tensor data and the available memory space of other tensor data in the M tensor data.

[0027] In a possible implementation, the allocation unit is further configured to: if the tensor data cannot reuse the allocated memory space, allocate other memory space for the tensor data, where the other memory space is different from the allocated memory space.

[0028] In one possible implementation, the constraint relationship indicates at least one of the following relationships: the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable, and the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable and continuous.

[0029] In one possible implementation, the constraint relationship is carried in a constraint relationship table, which includes the identifiers of M data tensors. In the constraint relationship table, the first value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the second value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and the third value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous.

[0030] In one possible implementation, when all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows out.

[0031] In one possible implementation, a computational graph includes multiple computational subtasks, each of which indicates a computational function through a group of nodes and edges associated with the group of nodes, and the execution relationship between the multiple computational subtasks is parallel execution; the device also includes: an updating computational graph unit, configured to add a directed edge between two adjacent nodes in a computational subtask if there is no directed edge between the two adjacent nodes to update the computational graph; wherein each added directed edge carries corresponding tensor data; the two adjacent nodes are two nodes adjacent in execution order in the computational subtask; and an information acquisition unit, configured to acquire information of each tensor data based on the updated computational graph.

[0032] In one possible implementation, the computation graph also includes a first computation subtask and a second computation subtask whose execution relationship is serial, and the execution order of the first computation subtask is before the execution order of the second computation subtask; the computation graph updating unit is also used to: if there is no directed edge between the last node of the first computation subtask and the first node of the second computation subtask, then add a directed edge between the last node of the first computation subtask and the first node of the second computation subtask.

[0033] In a possible implementation, in the computational graph, an identifier of a production node of tensor data is smaller than an identifier of a consumption node of the tensor data; the production node of the tensor data and the consumption node of the tensor data are two adjacent nodes.

[0034] In a possible implementation, the identifier of each node in the computation graph is used to determine information of each tensor data in the M tensor data.

[0035] In one possible implementation, the information of each tensor data indicates the constraint relationship corresponding to each tensor data, and the device also includes: a first sorting unit, used to obtain the constraint amount corresponding to each of the M tensor data according to the constraint relationship corresponding to each tensor data; the constraint amount is the number of tensor data in other tensor data that cannot reuse the same memory space with the tensor data; according to the constraint amount corresponding to each of the M tensor data, the M tensor data are sorted to obtain the sorting result of the M tensor data.

[0036] In one possible implementation, the information of each tensor data indicates the number of nodes to which each tensor data flows, and the device also includes: a second sorting unit, used to sort the M tensor data according to the number of consumer nodes corresponding to each of the M tensor data to obtain the sorting results of the M tensor data.

[0037] In a possible implementation, the apparatus further includes:

[0038] A third sorting unit is configured to sort the M tensor data using a heuristic algorithm based on information of each tensor data, so as to obtain a sorting result of the M tensor data within a preset time period.

[0039] In one possible implementation, the sorting result is an optimized sorting result, wherein the maximum memory size required to be occupied by the neural network corresponding to the optimized sorting result is smaller than the maximum memory size required to be occupied by the neural network determined based on the sorting result before optimization.

[0040] In a fourth aspect, an embodiment of the present application further provides a memory allocation device, which may include: an acquisition computation graph unit for acquiring a computation graph corresponding to a neural network; wherein the computation graph includes N nodes and directed edges connecting different nodes, the directed edges of the computation graph carry tensor data, and the computation graph includes M tensor data, where M is an integer greater than 1; an allocation unit for allocating memory space to the M tensor data in sequence based on the constraint relationship corresponding to each tensor data and in accordance with the execution order of the M tensor data in the neural network, wherein, if one of the M tensor data can reuse at least a part of the allocated memory space, at least a part of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data, and the constraint relationship indicates the relationship between the available memory space of one tensor data among the M tensor data and the available memory space of other tensor data among the M tensor data.

[0041] In a possible implementation, the allocating unit is further configured to: if the tensor data cannot reuse the allocated memory space, allocate other memory space for the tensor data, where the other memory space is different from the allocated memory space.

[0042] In the fifth aspect, an embodiment of the present application also provides a memory allocation device, which may include a memory and a processor, wherein the memory is used to store a computer program that supports the memory allocation device to execute the above-mentioned method, and the computer program includes program instructions. The processor is configured to call the program instructions to execute the memory allocation method of any one of the first aspects or any one of the second aspects.

[0043] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the memory allocation method of any one of the first aspects or any one of the second aspects mentioned above.

[0044] In the seventh aspect, an embodiment of the present application further provides a computer program, which includes computer software instructions, and when the computer software instructions are executed by a computer, the computer executes the memory allocation method as described in any one of the first aspect or any one of the second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1a A schematic diagram of the structure of a computational graph of a neural network provided in an embodiment of the present application;

[0046] Figure 1b A schematic diagram of the structure of another neural network calculation graph provided in an embodiment of the present application;

[0047] Figure 1c A schematic diagram of the execution order of operators in a computation graph provided in an embodiment of the present application;

[0048] Figure 1d A schematic diagram of the execution order of operators in a computation graph provided in an embodiment of the present application;

[0049] Figure 2a A schematic diagram of a computation graph of a neural network and the execution order of each operator in the computation graph provided in an embodiment of the present application;

[0050] Figure 2b A schematic diagram of allocating memory space for tensor data provided in an embodiment of the present application;

[0051] Figure 2cA schematic diagram of a computation graph of a neural network in a parallel scenario and the execution order of each operator in the computation graph provided in an embodiment of the present application;

[0052] Figure 3a A schematic diagram of the structure of a memory allocation device provided in an embodiment of the present application;

[0053] Figure 3b A schematic diagram of the server or terminal device side architecture provided in an embodiment of the present application;

[0054] Figure 3c A schematic diagram of a network architecture provided in an embodiment of the present application;

[0055] Figure 3d A schematic diagram of a directed acyclic graph (DAG) provided in an embodiment of the present application;

[0056] Figure 4a A flowchart of a memory allocation method provided in an embodiment of the present application;

[0057] Figure 4b A schematic diagram of the structure of a convolutional neural network 400 provided in an embodiment of the present application;

[0058] Figure 4c A schematic diagram of determining a sorting result provided in an embodiment of the present application;

[0059] Figure 4d A schematic diagram of memory space in an allocated set provided in an embodiment of the present application;

[0060] Figure 4e A schematic diagram of an updated computation graph provided in an embodiment of the present application;

[0061] Figure 4f A schematic diagram of memory space in an allocated set provided in an embodiment of the present application;

[0062] Figure 4g A schematic diagram of memory space in an allocated set provided in an embodiment of the present application;

[0063] Figure 5 A flowchart of another memory allocation method provided in an embodiment of the present application;

[0064] Figure 6 A schematic diagram of the structure of a memory allocation device 60 provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of the structure of a memory allocation device 70 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present application are clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0067] The terms "first" and "second" in the description and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects. In addition, the terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. It should be noted that in the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design method described in the embodiments of this application as "exemplarily" or "for example" should not be construed as being superior or more advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete way. In the embodiments of this application, "A and / or B" means both A and B, or A or B. “A, and / or B, and / or C” means any one of A, B, and C, or any two of A, B, and C, or A, B, and C.

[0068] In order to better understand the technical solutions described in this application, the following first explains the relevant technical terms involved in the embodiments of this application:

[0069] (1) Neural Network

[0070] A neural network can be composed of neural units, which can be represented by x s and intercept b as inputs, the output of the operation unit can be:

[0071]

[0072] Where, s = 1, 2, ... n, n is a natural number greater than 1, w s is x sThe weight of the neural unit, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer. The activation function can be a sigmoid function. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0073] (2) Deep Neural Networks

[0074] A deep neural network (DNN), also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here. Based on the position of different layers in DNN, the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers can be fully connected or not. When the layers are fully connected, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: in, is the input vector, is the output vector, b is the offset vector, w is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector. After such a simple operation, the output vector Since there are many DNN layers, the number of coefficients w and offset vectors b is also large. The definitions of these parameters in DNN are as follows: Take the coefficient w as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

[0075] In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as It's important to note that the input layer has no w parameter. In deep neural networks, more hidden layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors w across many layers).

[0076] (3) Computational graph

[0077] In this application, a computational graph is a way of describing the computational process of a neural network using a graph structure. If the computation has obvious modularity and there are obvious temporal and logical dependencies between the modules, a directed graph structure can usually be used to describe it. In practical applications, there are two basic elements of a graph structure, namely nodes and directed edges. A neural network model can be abstracted as a directed graph structure consisting of tensor data and operators. Nodes are also called operators. As the name implies, directed edges refer to edges with directions, which describe the pointing direction between operators and are used to characterize the dependency relationship between operators. In this application, for the convenience of explanation, nodes are used as an example for illustration.

[0078] like Figure 1a As shown, the computation graph contains 8 operators and 9 tensor data. Specifically, the 8 operators are operator a, operator b, operator c, operator d, operator e, operator f, operator g, and operator h, and the 9 tensor data are tensor data t0, tensor data t1, tensor data t2, tensor data t3, tensor data t4, tensor data t5, tensor data t6, tensor data t7, and tensor data t8. Taking operators a, b, and c as examples, for operator a, there is a directed edge between operator a and operator b. The direction of this directed edge is from operator a to operator b. This directed edge indicates that the tensor data t0 generated by operator a is the input of operator b. At the same time, there is a directed edge between operator a and operator c. The direction of this directed edge is from operator a to operator c. This directed edge indicates that the tensor data t0 generated by operator a is the input of operator b.

[0079] Generally speaking, describing the neural network model using a computational graph is conducive to an overall grasp of the entire neural network computing task. At the same time, the expression of the computational graph also facilitates the scheduling and parallel execution of computing tasks.

[0080] In this application, operators in a computation graph can be assigned to multiple computation subtasks, and different computation subtasks can be run in parallel or in series. The order in which operators in the same computation subtask run is serial. For example, Figure 1bAs shown, operators a and c are assigned to computation subtask 0, operators b, d, e, and f are assigned to computation subtask 1, and operators g and h are assigned to computation subtask 2. Regarding how to determine the number of computation subtasks in a computation graph, please refer to existing implementations and do not impose any restrictions here. For example, a neural network computation task can be split into multiple computation subtasks, thereby generating multiple computation subgraphs, with each computation subgraph representing a computation subtask.

[0081] In some embodiments, when computing subtask 0, computing subtask 1, and computing subtask 2 are in parallel, the execution order of operators between different computing subtasks depends on the direction of the directed edges between operators. For example, within computing subtask 0, the execution order between operators is operator a-operator c; within computing subtask 1, the execution order between operators is operator b-operator d / operator e-operator f; within computing subtask 2, the execution order between operators is operator g-operator h. Figure 1b In the example, operator c and operator g are executed in parallel in two computing subtasks. Since the tensor data t4 generated by operator c is the input of operator g, operator g will not be executed until operator c is finished. Operator e and operator g are executed in parallel in two computing subtasks. For operator e, the tensor data t6 generated by operator e is not the input of operator g; for operator g, the tensor data t8 generated by operator g is not the input of operator e. That is, there is no generation and consumption of tensor data between operator e and operator g. Therefore, the execution order between operator e and operator g is not limited. Specifically, the execution order of each operator in the computational graph can be as follows: Figure 1c It should be noted that Figure 1c In the example, the execution order between operator d and operator e is not limited. Based on this, they are expressed as operator d / operator e.

[0082] In some embodiments, when computing subtask 0 and computing subtask 1 are in a parallel state, and computing subtask 1 and computing subtask 2 are in a serial state, the execution order of operators between different computing subtasks depends on the direction of the directed edges between the operators. For example, within computing subtask 0, the execution order between operators is operator a-operator c; within computing subtask 1, the execution order between operators is operator b-operator d / operator e-operator f; within computing subtask 2, the execution order between operators is operator g-operator h. For computing subtask 1 and computing subtask 2, operator g in computing subtask 2 will not be executed until operator f in computing subtask 1 has finished running. Specifically, the execution order of each operator in the computational graph can be as follows: Figure 1d shown.

[0083] (4) Dependencies

[0084] In this application, operator A depends on operator B, which means that operator A must wait for the kernel function corresponding to operator B to be executed before starting its own computing task.

[0085] (5) Tensor

[0086] In this application, a tensor is merely a feature description of a piece of stored data. A tensor records information such as the shape and type of the data.

[0087] In this application, tensor should be understood as tensor data, which may include input tensor data and output tensor data in a neural network model, as well as feature tensor data, etc.

[0088] Taking the artificial intelligence deep learning framework TensorFlow as an example, rank, shape, and dimension number are generally used to describe the dimensions of a tensor. Their relationship can be expressed as shown in the following table:

[0089] Step shape Dimension Examples 0 [] 0-D 4 1 [D1] 1-D [2] 2 [D1,D2] 2-D [6,2] 3 [D1,D2,D3] 3-D [7,3,2] … … … n [D1,D2,D3,…Dn] nD A tensor of shape [D1, D2, D3, ... Dn]

[0090] As shown in the above table, the tensor A=4, which represents a number.

[0091] As shown in the table above, the tensor A = [6, 2] represents a two-dimensional matrix. Specifically, the matrix is ​​a matrix with 6 rows and 2 columns.

[0092] (6) Allocated Set

[0093] In this application, the allocated set refers to the set of memory space information that has been allocated in the process of storing tensor data of the neural network. The above-mentioned allocated set can also be called a shared memory queue, which is not specifically limited in this application.

[0094] (7) Memory allocation strategy for neural networks

[0095] The first strategy is called the In-Place strategy, which means that the input and output of each node in the neural network share a memory space.

[0096] The second strategy is called the Co-share strategy. This strategy states that a block of memory space can be used by multiple nodes in the neural network. When these nodes have completed execution, the lifecycle of the memory space ends and the memory space can be used by other nodes in the neural network. For example, the lifecycle of memory space A can be preset to (1,2,3), indicating that memory space A can be used by nodes 1, 2, and 3. When nodes 1, 2, and 3 have all completed execution, the lifecycle of memory space A ends and memory space A can be placed in the free list for use by other nodes in the neural network.

[0097] At present, for the second strategy mentioned above, the specific method of memory allocation is: allocating and reusing memory space according to the order of node execution in the neural network, and the memory allocation effect is poor.

[0098] For example, during the operation of a neural network, it needs to occupy 100M of memory space, 10M of memory space, and 50M of memory space in sequence. When the neural network requests 100M of memory space, a 100M memory space can be allocated for the neural network. Then, when the neural network requests 10M of memory space, it is determined whether the allocated 10M of memory space can be reused. If it can be reused, no new memory space is allocated for the requested 10M of memory space, but the 100M of memory space is reused. Similarly, when the neural network requests 50M of memory space, it is first determined whether the 50M of memory space can reuse the allocated 100M of memory space. If it can be reused, no new memory space is allocated for the requested 50M of memory space. However, if the requested 10M memory space and the requested 50M memory space can both reuse the allocated 100M memory space, the requested 10M memory space will reuse the allocated 100M memory space, and an additional 50M memory space will be allocated to the neural network, so that the entire neural network needs to occupy 150M of memory space, resulting in a large amount of memory occupied by the entire neural network and unreasonable memory allocation.

[0099] In response to the above, the present application provides a memory allocation method, the main principle of which is: according to the information of each tensor data, the sorting results of multiple tensor data in the entire neural network are obtained, wherein the information of each tensor data may include the size of the memory space required for each tensor data, the constraint relationship corresponding to each tensor data, and at least one of the consumption operators. In addition, for each tensor data, each corresponding identifier is included, and then, memory space is allocated to the tensor data in sequence according to the sorting results. The above method can avoid the unreasonable memory planning space situation that occurs in the above-mentioned prior art, thereby saving the memory required for the entire neural network and optimizing the memory allocation of the neural network. In addition, this method can also solve the problem of operator calculation results errors caused by operators reusing the same memory space in different computing subtasks in parallel scenarios.

[0100] For example, Figure 2a As shown, the entire neural network consists of 8 nodes, which are indexed from a to h in the order of operation. Figure 1aWhen the neural network shown is running, it needs to occupy 5 memory spaces in succession according to the execution order of the operators {a, b, c, d, e, f, g, h}. Each memory space is used for tensor data, namely the first memory space, the second memory space, the third memory space, the fourth memory space, and the fifth memory space.

[0101] Specifically, the implementation process of pre-allocating memory for tensor data can be as follows Figure 2b As shown, before simulating the execution of the operation logic of operator a, the first memory space is allocated to tensor data t0, and the second memory space is allocated to tensor data t1. Before simulating the execution of the operation logic of operator b, the third memory space is allocated to tensor data t2, and the fourth memory space is allocated to tensor data t3. Before simulating the execution of the operation logic of operator c, the fifth memory space is allocated to tensor data t4. Simultaneously, the first memory space is released, and the first memory space can be reused for the following tensor data. Before simulating the execution of the operation logic of operator d, the first memory space, which can be reused, is allocated to tensor data t5. Simultaneously, the third memory space is released, and the third memory space can be reused for the following tensor data. Before simulating the execution of the operation logic of operator e, the third memory space, which can be reused, is allocated to tensor data t6. Simultaneously, the fourth memory space is released, and the fourth memory space can be reused for the following tensor data. Before simulating the execution of the operation logic of operator f, the fourth memory space, which can be reused, is allocated to tensor data t7. At the same time, the first and third memory spaces are released, allowing them to be reused for subsequent tensor data. Before simulating the execution of the operator g's logic, the reusable first memory space is allocated to tensor data t8. At the same time, the fifth memory space is released. Before simulating the execution of the operator h's logic, the second, fourth, and first memory spaces are released. At the same time, the neural network's computational results are stored in the designated memory space.

[0102] like Figure 2b As shown in FIG, the size of the memory planning space determined for the entire neural network is the sum of the sizes of the above five memory spaces.

[0103] exist Figure 1b In the computational graph shown in the figure, the computational graph includes three computational subtasks, each of which represents a different neural network computational subtask. Generally speaking, the relationship between these three computational subtasks can be serial or parallel. Within the same computational subtask, the execution order between operators is serial. Figure 2cAs shown, when the relationship between computing subtask 0 and computing subtask 1 is parallel (for example, the two computing subtasks can be run through different processor cores), since the execution order between operator c and operator d is a parallel relationship, if the operation of operator d is executed, the operation of operator c is not completed. In this case, if tensor data t0 and tensor data t5 reuse the same memory space (the first memory space), the tensor data t0 generated by operator a will be overwritten, resulting in an error in the operation result of operator c. It can be understood that in the prior art, when there are parallel computing subtasks in the computation graph, Figure 2b The memory planning shown is unreasonable. This is reflected in the following: in two parallel computing subtasks, operators in different computing subtasks reuse the same memory space, which may cause the calculation result of one of the operators to be incorrect.

[0104] In response to the above, this application provides another memory allocation method. The main principle of this method is: in a parallel scenario, determine the constraint relationship corresponding to each tensor data in the calculation graph, and then allocate memory space for the tensor data based on the constraint relationship corresponding to each tensor data. The above method can avoid the unreasonable memory planning space that occurs in the above-mentioned prior art, avoid the situation where the operator calculation results are incorrect due to the reuse of the same memory space by operators in different computing subtasks, and ensure the accuracy of the calculation results of the neural network.

[0105] To facilitate a better understanding of this application, the following introduces several application scenarios in which the method described in this application can be applied:

[0106] like Figure 3a As shown, the method described in this application can be applied to both online training / inference of neural networks and offline training / inference of neural networks. Specifically, in the scenario of online training / inference of neural networks, the processor CPU communicates with the artificial intelligence processor through the I / O bus to allocate memory for the running neural network. In the scenario of offline / inference of neural networks, the general-purpose processor obtains the offline neural network file stored in the hard disk and allocates memory for it when the offline neural network file is called.

[0107] In this application, the memory allocation device can be specifically a server or a terminal device. Figure 3b As shown, the server or terminal device side may include deep learning algorithms, deep learning frameworks, computing resources, and memory resources, etc. Among them, the deep learning algorithm can call computing resources and memory resources through the deep learning framework.

[0108] Taking the Convolution Architecture For Fast Featureembedding (Caffe) as an example, Caffe can support various types of deep learning frameworks, image classification and image segmentation, as well as Convolutional Neural Networks (CNN), Region-CNN (RCNN) for target detection, Long Short-Term Memory (LSTM) and fully connected neural network designs. Figure 3c As shown, the deep learning algorithm may include a network model, and the deep learning framework may include NET classes, layers, Blobs, task management, and memory management modules (syncmem). Among them, a MemModel module may be set in the memory management module, and memory optimization may be achieved based on the original logic of the blob and memory management modules.

[0109] In the embodiment of the present application, the network model can be specifically a neural network model. The NET class can store a directed acyclic graph (DAG) corresponding to the neural network, for example, Figure 3d As shown, an example of DAG is provided. Figure 3d In the example, the neural network includes five nodes A, B, C, E, F and G. The output parameter of node A (for example, tensor data) is used as the input parameter of node B, the output parameter of node B is used as the input parameter of node C and node F respectively, the output parameter of node C is used as the input parameter of node E, and the output parameters of nodes E and node F are used as the input parameter of node G. Layer is used to store information about the nodes included in the neural network. Nodes can also be called layers. Blob is used to store information about the memory space occupied by the input parameters, output parameters and intermediate parameters corresponding to each node of the neural network during the calculation process. The memory management module is used to manage and allocate information about the memory space occupied by the neural network.

[0110] See Figure 4a , is a flow chart of a memory allocation method for a neural network provided by an embodiment of the present application. The execution subject of the method can be a server running a neural network, or a terminal device running a neural network. For the convenience of explanation, the execution subject is taken as an example of a terminal device running a neural network. Figure 4a In the flowchart of the method shown in FIG, it is possible to set the entire neural network to require multiple memory spaces during operation. Figure 4a As shown, the method may include but is not limited to the following steps:

[0111] Step S401: Obtain a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1.

[0112] In the embodiment of the present application, a node is used to indicate a computational logic in a neural network, that is, a function that implements a specific function. In practical applications, OP can be used to represent a node, and tensor can be used to represent tensor data.

[0113] For example, taking the convolutional neural network as an example, the specific structure of the convolutional neural network can be as follows: Figure 4b As shown, the convolutional neural network (CNN) 400 may include an input layer 410, a convolutional layer / pooling layer 420 (where the pooling layer is optional), a fully connected layer 430, and an output layer 440. Here, the fully connected layer 430 refers to a network structure with fully connected characteristics. Taking hidden layer 1 as an example, the fully connected characteristic can be represented by the product of the input data of hidden layer 1 and the weight tensor corresponding to hidden layer 1. For example, the fully connected characteristic can be quantized as ωx, where ω represents the weight tensor corresponding to hidden layer 1 and x represents the input data of hidden layer 1. Specifically, the convolutional layer 420 is used to extract features of the input data. For example, when the input data is an image, the convolutional layer 420 is used to extract features of the input image to reduce the parameters brought by the input image. The fully connected layer 430 is used to integrate local information with category discrimination in the convolutional layer 420 (or pooling layer). For example, the fully connected layer 430 can connect the features extracted by the convolutional layer 420. In practical applications, to improve the performance of convolutional neural network 400, the activation function for each neuron in fully connected layer 430 is typically a ReLU function. The output value of the final fully connected layer 430 is passed to an output, for example, where softmax regression can be used for classification, thereby obtaining a processing result. For example, the processing result can be the recognition probability of an image, which can then be output via output layer 440.

[0114] The terminal device can obtain the computational graph corresponding to the above-mentioned convolutional neural network, which includes convolution nodes, fully connected nodes (FC), activation nodes (Relu), pooling nodes (Pooling), classifier nodes (softmax), etc.

[0115] In an embodiment of the present application, a directed edge can be used to represent the connection relationship between nodes. The directed edge carries tensor data, and the direction of the directed edge is used to reflect the flow direction of the tensor data.

[0116] Step S402: Based on the sorting results of the M tensor data, memory space is allocated to the M tensor data in sequence, wherein, if one tensor data among the M tensor data can reuse at least a part of the allocated memory space, then at least a part of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data.

[0117] In an embodiment of the present application, the sorting result of M tensor data indicates the execution order when allocating memory space for the M tensor data. The sorting result is related to the information of each tensor data in the M tensor data, and the information of each tensor data indicates at least one of the following information: the constraint relationship corresponding to each tensor data and the number of nodes to which each tensor data flows.

[0118] In this application, tensor data may include input tensor data, output tensor data, and intermediate tensor data.

[0119] In this application, a consumption node refers to a node in a computation graph that consumes tensor data, i.e., a node to which tensor data flows. As the name implies, "consumption" refers to the use and consumption of material (e.g., tensor data) during the computation of a node.

[0120] In this application, a production node refers to a node in a computational graph that generates tensor data, i.e., a node from which tensor data flows out. As the name implies, "production" is the inverse of "consumption," representing the output of a node's computation.

[0121] In this application, node A is the upstream node of node B, which means that there is at least one path in the computation graph that can go from node A to node B. For example, in the computation graph, you can use node B as the starting point and obtain the upstream node corresponding to node B by traversing in the reverse direction (i.e., along the opposite direction of the directed edge) to obtain the upstream node of node B.

[0122] In the present application, the constraint relationship can be carried in a constraint relationship table, in which the first value can be used to indicate that each tensor data can reuse the same memory space with other tensor data, the second value can be used to indicate that each tensor data cannot reuse the same memory space with other tensor data, and the third value can be used to indicate that each tensor data can be continuously stored in the same memory space with other tensor data.

[0123] Specifically, the first value, the second value, and the third value may be values ​​that can be distinguished from each other. For example, the first value may be “0”, the second value may be “1”, and the third value may be “2”.

[0124] In some embodiments, the above constraints have different priorities. In other words, if the relationship between the available memory spaces of two tensors is non-reusable and continuous, then they must also be non-reusable. In this case, the relationship between the available memory spaces of the two tensors is indicated as non-reusable and continuous in the constraint relationship. In other words, it can be understood that non-reusable and continuous has a higher priority than non-reusable.

[0125] It should be noted that, in some embodiments, the constraint relationship is not limited to the expression form of the constraint relationship table, and can also be presented through other data structures.

[0126] In some embodiments, the implementation process of determining the constraint relationship corresponding to each tensor data in the computational graph may include: determining whether all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data; if so, determining that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, determining that the first tensor data cannot be reused as the memory space allocated for the second tensor data.

[0127] In some embodiments, the implementation process of determining the constraint relationship corresponding to each tensor data in the computational graph may include: determining whether all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data; if so, determining that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, determining that the first tensor data cannot be reused as the memory space allocated for the second tensor data.

[0128] In this application, tensor data A can reuse at least a portion of the memory space allocated for tensor data B, which means that tensor data A can completely reuse the memory space allocated for tensor data B, or that tensor data A can reuse a portion of the memory space allocated for tensor data B.

[0129] In this application, node A is the upstream node of node B, which means that there is at least one path in the computation graph that can go from node A to node B. For example, in the computation graph, you can use node B as the starting point and obtain the upstream node corresponding to node B by traversing in the reverse direction (i.e., along the opposite direction of the directed edge) to obtain the upstream node of node B.

[0130] In some embodiments, the implementation process of determining the size of the memory space required to be occupied by each tensor data in the computation graph may include: the terminal device runs the neural network, records the size of the memory space required to be occupied by each tensor data in the neural network, and determines the size of the memory space required to be occupied by each tensor data when the neural network is in operation based on the recorded size of the memory space required to be occupied by each tensor data, thereby providing a basis for subsequently allocating corresponding memory space to the tensor data. For example, the entire neural network includes node 1 and node 2, and the terminal device runs the neural network through an artificial intelligence processor. It can be recorded that during the operation of the neural network, the size of the memory space required to be occupied by tensor data 1 is 1000Kb, and the size of the memory space required to be occupied by tensor data 2 is 500Kb. Therefore, the size of the memory space required to be occupied by each tensor data when the neural network is in operation can be determined based on the recorded size of the memory space required to be occupied by each tensor data.

[0131] In addition, it should be noted that in this application, it is assumed that each tensor data has its own corresponding identifier. Figure 1b Taking the computation graph shown in FIG as an example, the computation graph includes 8 operators and 9 tensor data, wherein the 9 tensor data can be represented as tensor data t0, tensor data t1, tensor data t2, tensor data t3, tensor data t4, tensor data t5, tensor data t6, tensor data t7 and tensor data t8. It can be understood that the identifier corresponding to each tensor data is unique.

[0132] In some embodiments, the identifiers may be a series of sequential numbers, so that the order of the tensor data may be determined based on the identifiers corresponding to each tensor data.

[0133] In some embodiments, the constraint amount corresponding to each of the M tensor data can be obtained based on the constraint relationship corresponding to each tensor data. The constraint amount is the number of tensor data in other tensor data that cannot reuse the same memory space with the tensor data; then, according to the size of the constraint amount corresponding to each of the M tensor data, the M tensor data are sorted from large to small to obtain the sorting result of the M tensor data.

[0134] In some embodiments, the M tensor data may be sorted from large to small according to the number of consumer nodes corresponding to each of the M tensor data to obtain a sorting result of the M tensor data.

[0135] It is understandable that the M tensor data can also be sorted from large to small based on at least two of the information of each tensor data to obtain the sorting results of the M tensor data. For example, taking the example where the information of each tensor data includes the size of the memory space required for each tensor data and the constraint relationship corresponding to each tensor data, the calculation graph includes 2 tensor data, namely tensor data 1 and tensor data 2, wherein the memory space required for tensor data 1 is 1000Kb, and the constraint relationship between tensor data 1 and tensor data 2 is that tensor data 1 cannot reuse the same memory space with tensor data 2, and its constraint amount is 1; the memory space required for tensor data 1 is 500Kb, and the constraint relationship between tensor data 2 and tensor data 1 is that tensor data 2 cannot reuse the same memory space with tensor data 1, and its constraint amount is 1. The above two tensor data are sorted from large to small, and the sorting results obtained are: tensor data 1, tensor data 2.

[0136] In some embodiments, the M tensor data can be sorted using a heuristic algorithm to obtain a sorted result for the M tensor data within a preset time period. Here, a heuristic algorithm refers to an algorithm constructed based on intuition or experience that provides a feasible solution to each instance of the combinatorial optimization problem to be solved at an acceptable cost (referring to computing time and space). The degree of deviation of the feasible solution from the optimal solution is generally unpredictable.

[0137] As mentioned above, the information of each tensor data includes the identifier corresponding to each tensor data, and may also include one or more of the following: the size of the memory space required for each tensor data, the constraint relationship corresponding to each tensor data, and the number of consumer nodes to which each tensor data flows. When the terminal device sorts the M tensor data using a heuristic algorithm, it is necessary to consider the order between the information contained in each tensor data, and then sort it as an independent individual in this order. For example, when the information of each tensor data includes the identifier corresponding to each tensor data, the size of the memory space required for each tensor data, the constraint relationship corresponding to each tensor data, and the number of consumer nodes to which each tensor data flows, the mixed sorting results between these four information may include 632 types.

[0138] For example, a computation graph includes four tensor data items, namely tensor data 1, tensor data 2, tensor data 3, tensor data 4, and tensor data 5. The terminal device sorts the five tensor data items using the above heuristic algorithm and obtains a sorting sequence determined by the heuristic algorithm within a preset time period (for example, the sorting sequence is: tensor data 2, tensor data 3, tensor data 4, tensor data 1, and tensor data 5) as the sorting result of the five tensor data items. Thus, the terminal device can allocate memory space for the tensor data items based on the determined sorting result.

[0139] In some embodiments, the memory allocation device may use a constraint programming solver (CPsolver) to call a heuristic algorithm (e.g., a heuristic algorithm includes a deterministic algorithm and a random algorithm) to sort the M tensor data. It should be noted that the above sorting result may be a sorting result that requires optimization or a sorting result that does not require optimization.

[0140] In some embodiments, in order to save memory, the above sorting results are optimized sorting results, wherein the maximum memory size required for the neural network corresponding to the optimized sorting results is less than the maximum memory size required for the neural network determined based on the sorting results before optimization. For example, the computational graph includes 4 tensor data, namely tensor data 1, tensor data 2, tensor data 3, tensor data 4 and tensor data 5, wherein the arrangement order of tensor data 1, tensor data 2, tensor data 3 and tensor data 4 is: tensor data 1, tensor data 2, tensor data 3 and tensor data 4. In this sorting result, if Figure 4c As shown, there are four possible positions of tensor data 5 (possible position 1, possible position 2, possible position 3 and possible position 4), among which possible position 1 is the position between tensor data 1 and tensor data 2; possible position 2 is the position between tensor data 2 and tensor data 3; possible position 3 is the position between tensor data 3 and tensor data 4; possible position 4 is the position after tensor data 4. Figure 4cAs shown, the above four potential possible positions each correspond to a different memory space. Based on the above potential possible positions, the memory allocation device determines the position of the tensor data 5 according to different judgment conditions, wherein the judgment condition may include but is not limited to: for a certain tensor data, the first address corresponding to the memory space allocated for the tensor data is the smallest or largest; for a certain tensor data, it is determined that the difference between the size of the memory space corresponding to the potential possible position and the size of the memory space required to be occupied by the tensor data meets the threshold, for example, the threshold can be 0 or other values. When the threshold is 0, it means that the size of the memory space corresponding to the potential possible position is equal to the size of the memory space required to be occupied by the tensor data. When the position of tensor data 5 in the sorting result is possible position 1, the terminal device allocates memory space for the tensor data according to the sorting result, for example, the maximum memory size required to run the entire neural network is determined to be 4500Kb through the allocated memory space; when the position of tensor data 5 in the sorting result is possible position 2, the terminal device allocates memory space for the tensor data according to the sorting result, for example, the maximum memory size required to run the entire neural network is determined to be 3500Kb through the allocated memory space; when the position of tensor data 5 in the sorting result is possible position 3, the terminal device allocates memory space for the tensor data according to the sorting result, for example, the maximum memory size required to run the entire neural network is determined to be 5000Kb through the allocated memory space; when the position of tensor data 5 in the sorting result is possible position 4, the terminal device allocates memory space for the tensor data according to the sorting result, for example, the maximum memory size required to run the entire neural network is determined to be 4000Kb through the allocated memory space.

[0141] The following example illustrates how to allocate memory space for tensor data:

[0142] In some embodiments, Figure 1b Taking the computation graph shown in the figure as an example, computation subtask 0 contains nodes a and c, and its execution order is node a - node c; computation subtask 1 contains nodes b, d, e, and f, and its execution order is node b - node d / e - node f; computation subtask 2 contains nodes g and h, and its execution order is node g - node h. The execution relationship between computation subtasks 0, 1, and 2 is parallel. Since there are directed edges between adjacent nodes in each computation subtask, no adjustments to the computation graph are required.

[0143] Secondly, in the computational graph, determine the upstream node corresponding to each node, the output tensor data corresponding to each node, and the input tensor data corresponding to each node. Taking node A as the upstream node of node B as an example, it means that in the computational graph, there is at least one path from node A to node B. Specifically, the upstream node corresponding to each node, the output tensor data corresponding to each node, and the input tensor data can be shown in Table 1:

[0144] node Upstream node Output tensor data Input tensor data a - t0,t1 - b a t2,t3 t0 c a t4 t0 d a,b t5 t2 e a,b t6 t3 f a,b,c,d,e t7 t5,t6,t4 g a,c t8 t4 h a,b,c,d,e,f,g - t1,t7,t8

[0145] Table 1

[0146] As shown in Table 1, Figure 1b In the computational graph shown, taking node a as an example, node a is the starting node and has no corresponding upstream node. During the computation, node a can obtain output tensor data t0 and output tensor data t1. For another example, taking node b as an example, node a is the upstream node of node b, which means that there is a path from node a to node b in the computational graph. During the computation of node b, its input tensor data is t0, and it can obtain output tensor data t2 and output tensor data t3. The implementation process of determining the upstream nodes corresponding to other nodes, output tensor data, and input tensor data will not be elaborated here.

[0147] Afterwards, determine the constraint relationship corresponding to each tensor data. For example, it can be determined whether all the consumption nodes of the first tensor data are upstream nodes of the production node of the second tensor data. If so, it is determined that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, it is determined that the first tensor data cannot be reused as the memory space allocated for the second tensor data. For another example, it can be determined whether all the consumption nodes of the second tensor data are downstream nodes of the production node of the first tensor data. If so, it is determined that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, it is determined that the first tensor data cannot be reused as the memory space allocated for the second tensor data.

[0148] Specifically, the above-mentioned constraint relationship can be carried in a constraint relationship table, in which the first value can be used to indicate that each tensor data can reuse the same memory space with other tensor data, the second value can be used to indicate that each tensor data cannot reuse the same memory space with other tensor data, and the third value can be used to indicate that each tensor data can be stored continuously in the same memory space with other tensor data. For the convenience of explanation, the first value "0" is used to indicate that tensor data can reuse the same memory space with other tensor data other than itself; the second value "1" is used to indicate that tensor data cannot reuse the same memory space with other tensor data other than itself; and the third value "2" is used to indicate that tensor data can be stored continuously in the same memory space with other tensor data other than itself. It should be noted that the above description is only an example and should not constitute a limitation. Specifically, the constraint relationship table can be expressed as shown in Table 2:

[0149] t0 t1 t2 t3 t4 t5 t6 t7 t8 t0 - 1 1 1 1 1 1 1 1 t1 1 - 1 1 1 1 1 1 1 t2 1 1 - 1 1 1 0 0 1 t3 1 1 0 - 1 0 1 0 1 t4 1 1 1 1 - 1 1 1 1 t5 1 1 1 0 1 - 0 1 1 t6 1 1 0 1 1 0 - 1 1 t7 1 1 0 0 1 1 1 2 - t8 1 2 1 1 1 1 1 - 1

[0150] Table 2

[0151] It should be noted that, when considering whether to reuse memory space, the constraint relationship between two tensor data is symmetrical. This symmetry is reflected in the fact that the relationship is exactly the same as its inverse relationship. As shown in Table 2, taking the constraint relationship between tensor data t2 and tensor data t8 as an example, Figure 1b In the calculation graph shown, the production node of tensor data t7 is f, and the consumption node of tensor data t2 is d. Since the consumption node d of tensor data t2 is the upstream node of the production node f of tensor data t7, it can be determined that tensor data t7 can reuse the same memory space with tensor data t2. Figure 1b In the computational graph shown, within computation subtask 1, node b is a control selection node with two branches: one branch: node b - node d - node f; the other branch: node b - node e - node f. In a single neural network operation, only one branch is valid. For tensor data t2 and t3, the constraint relationship between them is that they do not require two separate memory spaces; in other words, they can reuse the same memory space. For tensor data t5 and t6, the constraint relationship between them is that they do not require two separate memory spaces; in other words, they can reuse the same memory space.

[0152] It should also be noted that when considering storing multiple tensor data in the same continuous memory space, the constraints between each pair of tensor data are asymmetric. In this case, the orderliness between each pair of memory spaces needs to be considered.

[0153] Then, taking the information of each tensor data including the memory space required by each tensor data and the constraint relationship corresponding to each tensor data as an example, the M tensor data are sorted from largest to smallest based on the memory space required by each tensor data and the size of the constraint amount corresponding to each tensor data, thereby obtaining a sorting result for the M tensor data. During the sorting, multiple tensor data whose constraints require continuous storage in the same memory space can be sorted as an independent entity.

[0154] Specifically, based on the constraint relationship shown in Table 2, the constraint amount corresponding to each tensor data can be obtained. For example, for tensor data t0, tensor data t0 cannot reuse the same memory space with other tensor data (t1, t2, t3, t4, t5, t6, t7, t8), and its constraint amount is 8; for tensor data t1, tensor data t1 cannot reuse the same memory space with other tensor data (t0, t2, t3, t4, t5, t6, t7, t8), and its constraint amount is 8; for tensor data t2, tensor data t2 cannot reuse the same memory space with tensor data (t0, t1, t3, t4, t5, t8), and its constraint amount is 6; for tensor data t3, tensor data t3 cannot reuse the same memory space with tensor data (t0, t1, t4, t6, t8). The memory space is limited to 5. For tensor data t4, tensor data t4 cannot be reused with tensor data (t0, t1, t2, t3, t5, t6, t7, t8), and the constraint amount is 8. For tensor data t5, tensor data t5 cannot reuse the same memory space with tensor data (t0, t1, t2, t4, t7, t8), and the constraint amount is 6. For tensor data t6, tensor data t4 cannot be reused with tensor data (t0, t1, t2, t4, t7, t8), and the constraint amount is 8. Data t6 cannot be combined with tensor data (t0, t1, t3, t4, t7, t8), and its constraint amount is 6; for tensor data t7, tensor data t7 cannot be combined with tensor data (t0, t1, t4, t5, t6), and its constraint amount is 5; for tensor data t8, tensor data t8 cannot be combined with tensor data (t0, t2, t3, t4, t5, t6, t7), and its constraint amount is 7.

[0155] Furthermore, the memory space required for tensor data t0 is 500Kb; the memory space required for tensor data t1 is 500Kb; the memory space required for tensor data t2 is 500Kb; the memory space required for tensor data t3 is 500Kb; the memory space required for tensor data t4 is 500Kb; the memory space required for tensor data t5 is 1000Kb; the memory space required for tensor data t6 is 1000Kb; the memory space required for tensor data t7 is 1000Kb; and the memory space required for tensor data t8 is 1000Kb.

[0156] Therefore, the above 9 tensor data can be sorted from large to small, and the sorting results can be shown in Table 3:

[0157] t1,t8,t7 t5 t6 t0 t4 t2 t3

[0158] Table 3

[0159] It should be noted that, when sorting, the method of sorting multiple tensor data items that are constrained to be stored contiguously in the same memory space as a single entity is merely an example and should not be construed as limiting. In practical applications, each tensor data item can also be sorted as a single entity.

[0160] Then, in this case, first, allocate the first memory space to the tensor data t1, t7 and t8. At this time, the allocated memory space includes the first memory space; wherein, the first memory space includes memory space a, second memory space b and third memory space c, memory space a, memory space b and memory space c are continuous memory spaces, wherein, memory space a is used to store tensor data t1 (that is: the size of memory space a is equal to the size of tensor data t1), memory space b is used to store tensor data t7 (that is: the size of memory space b is equal to the size of tensor data t7), and memory space c is used to store Storing tensor data t8 (that is, the size of memory space c is equal to the size of tensor data t8); thereafter, allocating memory space for tensor data t5, the implementation process may include: judging whether tensor data t5 can reuse the allocated memory space (first memory space) in combination with the constraint relationship between tensor data t5 and other tensor data in Table 2, since tensor data t5 cannot reuse the allocated first memory space, at this time, allocating a second memory space of corresponding size to tensor data t5 according to the memory space size required by tensor data t5, at this time, the allocated memory space includes the first memory space and the second memory space. Thereafter, allocating memory space for tensor data t6, the implementation process may include: judging whether tensor data t6 can reuse the allocated memory space (first memory space and second memory space) in combination with the constraint relationship between tensor data t6 and other tensor data in Table 2, since tensor data t6 can reuse the second memory space, allocating the second memory space to tensor data t6. Afterwards, memory space is allocated for the tensor data t0. The implementation process may include: combining the constraint relationship between the tensor data t0 and other tensor data in Table 2 to determine whether the tensor data t0 can reuse the allocated memory space (the first memory space and the second memory space). Since the tensor data t0 cannot reuse the allocated memory space, at this time, a third memory space of corresponding size is allocated to the tensor data t0 according to the size of the memory space required to be occupied by the tensor data t0. In this case, the allocated memory space includes the first memory space, the second memory space and the third memory space. Afterwards, memory space is allocated for tensor data t4. The implementation process may include: combining the constraint relationship between tensor data t4 and other tensor data in Table 2 to determine whether tensor data t4 can reuse the allocated memory space (first memory space, second memory space and third memory space). Since tensor data t4 cannot reuse the allocated memory space, at this time, a fourth memory space of corresponding size is allocated to tensor data t4 according to the size of the memory space required to be occupied by tensor data t4. In this case, the allocated memory space includes the first memory space, the second memory space, the third memory space and the fourth memory space.Afterwards, memory space is allocated for tensor data t2. The implementation process may include: judging whether tensor data t2 can reuse the allocated memory space (first memory space, second memory space, third memory space and fourth memory space) in combination with the constraint relationship between tensor data t2 and other tensor data in Table 1. Since tensor data t2 can reuse the first memory space, the first memory space is allocated to tensor data t2 (for example, memory space c in the first memory space can be allocated to tensor data t2). Afterwards, memory space is allocated for tensor data t3. The implementation process may include: judging whether tensor data can reuse the allocated memory space (first memory space, second memory space, third memory space and fourth memory space) in combination with the constraint relationship between tensor data t3 and other tensor data in Table 2. Since tensor data can reuse the first memory space, the first memory space is allocated to tensor data t3 (for example, memory space c in the first memory space can be allocated to tensor data t3). Specifically, the allocation process may be as follows. Figure 4d As shown. It can be understood that, through this implementation, an independent memory space is allocated to each tensor data, and the relationship between tensor data and memory space is one-to-one correspondence, that is, the number of tensor data is the same as the number of memory spaces.

[0161] It is understood that in the above process of allocating memory space, each memory space includes its own corresponding first address and storage space size. Specifically, the memory space allocated for each tensor data can be as shown in Table 4:

[0162] Logo Tensor data size First address Storage space size Sorting results Tensor data t0 500 3500 [3500,4000[ 6 Tensor data t1 500 0 [0,500[ 1 Tensor data t2 500 1500 [1500,2000[ 8 Tensor data t3 500 1500 [1500,2000[ 9 Tensor data t4 500 4000 [4000,4500[ 7 Tensor data t5 1000 2500 [2500,3500[ 4 Tensor data t6 1000 2500 [2500,3500[ 5 Tensor data t7 1000 1500 [1500,2500[ 3 Tensor data t8 1000 500 [500,1500[ 2

[0163] Table 4

[0164] As shown in Table 4, [3500,4000[ means including 3500-3999 but excluding 4000, and its storage space size is 500.

[0165] In some embodiments, after determining the allocated memory space, the maximum memory size required for the entire neural network can be determined based on the allocated memory space. Figure 4d The allocation process can determine Figure 1b The maximum memory size required for the neural network shown is 4500 KB. This implementation method can determine the maximum memory size required by the computer device to run the entire neural network, avoiding the situation where the allocated memory is insufficient to support the normal operation of the neural network.

[0166] In some embodiments, after allocating corresponding memory space to each tensor data, the allocated memory space is verified to be correct based on the constraint relationship corresponding to each tensor data. If not, the memory space is reallocated to the M tensor data. For example, for tensor data t8, in the first memory space, it is determined whether the memory space corresponding to tensor data t8 is to the right of the memory space corresponding to tensor data t1. For example, as shown in Table 4, the memory space corresponding to tensor data t1 is [0,500[, and the memory space corresponding to tensor data t8 is [500,1500[. Based on the memory space corresponding to tensor data t1 and the memory space corresponding to tensor data t8, it can be determined that the memory space corresponding to tensor data t8 is to the right of the memory space corresponding to tensor data t1, which means that the allocation is correct. For another example, for tensor data t1, tensor data t7, and tensor data t8, it is determined whether the memory space corresponding to each of tensor data t1, tensor data t8, and tensor data t7 is a continuous memory space. For example, as shown in Table 4, the memory space corresponding to each of tensor data t1, tensor data t8, and tensor data t7 is a continuous memory space, wherein the first memory space is [0,500[, the second memory space is [500,1500[, and the third memory space is [1500,2500[. Based on the storage space corresponding to each of tensor data t1, tensor data t8, and tensor data t7, it can be determined that the memory space corresponding to each of tensor data t1, tensor data t8, and tensor data t7 is a continuous memory space, which means that the allocation is correct. Through this implementation method, the situation of unreasonable memory allocation can be avoided. For example, the unreasonable memory allocation can be reflected in: an allocation result that is mutually exclusive with the constraint relationship appears in the allocated memory space.

[0167] In some embodiments, Figure 1b Taking the computational graph shown as an example, computational subtask 0 contains nodes a and c, and the execution order is node a-node c; computational subtask 1 contains nodes b, d, e, and f, and the execution order is node b-node d / e-node f; computational subtask 2 contains nodes g and h, and the execution order is node g-node h. Among them, the execution relationship between computational subtask 0 and computational subtask 1 is parallel, and the execution relationship between computational subtask 1 and computational subtask 2 is serial. Since there are directed edges between two adjacent nodes in the same computational subtask in the computational graph, there is no need to adjust the computational graph corresponding to each computational subtask at this time. For two computational subtasks with a serial execution relationship (for example, computational subtask 1 and computational subtask 2), since there is no directed edge between the last node f of computational subtask 1 and the first node g of computational subtask 2, at this time, a directed edge is added between the last node f of computational subtask 1 and the first node g of computational subtask 2, and the updated computational graph is obtained as shown below. Figure 4e shown.

[0168] Secondly, in the computational graph, determine the upstream node corresponding to each node, the output tensor data corresponding to each node, and the input tensor data. Specifically, the upstream node corresponding to each node, the output tensor data corresponding to each node, and the input tensor data can be shown in Table 5:

[0169] node Upstream node Output tensor data Input tensor data a - t0,t1 - b a t2,t3 t0 c a t4 t0 d a,b t5 t2 e a,b t6 t3 f a,b,c,d,e <![CDATA[t7,t dep ]]> t5,t6,t4 g a,b,c,d,e,f t8 <![CDATA[t4,t dep ]]> h a,b,c,d,e,f,g - t1,t7,t8

[0170] Table 5

[0171] As shown in Table 5, Figure 4e In the computational graph shown, taking node f as an example, since there are the following paths in the computational graph: node a-node b-node d-node f, node a-node b-node e-node f, node a-node c-node f, it can be determined that the upstream nodes of node f include node a, node b, node c, node d, and node e. The tensor data input to node f includes tensor data t5, tensor data t6, and tensor data t4, and the tensor data output by node f includes tensor data t7 and tensor data t dep The implementation process of determining the upstream nodes corresponding to other nodes, outputting tensor data, and inputting tensor data will not be elaborated here.

[0172] Afterwards, determine the constraint relationship corresponding to each tensor data. For example, it can be determined whether all the consumption nodes of the first tensor data are upstream nodes of the production node of the second tensor data. If so, it is determined that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, it is determined that the first tensor data cannot be reused as the memory space allocated for the second tensor data. For another example, it can be determined whether all the consumption nodes of the second tensor data are downstream nodes of the production node of the first tensor data. If so, it is determined that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, it is determined that the first tensor data cannot be reused as the memory space allocated for the second tensor data.

[0173] Specifically, the above-mentioned constraint relationship can be carried in a constraint relationship table, in which the first value can be used to indicate that each tensor data can reuse the same memory space with other tensor data, the second value can be used to indicate that each tensor data cannot reuse the same memory space with other tensor data, and the third value can be used to indicate that each tensor data can be stored continuously in the same memory space with other tensor data. For the convenience of explanation, the first value "0" is used to indicate that tensor data can reuse the same memory space with other tensor data other than itself; the second value "1" is used to indicate that tensor data cannot reuse the same memory space with other tensor data other than itself; and the third value "2" is used to indicate that tensor data can be stored continuously in the same memory space with other tensor data other than itself. It should be noted that the above description is only an example and should not constitute a limitation. Specifically, the data structure can be represented as shown in Table 6:

[0174] t0 t1 t2 t3 t4 t5 t6 t7 t8 t0 - 1 1 1 1 1 1 1 0 t1 1 - 1 1 1 1 1 1 1 t2 1 1 - 0 1 1 0 0 0 t3 1 1 0 - 1 0 1 0 0 t4 1 1 1 1 - 1 1 1 1 t5 1 1 1 0 1 - 0 1 0 t6 1 1 0 1 1 0 - 1 0 t7 1 1 0 0 1 1 1 2 - t8 0 2 0 0 1 0 0 - 1

[0175] Table 6

[0176] It should be noted that, when considering whether to reuse memory space, the constraint relationship between two tensor data is symmetrical. This symmetry is reflected in the fact that the relationship is exactly the same as its inverse relationship. As shown in Table 2, taking the constraint relationship between tensor data t2 and tensor data t8 as an example, Figure 4e In the calculation graph shown, the production node of tensor data t7 is f, and the consumption node of tensor data t2 is d. Since the consumption node d of tensor data t2 is the upstream node of the production node f of tensor data t7, it can be determined that tensor data t7 can reuse the same memory space with tensor data t2. Figure 4e In the computational graph shown, within computation subtask 1, node b is a control selection node with two branches: one branch: node b - node d - node f; the other branch: node b - node e - node f. In a single neural network operation, only one branch is valid. For tensor data t2 and t3, the constraint relationship between them is that they do not require two separate memory spaces; in other words, they can reuse the same memory space. For tensor data t5 and t6, the constraint relationship between them is that they do not require two separate memory spaces; in other words, they can reuse the same memory space.

[0177] It should also be noted that when considering storing multiple tensor data in continuous memory space, the constraints between two tensor data are asymmetric. Furthermore, the orderliness between two memory spaces needs to be considered.

[0178] Then, taking the information of each tensor data including the memory space required by each tensor data and the constraint relationship corresponding to each tensor data as an example, the M tensor data are sorted from largest to smallest based on the memory space required by each tensor data and the size of the constraint amount corresponding to each tensor data, thereby obtaining a sorting result for the M tensor data. During the sorting, multiple tensor data whose constraints require continuous storage in the same memory space can be sorted as an independent entity.

[0179] Based on the constraint relationship shown in Table 6, the constraint amount corresponding to each tensor data can be obtained. For example, for tensor data t0, tensor data t0 cannot reuse the same memory space with other tensor data (t1, t2, t3, t4, t5, t6, t7), and its constraint amount is 7; for tensor data t1, tensor data t1 cannot reuse the same memory space with other tensor data (t0, t2, t3, t4, t5, t6, t7, t8), and its constraint amount is 8; for tensor data t2, tensor data t2 cannot reuse the same memory space with tensor data (t0, t1, t4, t5), and its constraint amount is 4; for tensor data t3, tensor data t3 cannot reuse the same memory space with tensor data (t0, t1, t4, t6), and its constraint amount is 4; memory space, and the constraint amount is 4; for tensor data t4, tensor data t4 cannot reuse the same memory space with tensor data (t0, t1, t2, t3, t5, t6, t7, t8), and the constraint amount is 8; for tensor data t5, tensor data t5 cannot reuse the same memory space with tensor data (t0, t1, t2, t4, t7), and the constraint amount is 5; for tensor data t6, tensor data t6 cannot reuse the same memory space with tensor data (t0, t1, t3, t4, t7), and the constraint amount is 5; for tensor data t7, tensor data t7 cannot reuse the same memory space with tensor data (t0, t1, t4, t5, t6), and the constraint amount is 5; for tensor data t8, tensor data t8 cannot reuse the same memory space with tensor data (t4, t8), and the constraint amount is 2.

[0180] Furthermore, the memory space required for tensor data t0 is 500Kb; the memory space required for tensor data t1 is 500Kb; the memory space required for tensor data t2 is 500Kb; the memory space required for tensor data t3 is 500Kb; the memory space required for tensor data t4 is 500Kb; the memory space required for tensor data t5 is 1000Kb; the memory space required for tensor data t6 is 1000Kb; the memory space required for tensor data t7 is 1000Kb; and the memory space required for tensor data t8 is 1000Kb.

[0181] Therefore, the above 9 tensor data can be sorted from large to small, and the sorting results can be shown in Table 7:

[0182] t1,t8,t7 t5 t6 t0 t4 t2 t3

[0183] Table 7

[0184] It should be noted that, when sorting, the method of sorting multiple tensor data items that are constrained to be stored contiguously in the same memory space as a single entity is merely an example and should not be construed as limiting. In practical applications, each tensor data item can also be sorted as a single entity.

[0185] Then, in this case, first, a first memory space is allocated to tensor data t1, t7 and t8. At this time, the first memory space has been allocated; wherein, the first memory space includes memory space a, memory space b and memory space c, memory space a, memory space b and memory space c are continuous memory spaces, wherein memory space a is used to store tensor data t1 (that is, the size of memory space a is equal to the size of tensor data t1), memory space b is used to store tensor data t8 (that is, the size of memory space b is equal to the size of tensor data t8), and memory space c is used to store tensor data t7 (that is, the size of memory space c is equal to the size of tensor data t7); thereafter, memory space is allocated for tensor data t5. The implementation process may include: combining the constraint relationship between tensor data t5 and other tensor data in Table 7 to determine whether tensor data t5 can reuse the allocated memory space (first memory space); since tensor data t5 can reuse the first memory space, the second memory space is allocated to tensor data t5 (for example, memory space b in the first memory space can be allocated to tensor data t5). Afterwards, memory space is allocated for tensor data t6. The implementation process may include: judging whether tensor data t6 can reuse the allocated memory space (first memory space) in combination with the constraint relationship between tensor data t6 and other tensor data in Table 7. Since tensor data t6 can reuse the first memory space, the first memory space is allocated to tensor data t6 (for example, memory space b in the first memory space can be allocated to tensor data t6). Afterwards, memory space is allocated for tensor data t0. The implementation process may include: judging whether tensor data t0 can reuse the allocated memory space (first memory space) in combination with the constraint relationship between tensor data t0 and other tensor data in Table 7. Since tensor data t0 cannot reuse the allocated memory space, at this time, a second memory space of a corresponding size is allocated to tensor data t0 according to the size of the memory space required to be occupied by tensor data t0. In this case, the allocated memory space includes the first memory space and the second memory space. Afterwards, memory space is allocated for tensor data t4. The implementation process may include: combining the constraint relationship between tensor data t4 and other tensor data in Table 7 to determine whether tensor data t4 can reuse the allocated memory space (first memory space and second memory space). Since tensor data t4 cannot reuse the allocated memory space in the combination, at this time, a third memory space of corresponding size is allocated to tensor data t4 according to the size of the memory space required to be occupied by tensor data t4. In this case, the allocated memory space includes the first memory space, the second memory space and the third memory space.Afterwards, memory space is allocated for tensor data t2. The implementation process may include: judging whether tensor data t2 can reuse the allocated memory space (first memory space, second memory space and third memory space) in combination with the constraint relationship between tensor data t2 and other tensor data in Table 7. Since tensor data t2 can reuse the first memory space, the first memory space is allocated to tensor data t2 (for example, the third memory space in the first memory space can be allocated to tensor data t2). Afterwards, memory space is allocated for tensor data t3. The implementation process may include: judging whether tensor data can reuse the allocated memory space (first memory space, second memory space and third memory space) in combination with the constraint relationship between tensor data t3 and other tensor data in Table 7. Since tensor data can reuse the first memory space, the first memory space is allocated to tensor data t3 (for example, the third memory space in the first memory space can be allocated to tensor data t3). Specifically, the allocation process may be as follows. Figure 4f As shown. It can be understood that, through this implementation, an independent memory space is allocated to each tensor data, and the relationship between tensor data and memory space is one-to-one correspondence, that is, the number of tensor data is the same as the number of memory spaces.

[0186] It is understood that in the above process of allocating memory space, each memory space includes its own corresponding first address and storage space size. Specifically, the memory space allocated for each tensor data can be as shown in Table 8:

[0187] Logo Tensor data size First address Storage space size Sorting results t0 500 2500 [2500,3000[ 6 t1 500 0 [0,500[ 1 t2 500 1500 [1500,2000[ 8 t3 500 1500 [1500,2000[ 9 t4 500 3000 [3000,3500[ 7 t5 1000 500 [500,1500[ 4 t6 1000 500 [500,1000[ 5 t7 1000 1500 [1500,2500[ 3 t8 1000 500 [500,1000[ 2

[0188] Table 8

[0189] In some embodiments, after determining the allocated memory space, the maximum memory size required for the entire neural network can be determined using the allocated memory space. For example, Figure 4e The maximum memory size required for the neural network shown is 3500 KB. This implementation method can determine the maximum memory size required by the computer device to run the entire neural network, avoiding the situation where the allocated memory is insufficient to support the normal operation of the neural network.

[0190] In some embodiments, after allocating corresponding memory space to each tensor data, the allocated memory space is verified to be correct based on the constraint relationship corresponding to each tensor data. If not, the memory space is reallocated for the M tensor data. For example, for tensor data t8, in the first memory space, it is determined whether the memory space corresponding to tensor data t8 is to the right of the memory space corresponding to tensor data t1. For example, as shown in Table 4, the memory space corresponding to tensor data t1 is [0,500[, and the memory space corresponding to tensor data t8 is [500,1000[. Based on the memory space corresponding to tensor data t1 and the memory space corresponding to tensor data t8, it can be determined that the memory space corresponding to tensor data t8 is to the right of the memory space corresponding to tensor data t1, which means that the allocation is correct. Through this implementation method, the situation of unreasonable memory allocation can be avoided. For example, the unreasonable memory allocation can be reflected in: an allocation result that is mutually exclusive with the constraint relationship appears in the allocated memory space.

[0191] In some embodiments, Figure 1b Taking the calculation graph shown as an example, calculation subtask 0 contains nodes a and node c, and the execution order is node a-node c; calculation subtask 1 contains nodes b, node d, node e and node f, and the execution order is node b-node d / e-node f; calculation subtask 2 contains nodes g and node h, and the execution order is node g-node h. Among them, the execution relationship between calculation subtask 0, calculation subtask 1 and calculation subtask 2 is parallel. In the same calculation subtask in the calculation graph, the execution order of each node is obtained, and then the same node is encoded in sequence according to the execution order of each node to obtain the identifier (for example, serial number) corresponding to each node. Among two adjacent nodes, the identifier corresponding to the previous node is smaller than the identifier corresponding to the next node. For example, Figure 1b Taking the computation subtask 0 in the example, the identifier of each node in the computation subtask 0 can be shown in Table 9:

[0192]

[0193] Table 9

[0194] It should be noted that the identifiers shown in Table 9 are unique.

[0195] For example, if the End ID is greater than the Start ID, the node corresponding to the End ID is the upstream node of the node corresponding to the Start ID. Next, the constraints corresponding to each tensor data item are determined. For details on how to sort the M tensor data items based on their information and how to allocate memory for them, please refer to the previous description and will not be elaborated here.

[0196] In some embodiments, Figure 1b Taking the computation graph shown as an example, computation subtask 0 includes nodes a and c, and its execution order is node a-node c; computation subtask 1 includes nodes b, node d, node e, and node f, and its execution order is node b-node d / e-node f; computation subtask 2 includes nodes g and node h, and its execution order is node g-node h. The execution relationship between computation subtask 0 and computation subtask 1 is parallel, and the execution relationship between computation subtask 1 and computation subtask 2 is serial. Within the same computation subtask in the computation graph, the execution order of each node is obtained, and then the same node is encoded in sequence according to the execution order of each node to obtain the corresponding identifier (e.g., serial number) for each node. Within at least two computation subtasks with a serial execution relationship, the execution order of each node within the at least two computation subtasks is obtained, and the nodes within the at least two computation subtasks are encoded in sequence according to the execution order of each node to obtain the corresponding identifier for each node. Between two adjacent nodes, the identifier corresponding to the preceding node is smaller than the identifier corresponding to the following node.

[0197] For example, Figure 1b Taking the computing subtask 2 in the example, the identifier of each node in the computing subtask 2 can be shown in Table 10:

[0198]

[0199]

[0200] Table 10

[0201] It should be noted that the identifiers shown in Table 10 are unique.

[0202] For example, if the End ID is greater than the Start ID, the node corresponding to the End ID is the upstream node of the node corresponding to the Start ID. Next, the constraints corresponding to each tensor data item are determined. For details on how to sort the M tensor data items based on their information and how to allocate memory for them, please refer to the previous description and will not be elaborated here.

[0203] In some embodiments, the computational graph corresponding to the neural network contains three tensor data, which can be represented as tensor data t000, tensor data t100, and tensor data t200, respectively. The memory space occupied by tensor data t000 is 1000Kb, the memory space occupied by tensor data t100 is 600Kb, and the memory space occupied by tensor data t200 is 450Kb.

[0204] Furthermore, the constraint relationships corresponding to the above three tensor data can be shown in Table 11:

[0205] t000 t100 t200 t000 - 0 0 t100 0 - 1 t200 0 1 -

[0206] Table 11

[0207] In the constraint table shown in Table 11, the first value "0" indicates that the tensor data can reuse the same memory space with other data except itself; the second value "1" indicates that the tensor data cannot reuse the same memory space with other tensor data except itself. For details on how to determine the corresponding constraint relationship for each tensor data, please refer to the previous description and will not be elaborated here.

[0208] As shown in Table 11, for tensor data t000, tensor data t000 can reuse the same memory space with tensor data t100, and can also reuse the same memory space with tensor data t200; for tensor data t100, tensor data t100 can reuse the same memory space with tensor data t000, but cannot reuse the same memory space with tensor data t200; for tensor data t200, tensor data t200 can reuse the same memory space with tensor data t000, but cannot reuse the same memory space with tensor data t100.

[0209] Afterwards, the three tensor data are sorted from largest to smallest according to the memory space occupied by each tensor data to obtain a sorting result. Specifically, the sorting result can be shown in Table 12:

[0210] t000 t100 t200

[0211] Table 12

[0212] Then, in this case, when the memory allocation device allocates memory space for the above three tensor data, first, a first memory space is created, and the first memory space is used to store the tensor data t000. For example, the starting address of the created first memory space is a, and its storage space size is the size of the memory space required to be occupied by the tensor data t000; then, a second memory space is created, and the second memory space is used to store the tensor data t100. Since the tensor data t100 can reuse the memory space allocated for the tensor data t000, at this time, the starting address of the created second memory space is a (that is, the same as the starting address of the first memory space), and the size of its storage space is the memory space required to be occupied by the tensor data t100. As mentioned above, the size of the memory space required to be occupied by the tensor data t000 is 1000Kb, and the size of the memory space required to be occupied by the tensor data t100 is 600Kb, which means that the tensor data t100 reuses part of the memory space [a, 600[ in the first memory space. Afterwards, a third memory space is created, and the third memory space is used to store tensor data t200. Since tensor data t200 can reuse the memory space [a, 1000[ allocated for tensor data t000, but cannot reuse the memory space [a, 600[ allocated for tensor data t100, the first address of the third memory space is 600, and its storage space size is the memory space size required for tensor data t200, which means that tensor data t200 reuses part of the memory space [600, 1000[ in the first memory space. Specifically, the allocation process can be as follows Figure 4g shown.

[0213] It should be noted that, in some embodiments, the above-mentioned constraint relationship can also be reflected in: the memory space allocated to the tensor data and the memory space allocated to other tensor data other than itself are non-contiguous memory spaces (for example, memory space 1 is discarded for tensor data, and memory space 2 is allocated for tensor data 2, where there is a spatial gap between memory space 1 and memory space 2), whether the tensor data satisfies the spatial co-location constraint with other tensor data other than itself, where the spatial co-location constraint is reflected in: the i-th tensor data cannot reuse the same memory space with the j-th tensor data, the i-th tensor data can reuse the same memory space with the k-th tensor data, then the i-th tensor data cannot reuse the same memory space with the l-th tensor data, where the l-th tensor data is the tensor data that overlaps between the j-th tensor data and the k-th tensor data, and so on.

[0214] In implementing the embodiment of the present application, the memory allocation device allocates memory space of corresponding size to each tensor data in sequence based on the sorting results of M tensor data. Compared with the existing technology, the memory space is allocated and reused according to the order of the operation of the entire neural network, which can avoid the occurrence of unreasonable memory allocation, thereby saving the memory required by the entire neural network and optimizing the memory allocation of the neural network.

[0215] See Figure 5 , is a flow chart of a memory allocation method for a neural network provided by an embodiment of the present application. The execution subject of the method can be a server running a neural network, or a terminal device running a neural network. For the convenience of explanation, the execution subject is taken as an example of a terminal device running a neural network. Figure 5 In the flowchart of the method shown in FIG, it is possible to set the entire neural network to require multiple memory spaces during operation. Figure 5 As shown, the method may include but is not limited to the following steps:

[0216] Step S501: Obtain a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1.

[0217] Regarding the specific implementation of step S501, please refer to the above description and will not be elaborated here.

[0218] Step S502: Based on the constraint relationship corresponding to each tensor data, memory space is allocated to the M tensor data in sequence according to the execution order of the M tensor data in the neural network.

[0219] In an embodiment of the present application, the implementation process of determining the constraint relationship corresponding to each tensor data in the computational graph may include: determining whether all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, and if so, determining that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, determining that the first tensor data cannot be reused as the memory space allocated for the second tensor data. For another example, it may be determined whether all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, and if so, determining that the first tensor data can be reused as the memory space allocated for the second tensor data; if not, determining that the first tensor data cannot be reused as the memory space allocated for the second tensor data.

[0220] In an embodiment of the present application, the above method can be applied to scenarios in which multiple computing subtasks are executed in parallel (for example, the execution relationship between multiple computing subtasks is all parallel; for another example, the execution relationship between multiple computing subtasks includes serial and parallel), and the terminal device obtains the constraint relationship corresponding to each tensor data. Then, based on the execution order of the tensor data in the neural network and the constraint relationship corresponding to each tensor data, memory is allocated to the tensor data in turn. This can avoid the situation in which, in parallel scenarios, the operator calculation results of different computing subtasks are erroneous due to the operators reusing the same memory space, and can ensure the accuracy of the calculation results of the neural network.

[0221] Above Figure 1a-Figure 5 The memory allocation method involved in the embodiment of the present application is described in detail. The device involved in the embodiment of the present application is introduced below with reference to the accompanying drawings.

[0222] Figure 6 Schematic diagram of the structure of a memory allocation device 60 in an embodiment of the present application. Figure 6 The memory allocation device 60 shown may include:

[0223] Obtaining a computational graph unit 600, for obtaining a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1; an allocating unit 602, for allocating memory space to the M tensor data in sequence based on the sorting results of the M tensor data, wherein if one of the M tensor data can reuse at least a portion of the allocated memory space, then at least a portion of the memory space that can be reused by the tensor data is allocated to the tensor data, and the The allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data, and the sorting result indicates the order in which the memory space is allocated to the M tensor data. The sorting result is related to the information of each tensor data in the M tensor data, and the information of each tensor data indicates at least one of the following information: the constraint relationship corresponding to each tensor data and the number of nodes to which each tensor data flows, and the constraint relationship indicates the relationship between the available memory space of one tensor data in the M tensor data and the available memory space of other tensor data in the M tensor data. In a possible implementation, the allocation unit 606 is also used to: if the tensor data cannot reuse the allocated memory space, allocate other memory space to the tensor data, and the other memory space is different from the allocated memory space.

[0224] In one possible implementation, the constraint relationship indicates at least one of the following relationships: the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable, and the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable and continuous.

[0225] In one possible implementation, the constraint relationship is carried in a constraint relationship table, which includes the identifiers of the M data tensors. In the constraint relationship table, a first value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, a second value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and a third value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous.

[0226] In one possible implementation, when all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows out.

[0227] In one possible implementation, the computation graph includes a plurality of computation subtasks, each of which indicates a computation function through a set of nodes and edges associated with the set of nodes, and the execution relationship between the plurality of computation subtasks is parallel execution; the apparatus further includes:

[0228] The computation graph updating unit 604 is configured to, in a computation subtask, add a directed edge between two adjacent nodes if there is no directed edge between the two adjacent nodes, thereby updating the computation graph; wherein each added directed edge carries corresponding tensor data; and the two adjacent nodes are two adjacent nodes in the execution order of the computation subtask;

[0229] The information acquisition unit 606 is used to acquire information of each tensor data based on the updated computation graph.

[0230] In one possible implementation, the computation graph further includes a first computation subtask and a second computation subtask whose execution relationship is serial, and the first computation subtask is executed before the second computation subtask; and the computation graph updating unit 604 is further configured to:

[0231] If there is no directed edge between the last node of the first computing subtask and the first node of the second computing subtask, a directed edge is added between the last node of the first computing subtask and the first node of the second computing subtask.

[0232] In a possible implementation, in the computational graph, an identifier of a production node of tensor data is smaller than an identifier of a consumption node of the tensor data; the production node of the tensor data and the consumption node of the tensor data are two adjacent nodes.

[0233] In a possible implementation, the identifier of each node in the computation graph is used to determine information of each tensor data in the M tensor data.

[0234] In a possible implementation, the information of each tensor data indicates a constraint relationship corresponding to each tensor data, and the apparatus further includes:

[0235] The first sorting unit 608 is used to obtain the constraint amount corresponding to each of the M tensor data according to the constraint relationship corresponding to each tensor data; the constraint amount is the number of tensor data in other tensor data that cannot reuse the same memory space with the tensor data; according to the constraint amount corresponding to each of the M tensor data, the M tensor data are sorted to obtain the sorting result of the M tensor data.

[0236] In a possible implementation, the information of each tensor data indicates the number of nodes to which each tensor data flows, and the apparatus further includes:

[0237] The second sorting unit 6010 is used to sort the M tensor data according to the number of consumer nodes corresponding to each of the M tensor data to obtain a sorting result of the M tensor data.

[0238] In a possible implementation, the apparatus further includes:

[0239] The third sorting unit 6012 is configured to sort the M tensor data using a heuristic algorithm based on the information of each tensor data, so as to obtain a sorting result of the M tensor data within a preset time period.

[0240] In one possible implementation, the sorting result is an optimized sorting result, wherein the maximum memory size required to be occupied by the neural network corresponding to the optimized sorting result is smaller than the maximum memory size required to be occupied by the neural network determined based on the sorting result before optimization.

[0241] In the embodiments of the present application, the specific implementation of each unit can be found in the relevant description in the above embodiments, which will not be repeated here.

[0242] In implementing the embodiment of the present application, the memory allocation device obtains the sorting results of M tensor data based on the information of each tensor data, and allocates memory space of corresponding size to each tensor data according to the sorting results. Compared with the prior art, the memory space is allocated and reused according to the order of the entire neural network operation, which can avoid the unreasonable memory allocation phenomenon, thereby saving the memory required by the entire neural network and optimizing the memory allocation of the neural network.

[0243] like Figure 7 As shown, an embodiment of the present application provides a memory allocation device 70, which can be specifically a terminal device or a server. In some embodiments, the memory allocation device 70 can be specifically a central control module in a server, or its functions are implemented by the central control module in the server. In some embodiments, the memory allocation device 70 can be specifically a central control module in a terminal device, or its functions are implemented by the central control module in the terminal device. Figure 7 As shown, the memory allocation device may include a processor 701, a memory 702, a communication bus 703 and a communication interface 704, and the processor 701 is connected to the memory 702 and the communication interface 703 via the communication bus.

[0244] The processor 701 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), a neural network processor (NPU) or one or more integrated circuits to execute relevant programs to execute the memory allocation method described in the embodiment of the method of the present application.

[0245] The processor 701 may also be an integrated circuit chip with signal processing capabilities. During implementation, the various steps of the memory allocation method of the present application may be completed by hardware integrated logic circuits or software instructions in the processor 701. The aforementioned processor 701 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in the memory 701, and the processor 701 reads the information in the memory 702 and executes the memory allocation method of the embodiment of the method of the present application in combination with its hardware.

[0246] The memory 702 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 may store programs and data, such as the program of the memory allocation method in the embodiment of the present application. When the program stored in the memory 701 is executed by the processor 702, the processor 701 and the communication interface 704 are used to execute the various steps of the memory allocation method in the embodiment of the present application.

[0247] For example, the program used to implement the memory allocation method in the embodiment of the present application, etc.

[0248] The communication interface 704 uses a transceiver device such as, but not limited to, a transceiver to enable communication between the memory allocation device 700 and other devices or communication networks. For example, a trained neural network can be obtained through the communication interface 704 to enable information exchange with an execution device, client device, user device, or terminal device.

[0249] Optionally, the memory allocation device may further include an artificial intelligence processor 705, which may be a neural network processor (NPU), a tensor processing unit (TPU), or a graphics processing unit (GPU), or any other processor suitable for large-scale XOR operation processing. The artificial intelligence processor 705 can be mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks to it. The artificial intelligence processor 705 can implement one or more operations involved in the above-mentioned memory allocation method. For example, taking the NPU as an example, the core part of the NPU is the operation circuit, which is controlled by the controller to extract the matrix data in the memory 702 and perform multiplication and addition operations.

[0250] The processor 701 is used to call the data and program codes in the memory and execute:

[0251] Obtain a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1;

[0252] Based on the sorting results of the M tensor data, memory space is allocated to the M tensor data in sequence, wherein, if one tensor data among the M tensor data can reuse at least a part of the allocated memory space, then at least a part of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data. The sorting result indicates the order in which memory space is allocated to the M tensor data, and the sorting result is related to the information of each tensor data in the M tensor data. The information of each tensor data indicates at least one of the following information: the constraint relationship corresponding to each tensor data and the number of nodes to which each tensor data flows, and the constraint relationship indicates the relationship between the available memory space of one tensor data among the M tensor data and the available memory space of other tensor data among the M tensor data.

[0253] The processor 701 is further configured to:

[0254] If the tensor data cannot reuse the allocated memory space, other memory space is allocated for the tensor data, and the other memory space is different from the allocated memory space.

[0255] Among them, the constraint relationship indicates at least one of the following relationships: the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable, and the relationship between the available memory space of one tensor data and the available memory space of another tensor data is non-reusable and continuous.

[0256] In which, the constraint relationship is carried in a constraint relationship table, and the constraint relationship table includes the identifiers of the M data tensors. In the constraint relationship table, the first value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the second value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and the third value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous.

[0257] Among them, when all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows out.

[0258] The computation graph includes a plurality of computation subtasks, each of which indicates a computation function through a set of nodes and edges associated with the set of nodes, and the execution relationship between the plurality of computation subtasks is parallel execution; the processor 701 is further configured to:

[0259] In one of the computation subtasks, if there is no directed edge between two adjacent nodes, a directed edge is added between the two adjacent nodes to update the computation graph; wherein each added directed edge carries corresponding tensor data; the two adjacent nodes are two adjacent nodes in the execution order of the computation subtask;

[0260] Based on the updated computation graph, information about each tensor data is obtained.

[0261] The computation graph further includes a first computation subtask and a second computation subtask whose execution relationship is serial, and the execution order of the first computation subtask is before the execution order of the second computation subtask; the processor 701 updates the computation graph, further including:

[0262] If there is no directed edge between the last node of the first computation subtask and the first node of the second computation subtask, add a directed edge between the last node of the first computation subtask and the first node of the second computation subtask.

[0263] In the computational graph, the identifier of the tensor data production node is smaller than the identifier of the tensor data consumption node; the tensor data production node and the tensor data consumption node are two adjacent nodes.

[0264] The identifier of each node in the computational graph is used to determine information of each tensor data in the M tensor data.

[0265] The information of each tensor data indicates a constraint relationship corresponding to each tensor data, and the processor 701 is further configured to:

[0266] Obtaining a constraint amount corresponding to each of the M tensor data according to the constraint relationship corresponding to each tensor data; the constraint amount is the number of tensor data in other tensor data that cannot reuse the same memory space with the tensor data;

[0267] The M tensor data are sorted according to the constraint amounts corresponding to the respective M tensor data to obtain a sorting result of the M tensor data.

[0268] The information of each tensor data indicates the number of nodes to which each tensor data flows, and the processor 701 is further configured to:

[0269] The M tensor data are sorted according to the number of consumer nodes corresponding to each of the M tensor data to obtain a sorting result of the M tensor data.

[0270] The processor 701 is further configured to:

[0271] The M tensor data are sorted using a heuristic algorithm based on information of each tensor data, so as to obtain a sorting result of the M tensor data within a preset time period.

[0272] The sorting result is an optimized sorting result, wherein the maximum memory size required to be occupied by the neural network corresponding to the optimized sorting result is smaller than the maximum memory size required to be occupied by the neural network determined according to the sorting result before optimization.

[0273] It should be understood that the implementation of each device can also refer to the corresponding description in the above-mentioned memory allocation method embodiment, and the embodiments of this application will not be repeated.

[0274] The embodiments of the present application also provide a computer storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer or processor, cause the computer or processor to execute one or more steps of the method described in any of the above embodiments. If the various components of the above-mentioned device are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, and the computer product is stored in a computer-readable storage medium.

[0275] The computer-readable storage medium may be an internal storage unit of the device described in the aforementioned embodiment, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of the device, such as an equipped plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Furthermore, the computer-readable storage medium may include both an internal storage unit and an external storage device of the device. The computer-readable storage medium is used to store the computer program and other programs and data required by the device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0276] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0277] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0278] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0279] It is understood that those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the various embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0280] Those skilled in the art will appreciate that the functions described in the various illustrative logic blocks, modules, and algorithm steps disclosed in the various embodiments of this application can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any media that facilitates the transfer of computer programs from one place to another (e.g., according to a communication protocol). In this way, computer-readable media can generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this application. A computer program product can include computer-readable media.

[0281] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0282] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0283] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0284] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0285] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0286] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A memory allocation method, characterized in that: include: Obtaining a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1; the nodes are operators, and the directed edges indicate pointing directions between operators; Based on the sorting results of the M tensor data, memory space is allocated to the M tensor data in sequence, wherein, if one tensor data among the M tensor data can reuse at least a part of the allocated memory space, then at least a part of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data. The sorting result indicates the order in which memory space is allocated to the M tensor data, and the sorting result is related to the information of each tensor data in the M tensor data. The information of each tensor data indicates at least one of the following information: the constraint relationship corresponding to each tensor data and the number of nodes to which each tensor data flows, and the constraint relationship indicates the relationship between the available memory space of one tensor data among the M tensor data and the available memory space of other tensor data among the M tensor data.

2. The method according to claim 1, wherein If the tensor data cannot reuse the allocated memory space, other memory space is allocated for the tensor data, and the other memory space is different from the allocated memory space.

3. The method according to claim 1, wherein The constraint relationship indicates at least one of the following relationships: the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous.

4. The method according to claim 3, wherein The constraint relationship is carried in a constraint relationship table, which includes the identifiers of the M data tensors. In the constraint relationship table, a first value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, a second value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and a third value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous.

5. The method according to claim 1, wherein In the case where all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; in the case where all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows out.

6. The method according to claim 2, wherein In the case where all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; in the case where all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows out.

7. The method according to claim 3, wherein In the case where all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; in the case where all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows out.

8. The method according to claim 4, wherein In the case where all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; in the case where all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or, in the case where all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows out.

9. The method according to any one of claims 1 to 8, wherein The computation graph includes a plurality of computation subtasks, each of which indicates a computation function through a set of nodes and edges related to the set of nodes, and the execution relationship between the plurality of computation subtasks is parallel execution; The method further comprises: In one of the computation subtasks, if there is no directed edge between two adjacent nodes, a directed edge is added between the two adjacent nodes to update the computation graph; wherein each added directed edge carries corresponding tensor data; the two adjacent nodes are two adjacent nodes in the execution order of the computation subtask; Based on the updated computation graph, information about each tensor data is obtained.

10. The method according to claim 9, wherein The computation graph further includes a first computation subtask and a second computation subtask that are executed in a serial relationship, wherein the first computation subtask is executed before the second computation subtask; The updating of the computation graph further includes: If there is no directed edge between the last node of the first computing subtask and the first node of the second computing subtask, a directed edge is added between the last node of the first computing subtask and the first node of the second computing subtask.

11. The method according to any one of claims 1 to 8, wherein In the computational graph, an identifier of a production node of tensor data is smaller than an identifier of a consumption node of the tensor data; the production node of the tensor data and the consumption node of the tensor data are two adjacent nodes.

12. The method according to claim 11, wherein The identifier of each node in the computation graph is used to determine information of each tensor data in the M tensor data.

13. The method according to claim 1, wherein The information of each tensor data indicates a constraint relationship corresponding to each tensor data, and the method further includes: Obtaining a constraint amount corresponding to each of the M tensor data according to the constraint relationship corresponding to each tensor data; the constraint amount is the number of tensor data in other tensor data that cannot reuse the same memory space with the tensor data; The M tensor data are sorted according to the constraint amounts corresponding to the respective M tensor data to obtain a sorting result of the M tensor data.

14. The method according to claim 1, wherein The information of each tensor data indicates the number of nodes to which each tensor data flows, and the method further includes: The M tensor data are sorted according to the number of consumer nodes corresponding to each of the M tensor data to obtain a sorting result of the M tensor data.

15. The method according to claim 1, wherein The method further comprises: The M tensor data are sorted using a heuristic algorithm based on information of each tensor data, so as to obtain a sorting result of the M tensor data within a preset time period.

16. The method according to claim 15, wherein The sorting result is an optimized sorting result, wherein the maximum memory size required to be occupied by the neural network corresponding to the optimized sorting result is smaller than the maximum memory size required to be occupied by the neural network determined according to the sorting result before optimization.

17. A memory allocation method, characterized in that: include: Obtaining a computational graph corresponding to a neural network; wherein the computational graph includes N nodes and directed edges connecting different nodes, the directed edges of the computational graph carry tensor data, and the computational graph includes M tensor data, where M is an integer greater than 1; the nodes are operators, and the directed edges indicate pointing directions between operators; Based on the constraint relationship corresponding to each tensor data, memory space is allocated to the M tensor data in sequence according to the execution order of the M tensor data in the neural network, wherein, if one tensor data among the M tensor data can reuse at least a part of the allocated memory space, then at least a part of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data, and the constraint relationship indicates the relationship between the available memory space of one tensor data among the M tensor data and the available memory space of other tensor data among the M tensor data.

18. The method according to claim 17, wherein If the tensor data cannot reuse the allocated memory space, other memory space is allocated for the tensor data, and the other memory space is different from the allocated memory space.

19. A memory allocation device, characterized in that: include: A computation graph acquisition unit is configured to acquire a computation graph corresponding to a neural network; wherein the computation graph includes N nodes and directed edges connecting different nodes, the directed edges of the computation graph carry tensor data, and the computation graph includes M tensor data, where M is an integer greater than 1; the nodes are operators, and the directed edges indicate the direction between operators; An allocation unit is used to allocate memory space to the M tensor data in sequence based on the sorting results of the M tensor data, wherein if one of the M tensor data can reuse at least a part of the allocated memory space, then at least a part of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data. The sorting result indicates the order in which the memory space is allocated to the M tensor data, and the sorting result is related to the information of each tensor data in the M tensor data, and the information of each tensor data indicates at least one of the following information: the constraint relationship corresponding to each tensor data and the number of nodes to which each tensor data flows, and the constraint relationship indicates the relationship between the available memory space of one tensor data in the M tensor data and the available memory space of other tensor data in the M tensor data.

20. The device according to claim 19, wherein The allocation unit is further configured to: If the tensor data cannot reuse the allocated memory space, other memory space is allocated for the tensor data, and the other memory space is different from the allocated memory space.

21. The device according to claim 19, wherein The constraint relationship indicates at least one of the following relationships: the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous.

22. The device according to claim 21, wherein The constraint relationship is carried in a constraint relationship table, which includes the identifiers of the M data tensors. In the constraint relationship table, a first value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is reusable, a second value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable, and a third value indicates that the relationship between the available memory space of one tensor data and the available memory space of another tensor data is not reusable and continuous.

23. The device as described in claim 19, when all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows.

24. The device as described in claim 20, when all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows.

25. The device as described in claim 21, when all the consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all the consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all the consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all the consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows.

26. The device as described in claim 22, when all consumer nodes of the first tensor data are upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are downstream nodes of the production node of the first tensor data, the first tensor data can reuse the memory space allocated for the second tensor data; when all consumer nodes of the first tensor data are not upstream nodes of the production node of the second tensor data, or when all consumer nodes of the second tensor data are not downstream nodes of the production node of the first tensor data, the first tensor data cannot reuse the memory space allocated for the second tensor data; the first tensor data and the second tensor data are any two of the M tensor data; the consumer node is the node to which the tensor data flows, and the production node is the node from which the tensor data flows.

27. The device according to any one of claims 19 to 26, characterized in that The computation graph includes a plurality of computation subtasks, each of which indicates a computation function through a set of nodes and edges related to the set of nodes, and the execution relationship between the plurality of computation subtasks is parallel execution; The device further comprises: a computation graph updating unit configured to, in a computation subtask, add a directed edge between two adjacent nodes if there is no directed edge between the two adjacent nodes, thereby updating the computation graph; wherein each added directed edge carries corresponding tensor data; and the two adjacent nodes are two adjacent nodes in the execution order of the computation subtask; An information acquisition unit is used to acquire information of each tensor data based on the updated computation graph.

28. The device according to claim 27, wherein The computation graph further includes a first computation subtask and a second computation subtask that are executed in a serial relationship, wherein the first computation subtask is executed before the second computation subtask; and the computation graph updating unit is further configured to: If there is no directed edge between the last node of the first computing subtask and the first node of the second computing subtask, a directed edge is added between the last node of the first computing subtask and the first node of the second computing subtask.

29. The device according to any one of claims 19 to 26, characterized in that In the computational graph, an identifier of a production node of tensor data is smaller than an identifier of a consumption node of the tensor data; the production node of the tensor data and the consumption node of the tensor data are two adjacent nodes.

30. The device according to claim 29, wherein The identifier of each node in the computation graph is used to determine information of each tensor data in the M tensor data.

31. The device according to claim 19, wherein The information of each tensor data indicates a constraint relationship corresponding to each tensor data, and the device further includes: The first sorting unit is used to obtain the constraint amount corresponding to each of the M tensor data according to the constraint relationship corresponding to each tensor data; the constraint amount is the number of tensor data in other tensor data that cannot reuse the same memory space with the tensor data; according to the constraint amount corresponding to each of the M tensor data, the M tensor data are sorted to obtain the sorting result of the M tensor data.

32. The device according to claim 19, wherein The information of each tensor data indicates the number of nodes to which each tensor data flows, and the apparatus further includes: The second sorting unit is used to sort the M tensor data according to the number of consumer nodes corresponding to each of the M tensor data to obtain a sorting result of the M tensor data.

33. The device according to claim 19, wherein The device further comprises: A third sorting unit is configured to sort the M tensor data using a heuristic algorithm based on information of each tensor data, so as to obtain a sorting result of the M tensor data within a preset time period.

34. The device according to claim 33, wherein The sorting result is an optimized sorting result, wherein the maximum memory size required to be occupied by the neural network corresponding to the optimized sorting result is smaller than the maximum memory size required to be occupied by the neural network determined according to the sorting result before optimization.

35. A memory allocation device, characterized in that: include: A computation graph acquisition unit is configured to acquire a computation graph corresponding to a neural network; wherein the computation graph includes N nodes and directed edges connecting different nodes, the directed edges of the computation graph carry tensor data, and the computation graph includes M tensor data, where M is an integer greater than 1; the nodes are operators, and the directed edges indicate the direction between operators; An allocation unit is used to allocate memory space to the M tensor data in sequence based on the constraint relationship corresponding to each tensor data and in the order in which the M tensor data are executed in the neural network, wherein if one of the M tensor data can reuse at least a portion of the allocated memory space, at least a portion of the memory space that can be reused by the tensor data is allocated to the tensor data, and the allocated memory space is the memory space that has been allocated to the M tensor data before the tensor data, and the constraint relationship indicates the relationship between the available memory space of one tensor data among the M tensor data and the available memory space of other tensor data among the M tensor data.

36. The device according to claim 35, wherein The allocation unit is further configured to: If the tensor data cannot reuse the allocated memory space, other memory space is allocated for the tensor data, and the other memory space is different from the allocated memory space.

37. A memory allocation device, characterized in that: including processor and memory; The memory is used to store a computer program, wherein the computer program includes program instructions; The processor is configured to call the program instructions and execute the method according to any one of claims 1 to 18.

38. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 18.

Images