A method and device for space allocation of computing nodes

By classifying and fusing the computing nodes of neural networks, and using infectious algorithms and ONNX tools to derive storage space requirements, the problem that the inference engine cannot determine the storage space of the computing nodes is solved, and the computing efficiency is improved.

CN115018064BActive Publication Date: 2024-11-08UNIV OF SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210737352.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-27
Publication Date
2024-11-08
Estimated Expiration
2042-06-27

AI Technical Summary

Technical Problem

The inference engine cannot obtain the output shape and memory or video memory size of each computing node of the neural network model before execution, resulting in low computing efficiency.

Method used

By classifying the computing nodes of the neural network, fusing the computing shape nodes and the computing tensor nodes, generating new nodes, using the infectious algorithm and ONNX tool to derive the storage space requirements of the computing nodes, and allocating storage space for the computing nodes in advance.

Benefits of technology

Before the inference engine is executed, the storage space requirements of each computing node can be determined, saving time to open up storage space, and improving the computing efficiency of the inference engine.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115018064B_ABST
    Figure CN115018064B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a method and device for allocating space of computing nodes, which classifies each computing node of a neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes; merges the computing shape nodes related to each computing tensor node into a new node; inputs the data to be inferred into the neural network; deduces the storage space required for each computing node of the neural network to calculate the data to be inferred based on the attributes and computing characteristics of the computing tensor nodes, the attributes and computing characteristics of the new nodes, and the data to be inferred; and allocates the storage space to each computing node. It can be seen that in the embodiment of the present invention, the storage space corresponding to each computing node can be obtained by offline derivation before the execution of the inference engine, and the corresponding storage space can be allocated to each computing node before the calculation starts, which can save the time of opening up storage space during calculation, thereby improving the overall calculation efficiency of the inference engine.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of storage space allocation, and more specifically, to a method and device for allocating space of a computing node. Background Art

[0002] Currently, before the inference engine is executed, it is impossible to obtain the output shape of each computing node of the neural network model, that is, it is impossible to obtain the memory or video memory size required for the calculation of each computing node of the neural network model. Therefore, when the inference engine is executed, it takes time to determine the memory or video memory size corresponding to each computing node, which makes the calculation efficiency of the inference engine low. Summary of the invention

[0003] In view of this, an embodiment of the present invention discloses a method and device for allocating space of computing nodes to obtain storage space corresponding to each computing node, thereby improving the computing efficiency of an inference engine.

[0004] The technical solution provided by the embodiment of the present invention is as follows:

[0005] A first aspect of an embodiment of the present invention provides a method for allocating space of a computing node, the method comprising:

[0006] Classify each computing node of the neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes;

[0007] Merging the computation shape nodes associated with each computation tensor node into a new node; the new node includes attributes and a topological order of the computation shape nodes associated with each computation tensor node;

[0008] Inputting the data to be inferred into the neural network;

[0009] Derived according to the properties and computing characteristics of the computing tensor node, the properties and computing characteristics of the new node and the data to be inferred, the storage space required for each computing node of the neural network to calculate the data to be inferred;

[0010] The storage space is allocated to each computing node.

[0011] In a possible implementation, the step of fusing the computation shape nodes associated with each computation tensor node into a new node includes:

[0012] Traversing all the computation tensor nodes in the neural network in reverse order to obtain all computation shape nodes associated with the input of each computation tensor node;

[0013] Traversing and recording all the calculation shape nodes by using the contagion algorithm until finding the calculation shape node for obtaining the tensor shape corresponding to each of the calculation tensor nodes;

[0014] Obtaining a calculation shape set related to each of the calculation tensor nodes according to all the calculation shape nodes related to the input of the calculation tensor node and the calculation shape node for obtaining the tensor shape;

[0015] The attributes and topological order of all the computational shape nodes in the computational shape set are stored in a new node.

[0016] In a possible implementation, after storing the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node, the method further includes:

[0017] Deleting all the calculated shape nodes traversed and recorded by the infection algorithm;

[0018] The new nodes are inserted into the neural network in topological order.

[0019] In a possible implementation, storing the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node includes:

[0020] Generate a new node using the Generate Node method provided by Open Neural Network Exchange ONNX;

[0021] Generate a Json string according to the attributes and topological order of each calculation shape node in the calculation shape set through the Onnx2Json tool;

[0022] Store the Json string into the new node.

[0023] In a possible implementation, the classifying of the computing nodes of the neural network to obtain computing shape nodes and computing tensor nodes includes:

[0024] Traversing all computing nodes in the neural network through an infection algorithm;

[0025] If all inputs of the computing node are shape information, determining the computing node as the computing shape node;

[0026] If all inputs of the computing node are not shape information, the computing node is determined to be the computing tensor node.

[0027] A second aspect of an embodiment of the present invention provides a space allocation device for a computing node, the device comprising:

[0028] A classification unit, used for classifying each computing node of the neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes;

[0029] A fusion unit, used for fusing the calculation shape nodes associated with each of the calculation tensor nodes into a new node; the new node contains the attributes and topological order of the calculation shape nodes associated with each of the calculation tensor nodes;

[0030] An input unit, used for inputting data to be inferred into the neural network;

[0031] A derivation unit, used to derive the storage space required for each computing node of the neural network to calculate the data to be inferred according to the attributes and computing characteristics of the computing tensor node, the attributes and computing characteristics of the new node and the data to be inferred;

[0032] An allocating unit is used to allocate the storage space to each computing node.

[0033] In a possible implementation, the fusion unit includes:

[0034] The fusion unit comprises:

[0035] A first traversal unit, used for traversing all the computing tensor nodes in the neural network in reverse order to obtain all computing shape nodes related to the input of each computing tensor node;

[0036] A second traversal unit, used for traversing and recording all the calculation shape nodes by using an infection algorithm, until a calculation shape node for obtaining a tensor shape corresponding to each of the calculation tensor nodes is found;

[0037] A collection unit, used for obtaining a calculation shape set related to each of the calculation tensor nodes according to all the calculation shape nodes related to the input of the calculation tensor node and the calculation shape node for obtaining the tensor shape;

[0038] The fusion subunit is used to store the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node.

[0039] In a possible implementation manner, the device further includes:

[0040] A deleting unit, used to delete all the calculated shape nodes traversed and recorded by the contagion algorithm;

[0041] An inserting unit is used to insert the new node into the neural network in a topological order.

[0042] In a possible implementation, the fusion subunit includes:

[0043] A first generation unit, used for generating a new node by using a generation node method provided by the open neural network exchange ONNX;

[0044] A second generating unit is used to generate a Json string according to the attributes and topological order of each calculation shape node in the calculation shape set by using an Onnx2Json tool;

[0045] A storage unit is used to store the Json string in the new node.

[0046] In a possible implementation, the classification unit includes:

[0047] A third traversal unit, used for traversing all computing nodes in the neural network through an infection algorithm;

[0048] A first determining unit, configured to determine the computing node as the computing shape node if all inputs of the computing node are shape information;

[0049] The second determining unit is used to determine the computing node as the computing tensor node if all inputs of the computing node are not shape information.

[0050] From the above technical solution, it can be known that the embodiment of the present invention discloses a method and device for allocating space of computing nodes, classifies each computing node of a neural network, obtains a plurality of computing shape nodes and a plurality of computing tensor nodes; merges the computing shape nodes related to each computing tensor node into a new node; inputs the data to be inferred into the neural network; deduces the storage space required for each computing node of the neural network to calculate the data to be inferred according to the attributes and computing characteristics of the computing tensor nodes, the attributes and computing characteristics of the new nodes, and the data to be inferred; and allocates the storage space to each computing node. It can be seen that in the embodiment of the present invention, the storage space corresponding to each computing node can be obtained by offline derivation before the execution of the inference engine, and the corresponding storage space can be allocated to each computing node before the calculation starts, which can save the time of opening up storage space during calculation, thereby improving the overall calculation efficiency of the inference engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without paying creative work.

[0052] Figure 1A flowchart of a method for allocating space of a computing node disclosed in an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of a calculation tensor node and a calculation shape node disclosed in an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of a computing node disclosed in an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a new node disclosed in an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of the structure of a space allocation device for a computing node disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The embodiment of the present invention discloses a method and device for allocating space of computing nodes, which classifies each computing node of a neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes; merges the computing shape nodes related to each computing tensor node into a new node; inputs the data to be inferred into the neural network; deduces the storage space required for each computing node of the neural network to calculate the data to be inferred based on the attributes and computing characteristics of the computing tensor nodes, the attributes and computing characteristics of the new nodes, and the data to be inferred; and allocates the storage space to each computing node. It can be seen that in the embodiment of the present invention, the storage space corresponding to each computing node can be obtained by offline derivation before the execution of the inference engine, and the corresponding storage space can be allocated to each computing node before the calculation starts, which can save the time of opening up storage space during calculation, thereby improving the overall calculation efficiency of the inference engine.

[0059] See also Figure 1 , an embodiment of the present invention discloses a method for allocating space of computing nodes, the method comprising:

[0060] Step S101, classifying each computing node of the neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes;

[0061] See also Figure 2, is a schematic diagram of a computing tensor node and a computing shape node disclosed in an embodiment of the present invention. A computing tensor node refers to a computing node whose input includes a tensor input and a shape input, and a computing shape node refers to a computing node whose input is all shape input.

[0062] It should be noted that in the embodiment of the present invention, some prior knowledge can be used to classify some computing nodes in advance, such as directly dividing operators such as Conv convolution and MatMul matrix multiplication into computing tensor nodes, and Shape (extracting the shape of computing tensor nodes) into computing shape nodes, and then traversing all computing nodes in the entire network according to the contagion algorithm to divide the computing nodes into two categories. It should be noted that the above-mentioned operators refer to computing nodes.

[0063] Step S102, merging the calculation shape nodes associated with each of the calculation tensor nodes into a new node; the new node includes the attributes and topological order of the calculation shape nodes associated with each of the calculation tensor nodes;

[0064] It should be noted that since the new node contains the attributes and topological order of the calculation shape node related to each calculation tensor node, the new node can calculate the shape.

[0065] Step S103, inputting the data to be inferred into the neural network;

[0066] Step S104, deriving the storage space required by each computing node of the neural network to calculate the data to be inferred according to the attributes and computing characteristics of the computing tensor node, the attributes and computing characteristics of the new node, and the data to be inferred;

[0067] It should be noted that, in the embodiment of the present invention, the shape of the output can be deduced by the shape of the computing node input, and then the shape calculation of each computing node is performed in turn according to the topological order of the neural network, so that the output shape of any node of the entire neural network can be obtained. In this way, the storage space required by each computing node of the neural network to calculate the data to be inferred can be obtained through the output shape of each computing node.

[0068] It should be noted that for networks whose input is of indefinite length and needs to be batched to improve the efficiency of neural network reasoning, after different input lengths are batched, all inputs will be aligned to the longest input length. Therefore, after the whole is sent to the network for calculation, the output end cannot obtain the corresponding output size of different inputs sent from the same batch, and thus the effective output cannot be obtained. In this case, it is necessary to perform offline shape calculation on the shape of each input to obtain the output shape corresponding to the different input shapes, so as to obtain the corresponding output valid value. In the embodiment of the present invention, the corresponding storage space required when each computing node of the neural network calculates different data to be inferred (different input shapes) can be derived according to the properties and calculation characteristics of the calculation tensor nodes, the properties and calculation characteristics of the new nodes, and the different data to be inferred (different input shapes). In this way, the output size of the indefinite length input after batching is realized offline, so that the required memory size can be opened up in advance, and the real shape information is provided for subsequent calculations.

[0069] Step S105: Allocate the storage space to each computing node.

[0070] It should be noted that after allocating storage space to each computing node, the inference engine can be used to load the neural network and perform inference prediction on the inference data. In this way, the inference engine does not need to spend time determining the memory or video memory size required for calculation of each computing node, which can improve the overall computing efficiency of the inference engine.

[0071] It should be noted that the embodiments of the present invention can design the above-mentioned offline computational shape derivation method based on the existing general neural network description format Open Neural Network Exchange (ONNX), that is, the offline storage space derivation method. Among them, ONNX can represent dynamic neural network calculation graphs and support variable-length input. Therefore, it is more practical and applicable to design a set of offline computational shape derivation methods for ONNX.

[0072] The embodiment of the present invention discloses a method for allocating space of computing nodes, which classifies each computing node of a neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes; merges the computing shape nodes related to each computing tensor node into a new node; inputs the data to be inferred into the neural network; deduces the storage space required for each computing node of the neural network to calculate the data to be inferred based on the attributes and computing characteristics of the computing tensor nodes, the attributes and computing characteristics of the new nodes, and the data to be inferred; and allocates the storage space to each computing node. It can be seen that in the embodiment of the present invention, the storage space corresponding to each computing node can be obtained by offline derivation before the execution of the inference engine, and the corresponding storage space can be allocated to each computing node before the calculation starts, which can save the time of opening up storage space during calculation, thereby improving the overall calculation efficiency of the inference engine.

[0073] In order to further optimize the above embodiment, in the space allocation method for computing nodes provided in the embodiment of the present invention, step S101 may specifically include:

[0074] Step S1011, traversing all computing nodes in the neural network through the contagion algorithm;

[0075] Step S1012: if all inputs of the computing node are shape information, the computing node is determined as the computing shape node;

[0076] Step S1013: If all inputs of the computing node are not shape information, the computing node is determined as the computing tensor node.

[0077] See also Figure 3 , is a schematic diagram of a computing node disclosed in an embodiment of the present invention. For some computing nodes, the type of the computing node can be obtained through prior knowledge, and for other computing nodes, the type is determined based on their input type. Generally, if the input of a computing node is connected to the output of a Shape computing node or the output of a Shape computing node, then the computing node is considered to be a Shape computing node. Figure 3 If the input of the Gather computing node shown in is the output of the Shape computing node, then the Gather computing node is a shape computing node, and the input of the Mul computing node is the output of the Gather computing node, then the Mul computing node is a shape computing node. Otherwise, as long as there is tensor type data passed in, it must be a tensor computing node.

[0078] In the embodiment of the present invention, all computing nodes in the neural network are traversed by the contagion algorithm, and the computing nodes of the entire neural network are accurately divided into computing shape nodes and computing tensor nodes, which is convenient for subsequent calculations.

[0079] In order to further optimize the above embodiment, in the space allocation method for computing nodes provided in the embodiment of the present invention, step S102 may specifically include:

[0080] Step S1021, traversing all the calculation tensor nodes in the neural network in reverse order to obtain all calculation shape nodes related to the input of each calculation tensor node;

[0081] Step S1022: traverse and record all the calculation shape nodes by using the contagion algorithm until a calculation shape node for obtaining the tensor shape corresponding to each of the calculation tensor nodes is found;

[0082] It should be noted that the calculation shape node for obtaining the tensor shape corresponding to the calculation tensor node refers to the Shape node.

[0083] Step S1023, obtaining a calculation shape set related to each of the calculation tensor nodes according to all the calculation shape nodes related to the input of the calculation tensor node and the calculation shape node used to obtain the tensor shape;

[0084] It should be noted that in the embodiment of the present invention, based on the input-related nodes of the calculation tensor node and the calculation shape nodes used to obtain the tensor shape, all the calculation shape nodes between the calculation tensor node and the calculation shape nodes used to obtain the tensor shape are intercepted to obtain a calculation shape set related to each calculation tensor node.

[0085] Step S1024: Store the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node.

[0086] It should be noted that the above step S1024 may specifically include:

[0087] Step a, generate a new node through the generate node method provided by Open Neural Network Exchange ONNX;

[0088] Step b: Generate a Json string according to the attributes and topological order of each calculation shape node in the calculation shape set through the Onnx2Json tool;

[0089] Step c: Store the Json string in the new node.

[0090] It should be noted that, in the embodiment of the present invention, for new nodes, the internal implementation is mainly to obtain a network of calculation shape based on the newly generated operator attributes, that is, the Json string, and to rebuild a new execution network based on the Json string. All the computing nodes in the network are inserted into the execution queue in the existing order. After obtaining the input, the input is sent to the execution queue, and the computing nodes in the queue are executed in turn to obtain the final output of the network.

[0091] See also Figure 4 , which is a schematic diagram of a new node disclosed in an embodiment of the present invention. In an embodiment of the present invention, the fusion of computing nodes of the classified neural network is mainly to fuse all the computing shape nodes into a large CalcShape operator, that is, Figure 3 The OnnxlnferCalcShape fusion operator in the CalcShape operator is used to obtain a new node. The main process is to intercept all the calculation shape nodes between the calculation tensor node and the Shape calculation node, generate a json file, and add the generated json file to the CalcShape operator through attributes. Among them, the json file contains the attributes and topological order of all the calculation shape nodes between the calculation tensor node and the Shape calculation node, which can meet the needs of subsequent calculations.

[0092] In an embodiment of the present invention, the computational shape nodes associated with each computational tensor node are found by traversal, and then the computational shape nodes associated with each computational tensor node are merged into a new node through the node generation method and Onnx2Json tool provided by ONNX, so that the new node can be used to obtain the output shape of any computational node of the entire neural network.

[0093] In order to further optimize the above embodiment, the space allocation method for computing nodes provided in the embodiment of the present invention, after step S1024, further includes:

[0094] Step S1025, deleting all the calculated shape nodes traversed and recorded by the contagion algorithm;

[0095] It should be noted that the calculation shape nodes deleted here do not include the calculation shape nodes corresponding to each calculation tensor node for obtaining the tensor shape.

[0096] Step S1026: insert the new node into the neural network in topological order.

[0097] In the embodiment of the present invention, the integrity and correctness of the neural network are ensured by deleting the calculation shape nodes of the original neural network and sequentially inserting the new nodes into the original neural network.

[0098] See also Figure 5, a schematic diagram of a structure of a space allocation device for a computing node disclosed in an embodiment of the present invention, the device comprising:

[0099] A classification unit 501 is used to classify each computing node of the neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes;

[0100] A fusion unit 502, configured to fuse the calculation shape nodes associated with each of the calculation tensor nodes into a new node; the new node includes the attributes and topological order of the calculation shape nodes associated with each of the calculation tensor nodes;

[0101] An input unit 503, used to input the data to be inferred into the neural network;

[0102] A derivation unit 504, configured to derive the storage space required by each computing node of the neural network to calculate the data to be inferred according to the attributes and computing characteristics of the computing tensor node, the attributes and computing characteristics of the new node, and the data to be inferred;

[0103] The allocating unit 505 is configured to allocate the storage space to each computing node.

[0104] The embodiment of the present invention discloses a space allocation device for computing nodes, which classifies each computing node of a neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes; merges the computing shape nodes related to each computing tensor node into a new node; inputs the data to be inferred into the neural network; deduces the storage space required for each computing node of the neural network to calculate the data to be inferred based on the attributes and computing characteristics of the computing tensor nodes, the attributes and computing characteristics of the new nodes, and the data to be inferred; and allocates the storage space to each computing node. It can be seen that in the embodiment of the present invention, the storage space corresponding to each computing node can be obtained by offline derivation before the execution of the inference engine, and the corresponding storage space can be allocated to each computing node before the calculation starts, which can save the time of opening up storage space during calculation, thereby improving the overall calculation efficiency of the inference engine.

[0105] In order to further optimize the above embodiment, in the space allocation device for computing nodes provided in the embodiment of the present invention, the fusion unit 502 includes:

[0106] A first traversal unit, used for traversing all the computing tensor nodes in the neural network in reverse order to obtain all computing shape nodes related to the input of each computing tensor node;

[0107] A second traversal unit, used for traversing and recording all the calculation shape nodes by using an infection algorithm, until a calculation shape node for obtaining a tensor shape corresponding to each of the calculation tensor nodes is found;

[0108] A collection unit, used for obtaining a calculation shape set related to each of the calculation tensor nodes according to all the calculation shape nodes related to the input of the calculation tensor node and the calculation shape node for obtaining the tensor shape;

[0109] The fusion subunit is used to store the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node.

[0110] In order to further optimize the above embodiment, the space allocation device for computing nodes provided in the embodiment of the present invention further includes:

[0111] A deleting unit, used to delete all the calculated shape nodes traversed and recorded by the contagion algorithm;

[0112] An inserting unit is used to insert the new node into the neural network in a topological order.

[0113] In order to further optimize the above embodiment, in the space allocation device for computing nodes provided in the embodiment of the present invention, the fusion subunit includes:

[0114] A first generation unit, used for generating a new node by using a generation node method provided by the open neural network exchange ONNX;

[0115] A second generating unit is used to generate a Json string according to the attributes and topological order of each calculation shape node in the calculation shape set by using an Onnx2Json tool;

[0116] A storage unit is used to store the Json string in the new node.

[0117] In order to further optimize the above embodiment, in the space allocation device for computing nodes provided in the embodiment of the present invention, the classification unit 501 includes:

[0118] A third traversal unit, used for traversing all computing nodes in the neural network through an infection algorithm;

[0119] A first determining unit, configured to determine the computing node as the computing shape node if all inputs of the computing node are shape information;

[0120] The second determining unit is used to determine the computing node as the computing tensor node if all inputs of the computing node are not shape information.

[0121] It should be noted that the specific working principles of each component in the device embodiment can be found in the corresponding part of the method embodiment, which will not be repeated here.

[0122] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0123] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0124] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for spatial allocation of computing nodes, characterized in that: The method comprises: Classify each computing node of the neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes; Merging the computation shape nodes associated with each computation tensor node into a new node; the new node includes attributes and a topological order of the computation shape nodes associated with each computation tensor node; Inputting the data to be inferred into the neural network; Derived, based on the properties and computing characteristics of the computing tensor node, the properties and computing characteristics of the new node, and the data to be inferred, the storage space required for each computing node of the neural network to calculate the data to be inferred; Allocating the storage space to each computing node; The step of fusing the calculation shape nodes associated with each of the calculation tensor nodes into a new node comprises: Traversing all the computation tensor nodes in the neural network in reverse order to obtain all computation shape nodes associated with the input of each computation tensor node; Traversing and recording all the calculation shape nodes by using the contagion algorithm until finding the calculation shape node for obtaining the tensor shape corresponding to each of the calculation tensor nodes; Obtaining a calculation shape set related to each of the calculation tensor nodes according to all the calculation shape nodes related to the input of the calculation tensor node and the calculation shape node for obtaining the tensor shape; The attributes and topological order of all the computational shape nodes in the computational shape set are stored in a new node.

2. The method according to claim 1, characterized in that After storing the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node, the method further includes: Deleting all the calculated shape nodes traversed and recorded by the infection algorithm; The new nodes are inserted into the neural network in topological order.

3. The method according to claim 1, characterized in that The storing the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node includes: Generate a new node using the Generate Node method provided by Open Neural Network Exchange ONNX; Generate a Json string according to the attributes and topological order of each calculation shape node in the calculation shape set through the Onnx2Json tool; Store the Json string into the new node.

4. The method according to claim 1, characterized in that: The computing nodes of the neural network are classified to obtain computing shape nodes and computing tensor nodes, including: Traversing all computing nodes in the neural network through an infection algorithm; If all inputs of the computing node are shape information, determining the computing node as the computing shape node; If all inputs of the computing node are not shape information, the computing node is determined to be the computing tensor node.

5. A spatial allocation device for computing nodes, characterized in that: The device comprises: A classification unit, used for classifying each computing node of the neural network to obtain a plurality of computing shape nodes and a plurality of computing tensor nodes; A fusion unit, used for fusing the calculation shape nodes associated with each of the calculation tensor nodes into a new node; the new node contains the attributes and topological order of the calculation shape nodes associated with each of the calculation tensor nodes; An input unit, used for inputting data to be inferred into the neural network; A derivation unit, used to derive the storage space required for each computing node of the neural network to calculate the data to be inferred according to the attributes and computing characteristics of the computing tensor node, the attributes and computing characteristics of the new node and the data to be inferred; An allocating unit, configured to allocate the storage space to each computing node; The fusion unit comprises: A first traversal unit, used for traversing all the computing tensor nodes in the neural network in reverse order to obtain all computing shape nodes related to the input of each computing tensor node; A second traversal unit, used for traversing and recording all the calculation shape nodes by using an infection algorithm, until a calculation shape node for obtaining a tensor shape corresponding to each of the calculation tensor nodes is found; A collection unit, used for obtaining a calculation shape set related to each of the calculation tensor nodes according to all the calculation shape nodes related to the input of the calculation tensor node and the calculation shape node for obtaining the tensor shape; The fusion subunit is used to store the attributes and topological order of all the calculation shape nodes in the calculation shape set into a new node.

6. The device according to claim 5, characterized in that The device also includes: A deleting unit, used to delete all the calculated shape nodes traversed and recorded by the contagion algorithm; An inserting unit is used to insert the new node into the neural network in a topological order.

7. The device according to claim 5, characterized in that The fusion subunit comprises: A first generation unit, used for generating a new node by using a generation node method provided by the open neural network exchange ONNX; A second generating unit is used to generate a Json string according to the attributes and topological order of each calculation shape node in the calculation shape set by using an Onnx2Json tool; A storage unit is used to store the Json string in the new node.

8. The device according to claim 5, characterized in that The taxonomic units include: A third traversal unit, used for traversing all computing nodes in the neural network through an infection algorithm; A first determining unit, configured to determine the computing node as the computing shape node if all inputs of the computing node are shape information; The second determining unit is used to determine the computing node as the computing tensor node if all inputs of the computing node are not shape information.