Computational graph serialization method, device and storage medium
By determining the computational subgraph in the computational graph and generating the target computational sequence, the problems of wasted hardware resources and reduced inference speed caused by computational graph serialization are solved, and more efficient model inference is achieved.
Patent Information
- Application Number
- CN202311551613.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-20
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-11-20
AI Technical Summary
The existing computational graph serialization method leads to waste of hardware resources and reduces the speed of model inference in the inference stage.
The first processor obtains the calculation diagram of the target model, determines the calculation sub-graph, and generates the target calculation sequence based on the dominant tree and the latter dominant tree, so that it occupies a small amount of memory when executed on the second processor, and realizes the coordinated calculation of software and hardware. Graph serialization, search for the computed sequence with the smallest memory.
Save hardware resources and improve the inference speed of the target model on the second processor.
Smart Images

Figure CN117540808B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method, device and storage medium for serializing a computational graph. Background Art
[0002] With the continuous development of artificial intelligence (AI) technology, deep learning neural network models have been widely used in many fields and have achieved very good results. The intermediate layer representation of the deep learning neural network model adopts a data representation method called a directed acyclic graph called a computational graph. The computational graph is used to describe the dependencies between each operation in the neural network model, helping us to visualize and understand the structure of the neural network. The serialization of the computational graph refers to the conversion of the computational graph into a format that can be stored or transmitted. By serializing the computational graph, the computational graph can be saved to disk, or transferred and loaded between different devices, so that we can easily save and share the model and speed up the process of model reasoning.
[0003] Currently, when serializing computational graphs, there is still a problem of wasting hardware resources in the inference stage and reducing the model inference speed. Therefore, a new computational graph serialization method is urgently needed to save hardware resources and improve the model inference speed. Summary of the invention
[0004] In view of this, the present disclosure proposes a computational graph serialization method, device, and storage medium.
[0005] According to one aspect of the present disclosure, a method for serializing a computational graph is provided. The method comprises:
[0006] The first processor obtains a computation graph of the target model;
[0007] The first processor determines a computation subgraph in the computation graph;
[0008] The first processor generates a target computation sequence based on the computation subgraph. The target computation sequence is a computation sequence corresponding to the computation graph that occupies the least memory of the second processor when executed on the second processor. The target computation sequence is used to be executed on the second processor to achieve reasoning based on the target model.
[0009] In one possible implementation, the computation graph is a directed computation graph, and the computation subgraphs include a first computation subgraph, each node in the first computation subgraph has only one parent node and one child node, and the memory size occupied by the nodes in the first computation subgraph is monotonically increasing or monotonically decreasing according to the connection order.
[0010] In a possible implementation, the computation subgraph includes a second computation subgraph, and determining the computation subgraph in the computation graph includes:
[0011] The dominator tree of the computation graph is constructed starting from the input node of the computation graph, and the post-dominator tree of the computation graph is constructed starting from the output node of the computation graph;
[0012] The preset nodes in the computation graph are used as initial nodes in the second computation subgraph, and the second computation subgraph is determined based on the dominator tree and the post-dominator tree.
[0013] In a possible implementation, determining the second computation subgraph based on the dominator tree and the post-dominator tree includes:
[0014] Determine, based on the dominator tree, other second computational subgraphs to which the initial node in the second computational subgraph can be output according to the connection order;
[0015] In response to satisfying a first preset condition, nodes in other second computing subgraphs and nodes on a connection path from an initial node in a dominating tree to nodes in the other second computing subgraphs are added to the second computing subgraph, and deleted from the other second computing subgraphs to determine a second computing subgraph, wherein the first preset condition is that the memory occupied by the nodes in the other second computing subgraphs is less than the memory occupied by the initial nodes of the second computing subgraph.
[0016] In a possible implementation, determining the second computation subgraph based on the dominator tree and the post-dominator tree includes:
[0017] Based on the dominator tree, determine the output node and the output boundary node in the second computation subgraph, all the child nodes of the output node in the dominator tree are outside the second computation subgraph, and at least one child node of the output boundary node in the dominator tree is outside the second computation subgraph;
[0018] Based on the post-dominator tree, input nodes and input boundary nodes in the second computational subgraph are determined, all parent nodes of the input nodes in the post-dominator tree are outside the second computational subgraph, and at least one parent node of the input boundary node in the post-dominator tree is outside the second computational subgraph.
[0019] In a possible implementation, the preset node has multiple parent nodes and only one child node.
[0020] In a possible implementation, generating a target computation sequence based on a computation subgraph includes:
[0021] Generate an initial computation sequence based on the computation subgraph;
[0022] In response to the initial calculation sequence satisfying the second preset condition, the initial calculation sequence is used as the target calculation sequence.
[0023] In a possible implementation, generating a target computing sequence based on a computing subgraph further includes:
[0024] In response to the initial calculation sequence not satisfying the second preset condition, determining at least one node in the initial calculation sequence, the at least one node comprising a node with a peak memory usage and its associated nodes;
[0025] For each node associated with the node with the peak memory usage, if the child node of the node is a node in the second computational subgraph, the second computational subgraph is removed, and the input nodes, input boundary nodes, output nodes, and objects pointed to by output boundary nodes in the second computational subgraph are re-determined to update the computational subgraph;
[0026] Based on the updated computation subgraph, an initial computation sequence is generated, and steps after the initial computation sequence is generated are executed.
[0027] In a possible implementation, the second preset condition includes that the peak memory of the initial calculation sequence converges to a preset value, and / or the number of times the initial calculation sequence is generated reaches a preset threshold.
[0028] In a possible implementation, an initial computation sequence is generated based on the computation subgraph, including:
[0029] Determine the nodes with in-degree 0 in the computation graph, and for each node with in-degree 0, traverse each node in the computation graph based on the node with in-degree 0;
[0030] When the node is a node in the first computation subgraph, adding the node in the first computation subgraph corresponding to the node to the initial computation sequence;
[0031] When the node is a node in a second computation subgraph, in the second computation subgraph corresponding to the node, based on input nodes, input boundary nodes, output nodes, and output boundary nodes in the second computation subgraph, a computation sequence of the second computation subgraph is determined by a preset method and added to the initial computation sequence;
[0032] When the node is neither a node in the first computation subgraph nor a node in the second computation subgraph, adding the node to the initial computation sequence based on a preset method;
[0033] The nodes that have been added to the initial calculation sequence are removed from the calculation graph. If the number of nodes with in-degree 0 in the calculation graph is not 0, the nodes with in-degree 0 in the calculation graph are determined again. For each node with in-degree 0, the nodes in the calculation graph and the subsequent steps are traversed based on the node with in-degree 0 to generate the initial calculation sequence.
[0034] According to another aspect of the present disclosure, a computation graph serialization device is provided. The device includes:
[0035] An acquisition module, used for the first processor to acquire a calculation graph of the target model;
[0036] A determination module, used for the first processor to determine a computation subgraph in the computation graph;
[0037] A generation module is used for the first processor to generate a target calculation sequence based on the calculation subgraph. The target calculation sequence is the calculation sequence corresponding to the calculation graph, which occupies the least memory of the second processor when executed on the second processor. The target calculation sequence is used to be executed on the second processor to realize reasoning based on the target model.
[0038] In one possible implementation, the computation graph is a directed computation graph, and the computation subgraphs include a first computation subgraph, each node in the first computation subgraph has only one parent node and one child node, and the memory size occupied by the nodes in the first computation subgraph is monotonically increasing or monotonically decreasing according to the connection order.
[0039] In a possible implementation, the computation subgraph includes a second computation subgraph, a determination module, configured to:
[0040] The dominator tree of the computation graph is constructed starting from the input node of the computation graph, and the post-dominator tree of the computation graph is constructed starting from the output node of the computation graph;
[0041] The preset nodes in the computation graph are used as initial nodes in the second computation subgraph, and the second computation subgraph is determined based on the dominator tree and the post-dominator tree.
[0042] In a possible implementation, determining the second computation subgraph based on the dominator tree and the post-dominator tree includes:
[0043] Determine, based on the dominator tree, other second computational subgraphs to which the initial node in the second computational subgraph can be output according to the connection order;
[0044] In response to satisfying a first preset condition, nodes in other second computing subgraphs and nodes on a connection path from an initial node in a dominating tree to nodes in the other second computing subgraphs are added to the second computing subgraph, and deleted from the other second computing subgraphs to determine a second computing subgraph, wherein the first preset condition is that the memory occupied by the nodes in the other second computing subgraphs is less than the memory occupied by the initial nodes of the second computing subgraph.
[0045] In a possible implementation, determining the second computation subgraph based on the dominator tree and the post-dominator tree includes:
[0046] Based on the dominator tree, determine the output node and the output boundary node in the second computation subgraph, all the child nodes of the output node in the dominator tree are outside the second computation subgraph, and at least one child node of the output boundary node in the dominator tree is outside the second computation subgraph;
[0047] Based on the post-dominator tree, input nodes and input boundary nodes in the second computational subgraph are determined, all parent nodes of the input nodes in the post-dominator tree are outside the second computational subgraph, and at least one parent node of the input boundary node in the post-dominator tree is outside the second computational subgraph.
[0048] In a possible implementation, the preset node has multiple parent nodes and only one child node.
[0049] In a possible implementation, a module is generated to:
[0050] Generate an initial computation sequence based on the computation subgraph;
[0051] In response to the initial calculation sequence satisfying the second preset condition, the initial calculation sequence is used as the target calculation sequence.
[0052] In a possible implementation, generating a target computing sequence based on a computing subgraph further includes:
[0053] In response to the initial calculation sequence not satisfying the second preset condition, determining at least one node in the initial calculation sequence, the at least one node comprising a node with a peak memory usage and its associated nodes;
[0054] For each node associated with the node with the peak memory usage, if the child node of the node is a node in the second computational subgraph, the second computational subgraph is removed, and the input nodes, input boundary nodes, output nodes, and objects pointed to by output boundary nodes in the second computational subgraph are re-determined to update the computational subgraph;
[0055] Based on the updated computation subgraph, an initial computation sequence is generated, and steps after the initial computation sequence is generated are executed.
[0056] In a possible implementation, the second preset condition includes that the peak memory of the initial calculation sequence converges to a preset value, and / or the number of times the initial calculation sequence is generated reaches a preset threshold.
[0057] In a possible implementation, an initial computation sequence is generated based on the computation subgraph, including:
[0058] Determine the nodes with in-degree 0 in the computation graph, and for each node with in-degree 0, traverse each node in the computation graph based on the node with in-degree 0;
[0059] When the node is a node in the first computation subgraph, adding the node in the first computation subgraph corresponding to the node to the initial computation sequence;
[0060] When the node is a node in a second computation subgraph, in the second computation subgraph corresponding to the node, based on input nodes, input boundary nodes, output nodes, and output boundary nodes in the second computation subgraph, a computation sequence of the second computation subgraph is determined by a preset method and added to the initial computation sequence;
[0061] When the node is neither a node in the first computation subgraph nor a node in the second computation subgraph, adding the node to the initial computation sequence based on a preset method;
[0062] The nodes that have been added to the initial calculation sequence are removed from the calculation graph. If the number of nodes with in-degree 0 in the calculation graph is not 0, the nodes with in-degree 0 in the calculation graph are determined again. For each node with in-degree 0, the nodes in the calculation graph and the subsequent steps are traversed based on the node with in-degree 0 to generate the initial calculation sequence.
[0063] According to another aspect of the present disclosure, a computation graph serialization device is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.
[0064] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.
[0065] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0066] According to an embodiment of the present application, by enabling the first processor to obtain a computational graph of the target model, a target computational sequence is generated based on a computational subgraph in the computational graph, and the target computational sequence is made to be a computational sequence corresponding to the computational graph that occupies the smallest memory of the second processor when executed on the second processor. By serializing the computational graph in collaboration with software and hardware and searching for the computational sequence with the smallest memory, hardware resources can be saved and the inference speed of the target model on the second processor can be improved.
[0067] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
[0069] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present application.
[0070] Figure 2 A flowchart of a computational graph serialization method according to an embodiment of the present application is shown.
[0071] Figure 3 A flowchart of a computational graph serialization method according to an embodiment of the present application is shown.
[0072] Figure 4 A flowchart of a computational graph serialization method according to an embodiment of the present application is shown.
[0073] Figure 5 A flowchart of a computational graph serialization method according to an embodiment of the present application is shown.
[0074] Figure 6 A schematic diagram showing the memory usage of a computation graph according to an embodiment of the present application.
[0075] Figure 7 A schematic diagram showing the memory usage of a computation graph according to an embodiment of the present application.
[0076] Figure 8 A structural diagram of a computational graph serialization device according to an embodiment of the present application is shown.
[0077] Fig. 9 It is a block diagram of a device 1900 for performing computation graph serialization according to an exemplary embodiment. DETAILED DESCRIPTION
[0078] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0079] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0080] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.
[0081] With the continuous development of artificial intelligence (AI) technology, deep learning neural network models have been widely used in many fields and have achieved very good results. The intermediate layer representation of the deep learning neural network model adopts a data representation method called a directed acyclic graph called a computational graph. The computational graph is used to describe the dependencies between each operation in the neural network model, helping us to visualize and understand the structure of the neural network. The serialization of the computational graph refers to converting the computational graph into a format that can be stored or transmitted. By serializing the computational graph, the computational graph can be saved to disk or transferred and loaded between different devices, so that we can easily save and share the model and speed up the model reasoning process. At present, when the computational graph is serialized, there is still a waste of hardware resources in the reasoning stage and a reduction in the model reasoning speed. Therefore, a new method of computational graph serialization is urgently needed to save hardware resources and improve the model reasoning speed.
[0082] In view of this, the present application provides a method, device and storage medium for serializing a computation graph. The method of the embodiment of the present application enables a first processor to obtain a computation graph of a target model, generates a target computation sequence based on a computation subgraph in the computation graph, and makes the target computation sequence the computation sequence that occupies the smallest memory of the second processor when executed on the second processor among the computation sequences corresponding to the computation graph. The computation graph serialization can be performed considering the collaboration of software and hardware, and the computation sequence with the smallest memory can be searched, which can save hardware resources and improve the inference speed of the target model on the second processor.
[0083] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present application is shown. Figure 1 As shown, the calculation graph serialization method of the embodiment of the present application can be applied to the first processor. The first processor obtains the calculation graph of the target model, and the calculation graph can be serialized based on the method of the embodiment of the present application to obtain the target calculation sequence that occupies the minimum memory of the second processor and send it to the second processor. The first processor can be any type of processor including a CPU, and the second processor can be a neural network processing unit (NPU), so that the reasoning of the target model (such as a deep learning neural network model) can be implemented on the second processor based on the target calculation sequence. The second processor can also be other types of processors that can execute the target calculation sequence and implement the reasoning of the neural network model.
[0084] The first processor (or the second processor) can be deployed in a terminal device or a server. The on-chip network configuration method of the embodiment of the present application can be applied to a terminal device or a server. Among them, the terminal device can be any one or more of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a cellular phone, a personal digital assistant (personal digital assistant, PDA), and a vehicle-mounted device. The embodiment of the present application does not impose any special restrictions on the specific type of the terminal device, and it can have a wired or wireless communication function.
[0085] The server can be located locally or in the cloud, and can be a physical device or a virtual device, such as a virtual machine, a container, etc., and has a wireless communication function, wherein the wireless communication function can be set in the chip (system) or other parts or components of the server. The wireless communication function can be realized, for example, through mobile communication technologies such as 2G / 3G / 4G / 5G, as well as Wi-Fi, Bluetooth, frequency modulation (FM), digital radio, satellite communication, etc. Communication can also be carried out through wired connection to achieve interaction with other devices.
[0086] The following Figure 2-Figure 7 , the computational graph serialization method of the embodiment of the present application is introduced.
[0087] Figure 2 A flowchart of a method for serializing a computational graph according to an embodiment of the present application is shown. Figure 2 As shown, the method may include:
[0088] Step S201: The first processor obtains a computation graph of a target model.
[0089] An example of the first processor can be found in Figure 1 , the target model can be a neural network model. The computational graph is a directed computational graph, which can be used to represent the middle layer of the neural network model. Each node in the computational graph represents a computational operation, and the edge can represent the direction of data flow. Based on relevant technologies, the computational graph of the target model can be obtained.
[0090] Step S202: The first processor determines a computation subgraph in the computation graph.
[0091] Subsequently, dynamic programming can be used to globally search for the computation sequence with the minimum memory in the computation graph. Since the time complexity of dynamic programming is relatively large, the computation graph can be split first, and multiple computation subgraphs can be determined in the computation graph. When the computation sequence is subsequently obtained and executed on the second processor, the computation operations corresponding to the nodes in the same computation subgraph can be assigned to the same processor, and different computation subgraphs can be executed in parallel. The memory occupied by nodes with an in-degree of 0 in the computation graph can be estimated based on the size of the input data, and the memory size occupied by other nodes in the computation graph can be determined based on their position in the computation sequence and the size of the data input to their predecessor nodes and the size of the data output by their predecessor nodes. This application does not limit the method of determining the size of memory occupied by a node.
[0092] The computation subgraphs may be divided into multiple types, for example, they may include a first computation subgraph and a second computation subgraph, and they may also include other computation subgraphs other than the first computation subgraph and the second computation subgraph.
[0093] The computation subgraph may include a first computation subgraph. The computation subgraph may be composed of nodes having only one parent node and one child node and a connection relationship in the computation subgraph, and it is determined whether the memory size occupied by the node on the second processor is monotonic (monotonically increasing or monotonically decreasing) according to the connection order in the computation subgraph when the computation operation corresponding to each node is executed on the second processor. If it is monotonic, it can be considered that in this subgraph, the node corresponding to the peak memory occupation is the first node (when the monotonicity is monotonically decreasing) or the last node (when the monotonicity is monotonically increasing). In subsequent searches, the nodes in this computation subgraph may not be searched to speed up the search.
[0094] The above-mentioned computing subgraph can be determined as the first computing subgraph, so that each node in the first computing subgraph has only one parent node and one child node, and the memory size occupied by the nodes in the first computing subgraph monotonically increases or decreases according to the connection order.
[0095] For example, by polling the computational graph, we find nodes A, B, and C. These nodes have only one parent node and one child node, and have a connection relationship (i.e., there are no isolated nodes that are not connected to other nodes in the first computational subgraph). If the connection order of the nodes is node A-node B-node C, and the node memory size is node A>node B>node C (or node A<node B<node C), then the node corresponding to the peak memory usage is node A (or node C). These nodes can form the first computational subgraph.
[0096] The computation subgraph may further include a second computation subgraph. Figure 3 This section describes how to determine the second computation subgraph. Figure 3 , showing a flow chart of a method for serializing a computational graph according to an embodiment of the present application. Figure 3 As shown, S202 may include:
[0097] Step S301 , constructing a dominator tree of the computation graph starting from the input nodes of the computation graph, and constructing a post-dominator tree of the computation graph starting from the output nodes of the computation graph.
[0098] The input node of the computation graph can be the root node in the computation graph (that is, the node with in-degree 0), and the output node of the computation graph is the node with out-degree 0. In-degree refers to the number of edges associated with the node that end with it. An in-degree of 0 indicates that the node has no parent node; out-degree refers to the number of edges associated with the node that start with it. An out-degree of 0 indicates that the node has no child nodes.
[0099] In the dominator tree corresponding to the computation graph, the parent node has a dominator relationship with the child node. The dominator relationship means that all paths from the root node of the computation graph to the child node must pass through the parent node.
[0100] In the post-dominator tree corresponding to the computation graph, the parent node has a post-dominator relationship with the child node, and the post-dominator relationship means that all paths from the child node to the output node of the computation graph must pass through the parent node.
[0101] Step S302: Using a preset node in the computation graph as an initial node in a second computation subgraph, and determining a second computation subgraph based on a dominator tree and a post-dominator tree.
[0102] The preset node may be a node pre-specified in the computational graph, or a node determined in the process of determining and updating the second computational subgraph, for example, an output node of the updated second computational subgraph. Exemplarily, the preset node may have multiple parent nodes and only one child node, such as a concat-type node. Since concat-type nodes have multiple parent nodes, by using such nodes as preset nodes, a computational sequence that occupies less memory can be searched based on this.
[0103] In S302, it is possible to:
[0104] Based on the dominator tree, determine other second computational subgraphs to which the initial node in the second computational subgraph can be output in the connection order; in response to satisfying the first preset condition, add the nodes in the other second computational subgraphs and the nodes on the connection path from the initial node in the dominator tree to the nodes in the other second computational subgraphs to the second computational subgraphs, and delete them from the other second computational subgraphs to determine the second computational subgraph.
[0105] For example, a dominator tree based on the initial node as the root node can be obtained based on the dominator tree constructed starting from the input node of the computation graph, and the other second computation subgraphs to which the initial node in the second computation subgraph can be output in sequence can be determined according to the connection order in the dominator tree.
[0106] The first preset condition is that the memory occupied by the nodes in other second computing subgraphs is smaller than the memory occupied by the initial node of the second computing subgraph.
[0107] For example, based on the computation graph, the initial node Z can be found, and four nodes X, Y, Z, and N can be determined, where nodes X and Y are the parent nodes of node Z, and node N is the child node of Z. An initial second computation subgraph can be constructed with the concat type node (node Z). Other initial second computation subgraphs can be constructed in the computation graph in the same manner.
[0108] From the above second computation subgraph (referred to as the second computation subgraph) Figure 1 ) starts from the initial node (i.e., the preset node Z), searches downward along the dominator tree, passes through node N, node O, node P, and finally outputs to the second computing sub Figure 2 Node Q. At this point, you can look for the second computation sub Figure 2 The memory occupied by the node in the dominator tree is less than the memory occupied by the node Z, and the node and the nodes on the connection path from the node N to the node in the dominator tree can be added to the second computing sub Figure 1 and the second calculator Figure 2 The node in the second computation Figure 2 Delete it.
[0109] For example, if you get the second operator Figure 2 If the memory occupied by node Q in the calculation is less than that of node Z, then nodes N, O, P, and Q on the output path of node Z can be added to the second calculation sub- Figure 1 , and from the second calculator Figure 2 Delete node Q. If in the second computation Figure 2 If we find that the memory occupied by nodes Q and S (for example, S is a child node of Q) is smaller than that of node Z, we can add nodes N, O, P, Q and S on the output path of node Z to the second computing subnode. Figure 1 , and in the second operator Figure 2 Delete nodes Q and S.
[0110] In some optional examples, when the second computation subgraph is determined by the above scheme, the output nodes of other incorporated second computation subgraphs can be updated to the initial nodes of the determined second computation subgraph. For example, when adding nodes N, O, P, and Q on the output path of node Z to the second computation subgraph, Figure 1 In the case of, the node Q can be determined as the initial node to perform the step of determining the second computational subgraph based on the initial node, the dominator tree and the post-dominator tree.
[0111] In S302, it is also possible to:
[0112] Based on the dominator tree, determine the output nodes and output boundary nodes in the second computation subgraph;
[0113] Based on the post-dominator tree, input nodes and input boundary nodes in the second computation subgraph are determined.
[0114] Among them, a dominating tree and a post-dominating tree with a preset node as a root node can be obtained. Based on the connection relationship between nodes in the dominating tree, the output node and the output boundary node in the second computational subgraph are determined among the nodes dominated by the preset node, all the child nodes of the output node in the dominating tree are outside the second computational subgraph, and at least one child node of the output boundary node in the dominating tree is outside the second computational subgraph. Based on the connection relationship between nodes in the post-dominating tree, the input node and the input boundary node in the second computational subgraph can be determined, all the parent nodes of the input node in the post-dominating tree are outside the second computational subgraph, and at least one parent node of the input boundary node in the post-dominating tree is outside the second computational subgraph.
[0115] The output nodes and output boundary nodes may be made to point to the second computational subgraph so as to be associated with the second computational subgraph, and the input nodes and input boundary nodes may be made to point to the second computational subgraph so as to be associated with the second computational subgraph.
[0116] Thus, the second computing subgraph, as well as the input node, input boundary node, output node, and output boundary node pointing to the second computing subgraph can be determined to determine the scope of the second computing subgraph corresponding to the node.
[0117] Step S203: The first processor generates a target computing sequence based on the computing subgraph.
[0118] The target computation sequence is a computation sequence corresponding to the computation graph, which occupies the least memory of the second processor when executed on the second processor, and the target computation sequence is used to execute on the second processor to implement reasoning based on the target model. The method for determining the target computation sequence can be seen below.
[0119] According to an embodiment of the present application, by enabling the first processor to obtain a computational graph of the target model, a target computational sequence is generated based on a computational subgraph in the computational graph, and the target computational sequence is made to be a computational sequence corresponding to the computational graph that occupies the smallest memory of the second processor when executed on the second processor. By serializing the computational graph in collaboration with software and hardware and searching for the computational sequence with the smallest memory, hardware resources can be saved and the inference speed of the target model on the second processor can be improved.
[0120] Figure 4 A flowchart of a method for serializing a computational graph according to an embodiment of the present application is shown. Figure 4 As shown, the S203 includes:
[0121] Step S401, generating an initial calculation sequence based on the calculation subgraph.
[0122] The method of generating the initial calculation sequence can be found in Figure 5 , showing a flow chart of a method for serializing a computational graph according to an embodiment of the present application. Figure 5 As shown, S401 includes:
[0123] Step S501, determine the nodes with in-degree 0 in the computation graph, and for each node with in-degree 0, traverse the nodes in the computation graph based on the node with in-degree 0.
[0124] The node with in-degree 0 may be the root node (i.e., input) in the computation graph. For each node with in-degree 0, it is stored in a preset container (which may be called zeroin vector) as the starting point for searching. For each node in the computation graph, a dictionary M for recording the memory data size currently occupied by the node on the second processor may be initialized. The key of the dictionary M may be the node with in-degree 0, and the value of the dictionary M may be the memory size occupied by the second processor under the current node.
[0125] Each node with in-degree 0 in the zeroin vector can be traversed, and based on each node with in-degree 0, each node in the computational graph can be traversed to determine whether the node belongs to a node in the first computational subgraph (which can be called group1), a node in the second computational subgraph (which can be called group2), or an ordinary node. Different search strategies can be applied to different types of nodes.
[0126] Step S502: When the node is a node in the first computation subgraph, add the node in the first computation subgraph corresponding to the node to the initial computation sequence.
[0127] Since the nodes in the first computing subgraph are connected one by one, and the occupied memory size is monotonic according to the connection order. At this time, there is no need to search the calculation sequence again in the first computing subgraph. The nodes in the first computing subgraph can be directly added to the initial calculation sequence one by one and the memory size occupied by each node in the second processor can be stored. This can reduce the search time. For example, the first computing subgraph includes node A, node B, and node C in the order of connection. Then, the order of the nodes in the first computing subgraph in the initial calculation sequence is node A, node B, and node C.
[0128] Step S503, when the node is a node in the second computing subgraph, in the second computing subgraph corresponding to the node, based on the input nodes, input boundary nodes, output nodes and output boundary nodes in the second computing subgraph, a computing sequence of the second computing subgraph is determined by a preset method and added to the initial computing sequence.
[0129] The preset method can be any one of pre-order traversal, in-order traversal, and post-order traversal.
[0130] The scope of the second computation subgraph corresponding to the node can be determined based on the input node, input boundary node, output node and output boundary node pointing to the second computation subgraph. In the second computation subgraph, a computation sequence is obtained by searching in a preset manner, and the memory size of the second processor occupied by each node in the computation sequence is stored.
[0131] For example, see Figure 6 , showing a schematic diagram of the memory usage of a computation graph according to an embodiment of the present application. Figure 6 In the example, according to the input node, input boundary node, output node and output boundary node, it can be determined that node1-node4 belong to the same second computational subgraph (node3 is the left node of node1, node4 is the right node of node3). Then when the node is any one of node1-node4, the computational sequence corresponding to the second computational subgraph can be obtained by traversing from the input node (node1, node2) based on the preset method. For example, when the preset method is pre-order traversal, the computational sequence node1, node3, node4, node2 can be obtained; when the preset method is in-order traversal, the computational sequence node3, node4, node1, node2 can be obtained; when the preset method is post-order traversal, the computational sequence node4, node3, node1, node2 can be obtained.
[0132] Step S504: when the node is neither a node in the first computing subgraph nor a node in the second computing subgraph, add the node to the initial computing sequence based on a preset method.
[0133] At this time, it can be considered as an ordinary node, so the position of the ordinary node and other nodes connected to it can be determined in a preset manner, and the ordinary node is added to the initial calculation sequence. For example, ordinary nodes can be searched based on any preset method of pre-order traversal, in-order traversal, and post-order traversal, and based on the connection relationship between the ordinary node and other nodes, the ordinary node A is added to the initial calculation sequence. For example, the current traversal to ordinary node A, if the preset method is post-order traversal, and node A is post-order connected to the left child node B and the right child node C, if nodes B and C are also ordinary nodes, then in the manner of post-order traversal, traverse nodes B and node C and add the nodes to the initial calculation sequence in the order of node B, node C, and node A. If the other nodes connected to the ordinary node are nodes in the first calculation subgraph or the second calculation subgraph, the method of step S502 and step S503 can be referred to, and the other nodes can be processed. In addition, the memory size occupied by the second processor under each node added to the initial calculation sequence can also be stored.
[0134] Step S505, remove the nodes that have been added to the initial calculation sequence from the calculation graph. When the number of nodes with in-degree 0 in the calculation graph is not 0, re-execute S501 and subsequent steps to generate an initial calculation sequence.
[0135] After each execution of S502 - S504 , S505 may be executed to update the zeroin vector, and S501 and subsequent steps may be re-executed until the zeroin vector is empty, thereby obtaining an initial calculation sequence.
[0136] After obtaining an initial calculation sequence, you can return to see Figure 4 , determine whether the initial calculation sequence meets the second preset condition to continue searching for a calculation sequence with a smaller memory:
[0137] Step S402: in response to the initial calculation sequence not satisfying the second preset condition, determining at least one node in the initial calculation sequence.
[0138] The second preset condition may include that the peak memory of the initial calculation sequence converges to a preset value. The second preset condition may also be that the number of times the initial calculation sequence is generated reaches a preset threshold, etc. The peak memory of the initial calculation sequence represents the memory value occupied by the node with the largest memory among the nodes of the initial calculation sequence.
[0139] The at least one node includes a node with a peak memory usage and its associated nodes. The associated nodes may include one or more nodes on an input path (ie, a preceding path) of the node with a peak memory usage.
[0140] An example can be found in Figure 6 , nodes node1~node4 can belong to a group2 (i.e., the second computational subgraph, such as subgraph1 in the figure), and node node8 belongs to a node in another computational subgraph (such as subgraph2 in the figure), where node1 can be a concat type node, and node1 has multiple parent nodes (including node9 and node10). According to the computation sequence obtained at this time, assuming that the memory occupied by the data input to node1 by node9 and node10 is 1 in total and the input is always passed to node4, the memory occupied by the data input to node2 is 1, and the memory occupied by the data output from node2 to node4 is 0.3. Since node2 belongs to group2 at this time, the data input to node2 will not be released until node4 is executed. At this time, assuming that the node with the peak memory occupation is node4, then the memory occupied by node4 at this time is 1 (input from node1) + 1 (input from node2) + 0.3 (output from node2), that is, the peak memory of the initial computation sequence is 2.3.
[0141] If the preset value is not 2.3, or the distance between 2.3 and the preset value does not meet the convergence condition, and / or the number of times the initial calculation sequence is generated does not reach the preset threshold, it is considered that the second preset condition is not met. Then the associated nodes can be determined, for example, one or more nodes on the input path (i.e., the preceding path) of the node with the peak memory usage. Figure 6 In the example, nodes associated with node4 may include node1 to node7, ie, nodes on the input path of node4.
[0142] Step S403, for each node associated with the node with the peak memory usage, if the child node of the node is a node in the second computing subgraph, the second computing subgraph is removed, and the input nodes, input boundary nodes, output nodes, and objects pointed to by the output boundary nodes in the second computing subgraph are re-determined to update the computing subgraph.
[0143] For example, in determining the node with the peak memory usage and its associated nodes, Figure 6 In the example, for each node, we can determine that the child nodes of node1 (or node2, node3) are nodes in group2, remove group2, and redefine the input nodes, input boundary nodes, output nodes, and output boundary nodes of group2 as ordinary nodes. The updated computational graph can be seen in Figure 7 , showing a schematic diagram of the memory usage of a computation graph according to an embodiment of the present application. Figure 7 As shown, in Figure 6On this basis, group2 corresponding to node1 to node4 is removed.
[0144] Step S404: Generate an initial calculation sequence based on the updated calculation subgraph, and execute steps after generating the initial calculation sequence.
[0145] Based on the method in S401, an initial calculation sequence corresponding to the calculation graph can be generated based on the updated calculation subgraph. Based on the initial calculation sequence, it can be determined whether it meets the second preset condition. When the second preset condition is not met, S402 and subsequent steps can be executed. When the second preset condition is met, S405 can be executed.
[0146] See also Figure 7 In the example, after removing group2, the data input to node2 will be released after node2 is executed, so the memory occupied by node4 is 1 (input from node1) + 0.3 (output from node2), that is, 1.3. Compared with before removing group2, the peak memory is significantly reduced.
[0147] Step S405 : in response to the initial calculation sequence satisfying the second preset condition, taking the initial calculation sequence as the target calculation sequence.
[0148] The initial calculation sequence may be the one obtained in S401, or the initial calculation sequence generated based on the updated calculation subgraph obtained in S404. Figure 6 , Figure 7 In the example, if the node corresponding to the peak memory is always node4, and after generating the initial calculation sequence multiple times, the memory occupied by node4 in the second processor is stabilized at 1.3, and the distance between the node4 and the preset value meets the condition, or the number of executions of step S401 has reached the threshold, then it can be considered that the initial calculation sequence meets the second preset condition, and the last generated initial calculation sequence can be used as the target calculation sequence.
[0149] The target calculation sequence may be stored and sent to a second processor to execute the target calculation sequence on the second processor to implement reasoning based on the target model.
[0150] Figure 8 FIG. 2 shows a structural diagram of a computation graph serialization device according to an embodiment of the present application. Figure 8 As shown, the device comprises:
[0151] An acquisition module 801 is used for a first processor to acquire a computation graph of a target model;
[0152] Determining module 802, used for the first processor to determine a computation subgraph in the computation graph;
[0153] Generation module 803 is used for the first processor to generate a target calculation sequence based on the calculation subgraph. The target calculation sequence is the calculation sequence corresponding to the calculation graph, which occupies the least memory of the second processor when executed on the second processor. The target calculation sequence is used to be executed on the second processor to realize reasoning based on the target model.
[0154] In one possible implementation, the computation graph is a directed computation graph, and the computation subgraphs include a first computation subgraph, each node in the first computation subgraph has only one parent node and one child node, and the memory size occupied by the nodes in the first computation subgraph is monotonically increasing or monotonically decreasing according to the connection order.
[0155] In a possible implementation, the computation subgraph includes a second computation subgraph, and a determination module 802 is configured to:
[0156] The dominator tree of the computation graph is constructed starting from the input node of the computation graph, and the post-dominator tree of the computation graph is constructed starting from the output node of the computation graph;
[0157] The preset nodes in the computation graph are used as initial nodes in the second computation subgraph, and the second computation subgraph is determined based on the dominator tree and the post-dominator tree.
[0158] In a possible implementation, determining the second computation subgraph based on the dominator tree and the post-dominator tree includes:
[0159] Determine, based on the dominator tree, other second computational subgraphs to which the initial node in the second computational subgraph can be output according to the connection order;
[0160] In response to satisfying a first preset condition, nodes in other second computing subgraphs and nodes on a connection path from the initial node in the dominator tree to nodes in the other second computing subgraphs are added to the second computing subgraph, and deleted from the other second computing subgraphs to determine the second computing subgraph, wherein the first preset condition is that the memory occupied by the nodes in the other second computing subgraphs is less than the memory occupied by the initial nodes of the second computing subgraph.
[0161] In a possible implementation, determining the second computation subgraph based on the dominator tree and the post-dominator tree includes:
[0162] Based on the dominator tree, determine the output node and the output boundary node in the second computation subgraph, all the child nodes of the output node in the dominator tree are outside the second computation subgraph, and at least one child node of the output boundary node in the dominator tree is outside the second computation subgraph;
[0163] Based on the post-dominator tree, input nodes and input boundary nodes in the second computational subgraph are determined, all parent nodes of the input nodes in the post-dominator tree are outside the second computational subgraph, and at least one parent node of the input boundary node in the post-dominator tree is outside the second computational subgraph.
[0164] In a possible implementation, the preset node has multiple parent nodes and only one child node.
[0165] In a possible implementation, the generating module 803 is configured to:
[0166] Generate an initial computation sequence based on the computation subgraph;
[0167] In response to the initial calculation sequence satisfying the second preset condition, the initial calculation sequence is used as the target calculation sequence.
[0168] In a possible implementation, generating a target computing sequence based on a computing subgraph further includes:
[0169] In response to the initial calculation sequence not satisfying the second preset condition, determining at least one node in the initial calculation sequence, the at least one node comprising a node with a peak memory usage and its associated nodes;
[0170] For each node associated with the node with the peak memory usage, if the child node of the node is a node in the second computational subgraph, the second computational subgraph is removed, and the input nodes, input boundary nodes, output nodes, and objects pointed to by output boundary nodes in the second computational subgraph are re-determined to update the computational subgraph;
[0171] Based on the updated computation subgraph, an initial computation sequence is generated, and steps after the initial computation sequence is generated are executed.
[0172] In a possible implementation, the second preset condition includes that the peak memory of the initial calculation sequence converges to a preset value, and / or the number of times the initial calculation sequence is generated reaches a preset threshold.
[0173] In a possible implementation, an initial computation sequence is generated based on the computation subgraph, including:
[0174] Determine the nodes with in-degree 0 in the computation graph, and for each node with in-degree 0, traverse each node in the computation graph based on the node with in-degree 0;
[0175] When the node is a node in the first computation subgraph, adding the node in the first computation subgraph corresponding to the node to the initial computation sequence;
[0176] When the node is a node in a second computation subgraph, in the second computation subgraph corresponding to the node, based on input nodes, input boundary nodes, output nodes, and output boundary nodes in the second computation subgraph, a computation sequence of the second computation subgraph is determined by a preset method and added to the initial computation sequence;
[0177] When the node is neither a node in the first computation subgraph nor a node in the second computation subgraph, adding the node to the initial computation sequence based on a preset method;
[0178] The nodes that have been added to the initial calculation sequence are removed from the calculation graph. If the number of nodes with in-degree 0 in the calculation graph is not 0, the nodes with in-degree 0 in the calculation graph are determined again. For each node with in-degree 0, the nodes in the calculation graph and the subsequent steps are traversed based on the node with in-degree 0 to generate the initial calculation sequence.
[0179] According to an embodiment of the present application, by enabling the first processor to obtain a computational graph of the target model, a target computational sequence is generated based on a computational subgraph in the computational graph, and the target computational sequence is made to be a computational sequence corresponding to the computational graph that occupies the smallest memory of the second processor when executed on the second processor. By serializing the computational graph in collaboration with software and hardware and searching for the computational sequence with the smallest memory, hardware resources can be saved and the inference speed of the target model on the second processor can be improved.
[0180] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0181] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.
[0182] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.
[0183] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.
[0184] Fig. 91 is a block diagram of a device 1900 for performing computation graph serialization according to an exemplary embodiment. For example, the device 1900 may be provided as a server. Fig. 9 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.
[0185] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.
[0186] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.
[0187] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0188] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0189] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0190] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0191] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0192] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0193] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0194] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0195] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for serializing a computational graph, characterized in that: The method comprises: The first processor obtains a computation graph of the target model; The first processor determines a computation subgraph in the computation graph, the computation subgraph comprising a second computation subgraph, the second computation subgraph being determined based on an initial node, a dominating tree of the computation graph, and a post-dominator tree of the computation graph; The first processor generates a target calculation sequence based on the calculation subgraph, and the target calculation sequence is a calculation sequence corresponding to the calculation graph, which occupies the smallest memory of the second processor when executed on the second processor. The target calculation sequence is used to be stored by the first processor and sent to the second processor for execution on the second processor to achieve reasoning based on the target model.
2. The method according to claim 1, characterized in that: The computation graph is a directed computation graph, the computation subgraph includes a first computation subgraph, each node in the first computation subgraph has only one parent node and one child node, and the memory size occupied by the nodes in the first computation subgraph increases monotonically or decreases monotonically according to the connection order.
3. The method according to claim 1, characterized in that: The determining of the computation subgraph in the computation graph includes: Starting from the input nodes of the computation graph, a dominator tree of the computation graph is constructed, and starting from the output nodes of the computation graph, a post-dominator tree of the computation graph is constructed; The preset node in the computation graph is used as the initial node in the second computation subgraph, and the second computation subgraph is determined based on the dominator tree and the post-dominator tree.
4. The method according to claim 3, characterized in that The determining of a second computation subgraph based on the dominator tree and the post-dominator tree comprises: Determine, based on the dominator tree, other second computational subgraphs to which the initial node in the second computational subgraph can be output according to a connection order; In response to satisfying a first preset condition, nodes in other second computing subgraphs and nodes on a connection path from the initial node in the dominator tree to nodes in the other second computing subgraphs are added to the second computing subgraph, and deleted from the other second computing subgraphs to determine the second computing subgraph, wherein the first preset condition is that the memory occupied by the nodes in the other second computing subgraphs is less than the memory occupied by the initial nodes of the second computing subgraph.
5. The method according to claim 3, characterized in that: The determining of a second computation subgraph based on the dominator tree and the post-dominator tree comprises: Based on the dominator tree, determine an output node and an output boundary node in a second computation subgraph, wherein all child nodes of the output node in the dominator tree are outside the second computation subgraph, and at least one child node of the output boundary node in the dominator tree is outside the second computation subgraph; Based on the post-dominator tree, input nodes and input boundary nodes in a second computational subgraph are determined, wherein all parent nodes of the input nodes in the post-dominator tree are outside the second computational subgraph, and at least one parent node of the input boundary nodes in the post-dominator tree is outside the second computational subgraph.
6. The method according to claim 3, characterized in that The preset node has multiple parent nodes and only one child node.
7. The method according to claim 1, characterized in that The generating a target computing sequence based on the computing subgraph includes: Generate an initial computation sequence based on the computation subgraph; In response to the initial calculation sequence satisfying the second preset condition, the initial calculation sequence is used as the target calculation sequence.
8. The method according to claim 7, characterized in that The generating a target computing sequence based on the computing subgraph further includes: In response to the initial calculation sequence not satisfying the second preset condition, determining at least one node in the initial calculation sequence, the at least one node comprising a node with a peak memory usage and its associated nodes; For each node associated with the node with the peak memory usage, if the child node of the node is a node in the second computational subgraph, the second computational subgraph is removed, and the input nodes, input boundary nodes, output nodes, and objects pointed to by output boundary nodes in the second computational subgraph are re-determined to update the computational subgraph; Based on the updated computation subgraph, an initial computation sequence is generated, and steps after the initial computation sequence is generated are executed.
9. The method according to claim 7 or 8, characterized in that: The second preset condition includes that the peak memory of the initial calculation sequence converges to a preset value, and / or the number of times the initial calculation sequence is generated reaches a preset threshold.
10. The method according to claim 7, characterized in that The generating of the initial calculation sequence based on the calculation subgraph includes: Determine the nodes with in-degree 0 in the computation graph, and for each node with in-degree 0, traverse each node in the computation graph based on the node with in-degree 0; When the node is a node in the first computation subgraph, adding the node in the first computation subgraph corresponding to the node to the initial computation sequence; When the node is a node in a second computation subgraph, in the second computation subgraph corresponding to the node, based on input nodes, input boundary nodes, output nodes, and output boundary nodes in the second computation subgraph, a computation sequence of the second computation subgraph is determined by a preset method and added to the initial computation sequence; When the node is neither a node in the first computing subgraph nor a node in the second computing subgraph, adding the node to the initial computing sequence based on the preset method; The nodes that have been added to the initial calculation sequence are removed from the calculation graph. If the number of nodes with in-degree 0 in the calculation graph is not 0, the nodes with in-degree 0 in the calculation graph are determined again. For each node with in-degree 0, the nodes in the calculation graph and the subsequent steps are traversed based on the node with in-degree 0 to generate the initial calculation sequence.
11. A computational graph serialization device, characterized in that: The device comprises: An acquisition module, used for the first processor to acquire a calculation graph of the target model; A determination module, configured for the first processor to determine a computation subgraph in the computation graph, the computation subgraph comprising a second computation subgraph, the second computation subgraph being determined based on an initial node, a dominating tree of the computation graph, and a post-dominator tree of the computation graph; A generation module is used for the first processor to generate a target calculation sequence based on the calculation subgraph, wherein the target calculation sequence is a calculation sequence corresponding to the calculation graph, which occupies the smallest memory of the second processor when executed on the second processor. The target calculation sequence is used to be stored by the first processor and sent to the second processor for execution on the second processor to achieve reasoning based on the target model.
12. A computational graph serialization device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 10 when executing the instructions stored in the memory.
13. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.
Citation Information
Patent Citations
Model deployment method and device, model operation method and device, offline analysis tool and electronic equipment
CN115525436A
