Data processing method and apparatus, electronic device, and storage medium

Abstract

A data processing method, data processing apparatus, electronic device, and storage medium are disclosed. The data processing method includes: acquiring a computation graph; traversing the computation graph sequentially according to data flow direction, branch by branch, to generate an execution sequence that satisfies data dependencies and has a defined temporal order on the computation graph. The traversal process includes at least one branch search operation, each branch search operation including: acquiring at least one branch corresponding to the branch search operation; adding at least one branch to a branch container; sequentially acquiring branches that meet processing conditions from the branch container and performing operator fusion and other processing, then generating the execution sequence portion corresponding to that branch on the computation graph. This computation graph processing method integrates the traversal of a directed acyclic graph, operator fusion, and execution sequence generation into one process, requiring only one graph traversal to complete the processing from the computation graph to the execution sequence, thus improving the processing efficiency of the computation graph.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data processing method, a data processing apparatus, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] A graph is a mathematical structure used to model pairwise relationships between objects. It consists of "nodes" (also called "vertices") and "edges" connecting these nodes. In graph theory, if a directed graph cannot be returned to a vertex by following several edges, it is called a directed acyclic graph (DAG). Summary of the Invention

[0003] At least one embodiment of this disclosure provides a data processing method, comprising: a data processing method including: acquiring a computation graph, wherein the computation graph includes multiple nodes; traversing the computation graph sequentially according to data flow direction, using branches as units, to generate an execution sequence corresponding to the computation graph that is executed in chronological order and has data dependencies, wherein the traversal process includes at least one branch search processing operation, each branch search processing operation including: acquiring at least one branch corresponding to the branch search processing operation, wherein each branch includes at least one node; adding the at least one branch to a branch container; acquiring M branches from the branch container as M branches to be processed, wherein when acquiring the M branches to be processed, branches in the branch container that do not include a convergence node are preferentially selected as branches to be processed, the convergence node is a node with multiple inputs, and M is a positive integer; processing the M branches to be processed to generate an execution sequence portion in the execution sequence corresponding to the computation graph that corresponds to the M branches to be processed respectively.

[0004] For example, in the data processing method provided in at least one embodiment of this disclosure, obtaining at least one branch corresponding to the branch search processing operation includes: determining N current nodes, wherein the N current nodes are the start node of the computation graph or one or more adjacent nodes in the output direction of the end node of the branch to be processed in the previous branch search processing operation, and N is a positive integer; for each current node, traversing the computation graph according to the data flow direction with the current node as the starting node to obtain the branch corresponding to the current node; and taking the N branches corresponding to the N current nodes as at least one branch corresponding to the branch search processing operation.

[0005] For example, in a data processing method provided in at least one embodiment of this disclosure, for each current node, the computation graph is traversed along the data flow direction, starting from the current node, to obtain the branch corresponding to the current node. This includes: in response to the current node satisfying the endpoint node condition, determining that the branch corresponding to the current node includes the current node; in response to the current node not satisfying the endpoint node condition, continuing to search along the data flow direction for nodes among one or more nodes that have a direct or indirect adjacency relationship with the current node that satisfy the endpoint node condition, and determining that the branch corresponding to the current node includes nodes between the current node and the nodes that satisfy the endpoint node condition, the current node, and the nodes that satisfy the endpoint node condition; wherein, the nodes that satisfy the endpoint node condition include the end node of the computation graph, a branch node, or an adjacent node in the input direction of a traversed node, and the branch node is a node with multiple adjacent nodes in the output direction.

[0006] For example, in a data processing method provided in at least one embodiment of this disclosure, the branch container includes a first container and a second container. Adding the at least one branch to the branch container includes: for each branch: determining that the branch does not include a convergence node, and placing the branch into the first container; determining that the branch includes a convergence node, and placing the branch into the second container.

[0007] For example, in a data processing method provided in at least one embodiment of this disclosure, obtaining M branches from the branch container as M branches to be processed includes: in response to the existence of a branch in the first container, obtaining M branches from the first container as the M branches to be processed; and in response to the absence of a branch in the first container, obtaining M branches from the second container as the M branches to be processed.

[0008] For example, in a data processing method provided in at least one embodiment of this disclosure, in response to the absence of branches in the first container, M branches are obtained from the second container as the M branches to be processed, including: selecting from all branches stored in the second container one of the branches that satisfies the processing conditions of the convergence node as the branch to be processed; wherein, for any branch among all branches, in response to the multiple adjacent nodes in the input direction of the convergence node in any branch, only one adjacent node has been traversed but not processed, and all other adjacent nodes except the one adjacent node have been traversed and processed, it is determined that any branch satisfies the processing conditions of the convergence node.

[0009] For example, in the data processing method provided in at least one embodiment of this disclosure, each node has a first marker, which is used to indicate whether the node has been traversed. During the traversal, in response to the node being traversed and belonging to any branch, the first marker of the node is modified from never traversed to having been traversed.

[0010] For example, in a data processing method provided in at least one embodiment of this disclosure, each node has a second marker, which is used to indicate whether the node has been processed; processing the M branches to be processed to generate an execution sequence portion in the execution sequence corresponding to the M branches to be processed, including: for each branch to be processed: removing the branch to be processed from the branch container; establishing the input-output relationship of the branch to be processed, generating an execution sequence portion in the execution sequence corresponding to the branch to be processed; marking the second marker of one or more nodes included in the branch to be processed as processed.

[0011] For example, in a data processing method provided in at least one embodiment of this disclosure, processing the M branches to be processed to generate execution sequence portions in the execution sequence corresponding to the M branches to be processed in the computation graph includes: for each branch to be processed: removing the branch to be processed from the branch container; performing operator fusion processing on the branch to be processed to obtain at least one fused execution unit; reconstructing the input-output relationship of the at least one fused execution unit to generate the execution sequence portion corresponding to the branch to be processed in the execution sequence, wherein the number of execution sequence portions corresponding to the branch to be processed is at least one, and the at least one execution sequence portion corresponds one-to-one with the at least one fused execution unit; marking a second flag indicating that one or more nodes included in the branch to be processed have been processed.

[0012] For example, in the data processing method provided in at least one embodiment of this disclosure, the computation graph is traversed sequentially according to the data flow direction, with branches as units, to generate an execution sequence corresponding to the computation graph that is executed in chronological order and has data dependencies. This includes: obtaining the execution sequence parts corresponding to all branches to be processed in each branch search processing operation; and combining all execution sequence parts obtained by the at least one branch search processing operation according to the generation order and input-output relationship to obtain the execution sequence corresponding to the computation graph.

[0013] For example, in the data processing method provided in at least one embodiment of this disclosure, the first container is a linear storage structure that follows the first-in-first-out principle.

[0014] For example, in the data processing method provided in at least one embodiment of this disclosure, the computation graph is a directed acyclic graph, which represents the data relationship of the neural network; the execution sequence portion corresponding to each branch to be processed constitutes a fusion layer, and the neural network performs the computation based on the execution sequence layer by layer in units of the fusion layer.

[0015] At least one embodiment of this disclosure provides a data processing apparatus, comprising: an acquisition unit configured to acquire a computation graph, wherein the computation graph includes multiple nodes; and a processing unit configured to traverse the computation graph sequentially according to data flow direction, branch by branch, to generate an execution sequence corresponding to the computation graph that is executed in chronological order and has data dependencies, wherein the traversal process includes at least one branch search processing operation, and the processing unit performs the following operations when executing each branch search processing operation: acquiring at least one branch corresponding to the branch search processing operation, wherein each branch includes at least one node; adding the at least one branch to a branch container; acquiring M branches from the branch container as M branches to be processed, wherein when acquiring the M branches to be processed, branches in the branch container that do not include a convergence node are preferentially selected as branches to be processed, the convergence node being a node with multiple inputs, and M being a positive integer; and processing the M branches to be processed to generate portions of the execution sequence corresponding to the computation graph that correspond to the M branches to be processed.

[0016] At least one embodiment of this disclosure provides an electronic device, including: a memory that non-transitoryly stores computer-executable instructions; and a processor configured to execute the computer-executable instructions, wherein the computer-executable instructions are executed by the processor to implement a data processing method according to any embodiment of this disclosure.

[0017] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the data processing method according to any embodiment of this disclosure. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0019] Figure 1 A schematic flowchart illustrating a data processing method provided for at least one embodiment of this disclosure;

[0020] Figure 2AA schematic diagram of a computational graph provided in at least one embodiment of the present disclosure is shown;

[0021] Figure 2B It shows Figure 2A A schematic diagram of the execution sequence corresponding to the computation graph shown;

[0022] Figure 3 A schematic flowchart illustrating a branch search processing operation provided for at least one embodiment of this disclosure;

[0023] Figures 4A to 4I A schematic diagram illustrating the execution process of a data processing method provided in at least one embodiment of this disclosure;

[0024] Figure 5 A schematic block diagram of a data processing apparatus provided in at least one embodiment of this disclosure;

[0025] Figure 6 A schematic diagram of an electronic device provided for at least one embodiment of this disclosure;

[0026] Figure 7 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0028] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0029] Currently, with the development of artificial intelligence technology, more and more application scenarios not only require software-implemented artificial intelligence algorithm models, but also require hardware products with higher processing efficiency and stronger specialization to handle large amounts of tasks and data.

[0030] Artificial Neural Networks (ANNs), also simply called neural networks, are mathematical models that mimic the behavioral characteristics of animal neural networks to perform distributed parallel information processing. These networks rely on the complexity of the system, adjusting the connections between a large number of internal nodes to achieve information processing. Regardless of the type, artificial neural networks share common characteristics such as massively parallel processing, distributed storage, flexible topology, high redundancy, and nonlinear operations. They possess capabilities in areas such as processing speed, associative ability, adaptability, fault tolerance, and self-organization. These characteristics and capabilities form the technological foundation for artificial neural networks to simulate intelligent activities and have found important applications in various technological fields. For example, artificial neural networks can be used in data compression, image processing, video coding, and signal processing.

[0031] Due to hardware resource limitations of computing platforms and the demands of high-performance computing, multiple operators in a neural network that meet certain conditions or rules are typically fused together to form a fused operator in order to improve the computational efficiency of the neural network. A fused operator, or a single operator that cannot be fused, can be represented as a fusion layer, or simply a layer. The computation process of a neural network is performed layer-by-layer, using fused layers as units. Typically, the output of the previous layer (or the first few layers) serves as the input of the next layer (or the next few layers), thus creating data dependencies between fused layers.

[0032] Neural networks can typically be represented or transformed into a directed acyclic graph (DAG), and inference or training is based on the DAG. For example, during inference or training, the DAG needs to be transformed into an execution sequence in units of fusion layers, according to the data dependencies between nodes on the DAG.

[0033] Both operator fusion and execution sequence generation are based on the traversal of directed acyclic graphs (DAGs). The traversal of DAGs must take into account the data dependencies between nodes in the graph, thereby determining the traversal order of nodes in the DAG.

[0034] Traditional graph traversal methods are mainly divided into two types: Depth-First Search (DFS) and Breadth-First Search (BFS). DFS uses a recursive approach, starting from the beginning node of the graph (e.g., the node that receives external input) and traversing to the end of the graph before returning to traverse other branches. BFS uses a queue data structure to traverse all branches in parallel until the end of the graph.

[0035] Neither depth-first search nor breadth-first search can guarantee the correctness of data dependencies during the traversal process. To maintain the correctness of data dependencies, topological sorting is typically used. Topological sorting yields a linear sequence of all vertices of a directed acyclic graph, satisfying the following two conditions:

[0036] 1. Each node appears exactly once.

[0037] 2. If there exists a path from node A to node B, then node A appears before node B in the sequence.

[0038] For example, one topological sorting method is to first traverse the entire directed acyclic graph to determine the in-degree of each node; then start the calculation from the nodes with an in-degree of 0. After calculating the nodes with an in-degree of 0, decrement the in-degree of all adjacent nodes in the output direction of that node by 1, and then calculate all nodes with an in-degree of 0 again; this process is repeated iteratively until all nodes have been calculated.

[0039] While topological sorting resolves data dependencies, it is not well-integrated with operator fusion. As mentioned earlier, to improve the computational efficiency of neural networks, operators in these networks typically need to be fused according to certain fusion strategies. Using topological sorting requires three traversals of the graph to obtain the final linear execution sequence: the first traversal identifies multiple operators in the network that meet certain conditions or rules as target operators for fusion and fuses them; the second traversal reconstructs the input-output relationships of these fused operators; and the third traversal uses topological sorting to transform the reconstructed computation graph into a linear execution sequence. Therefore, this approach has high time complexity and low execution efficiency.

[0040] At least one embodiment of this disclosure provides a data processing method, a data processing apparatus, an electronic device, and a non-transitory computer-readable storage medium. The data processing method includes: acquiring a computation graph, wherein the computation graph includes multiple nodes; traversing the computation graph sequentially according to data flow direction, branch by branch, to generate an execution sequence corresponding to the computation graph that is executed in time order and has data dependencies, wherein the traversal process includes at least one branch search processing operation, each branch search processing operation including: acquiring at least one branch corresponding to the branch search processing operation, wherein each branch includes at least one node; adding at least one branch to a branch container; acquiring M branches from the branch container as M branches to be processed, wherein when acquiring the M branches to be processed, branches in the branch container that do not include a convergence node are preferentially selected as branches to be processed, the convergence node is a node with multiple inputs, and M is a positive integer; processing the M branches to be processed to generate an execution sequence portion in the execution sequence corresponding to the computation graph that corresponds to the M branches to be processed respectively.

[0041] This data processing method prioritizes branches without a convergence node to generate their corresponding execution sequence parts, followed by branches with a convergence node. This ensures that the computation of the convergence node occurs after its input node, reducing the waiting time for processing different branches. Simultaneously, the computation graph is traversed branch by branch. During this traversal, branch search operations enable input-output relationship reconstruction and operator fusion. Thus, an execution sequence of the computation graph can be generated with a single traversal. This execution sequence satisfies data dependencies and can also perform operator fusion when necessary, resulting in more efficient traversal and execution sequence generation of directed acyclic graphs, thus improving execution efficiency.

[0042] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0043] Figure 1 This is a schematic flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0044] like Figure 1 As shown, at least one embodiment of the present disclosure provides a data processing method including steps S10 to S20.

[0045] Step S10: Obtain the computation graph.

[0046] For example, a computation graph consists of multiple nodes. For instance, a neural network can be viewed as a directed acyclic graph (DAG) composed of many nodes, each node corresponding to an operator. For example, a computation graph can be a directed acyclic graph, which represents the data relationships within a neural network.

[0047] For example, the implementation of an artificial intelligence algorithm model on a hardware platform can be achieved by breaking down the algorithm model into many processing steps, each corresponding to an operator. An operator is a mathematical operation used to implement a certain computational function, such as convolution, batch normalization (BN), pooling, and so on.

[0048] For example, step S10 may include: generating a directed acyclic graph based on the network structure of a neural network, wherein the directed acyclic graph includes nodes and connections between nodes, and the connections represent the data dependencies and data flow directions between nodes; and using the directed acyclic graph as a computational graph.

[0049] Step S20: Traverse the computation graph sequentially according to the data flow, branch by branch, to generate an execution sequence that is executed in chronological order and has data dependencies.

[0050] For example, the execution sequence reflects the data dependencies and execution order between fusion operators or fusion layers formed by individual operators.

[0051] Figure 2A A schematic diagram of a computational graph provided in at least one embodiment of the present disclosure is shown. Figure 2B A schematic diagram of the execution sequence corresponding to this computation graph is shown.

[0052] like Figure 2A As shown, the computation graph includes 12 nodes, 1 to 12. The arrowed lines connecting the nodes represent data dependencies and data flow directions, with the arrows pointing in the direction of the data flow. For example, this computation graph can illustrate the network structure of a neural network.

[0053] Each node can represent a computational operation of an operator, such as convolution or batch normalization. For example, ... Figure 2A As shown, the output data of node 1 flows to nodes 2 and 7, the output data of node 2 flows to nodes 3 and 5, and so on. Therefore, it can be determined that node 1 has a data dependency relationship with nodes 2 and 7, and node 2 has a data dependency relationship with nodes 1, 3, and 5. This is understandable. Figure 2A The computational graph shown is merely illustrative. The method according to the embodiments of this disclosure can be applied to various types of neural network structures. The number of nodes in the computational graph and the connection relationships between nodes are determined according to the specific neural network structure.

[0054] right Figure 2A The computation graph shown applies an execution sequence obtained by applying at least one embodiment of this disclosure, as follows: Figure 2B As shown.

[0055] According to embodiments of this disclosure, a nonlinear graph is transformed into a linear execution sequence with a time order based on data dependencies on the computation graph and graph traversal rules. The execution sequence preserves the data dependencies on the graph while ensuring that all data dependencies are unidirectional along the time axis; that is, the execution sequence has the following characteristics: Figure 2B The structure shown.

[0056] For example, an execution sequence can consist of multiple execution sequence parts. Figure 2B As shown, the execution sequence consists of 8 execution sequence parts, that is... Figure 2B The eight ellipses labeled AH represent the eight nodes in the sequence, with each ellipse containing a number indicating which nodes' execution sequence segments it corresponds to. For example... Figure 2B As shown, each execution sequence part may include one node in the computation graph or multiple nodes. The number of nodes in each execution sequence part is related to whether the nodes corresponding to that execution sequence part include a convergence node, and is also related to the fusion strategy when operator fusion is required.

[0057] For example, when representing a neural network using a computational graph, each execution sequence part constitutes a fusion layer. The execution sequence parts in the execution sequence are started sequentially and timely by the fusion layer, thereby completing the inference or training of the neural network.

[0058] In the data processing method provided in at least one embodiment of this disclosure, the data can be processed by a single traversal. Figure 2A The computational graph shown is converted to Figure 2B The execution sequence shown demonstrates a more efficient implementation of traversal and execution sequence generation for directed acyclic graphs, reducing the time required to generate execution sequences and improving execution efficiency.

[0059] The following section, with reference to the accompanying drawings, details the specific process of generating the execution sequence from the computation graph.

[0060] For example, the traversal process includes at least one branch search operation. Each branch search operation can search for at least one branch and select some or all of the branches found in the current or previous searches as branches to be processed to obtain the execution sequence portion corresponding to the branch to be processed. For example, in the process of traversing the computation graph branch by branch, two types of branches will be encountered: branches without a convergence node and branches with a convergence node. The key to resolving data dependencies is to process the branches without a convergence node first to generate the corresponding execution sequence portion, and then process the branches with a convergence node to generate the corresponding execution sequence portion, so as to ensure that the computation of the convergence node occurs after the input node of the convergence node. For example, the input node of the convergence node is the adjacent node in the input direction of the convergence node.

[0061] For example, after a branch found by the branch search operation is processed, such as after the execution sequence corresponding to the found branch is generated, other branches are traversed from the end node of the branch, and the next branch search operation is executed. This process is repeated to complete the traversal of the entire computation graph. An execution sequence can be generated in one traversal. Furthermore, when operator fusion is required, operator fusion of some or all nodes within a branch can be completed according to the operator fusion strategy.

[0062] Figure 3 This is a schematic flowchart illustrating a branch search processing operation provided for at least one embodiment of the present disclosure.

[0063] like Figure 3 As shown, the branch search processing operation may include steps S201-S204.

[0064] Step S201: Obtain at least one branch corresponding to the branch search processing operation.

[0065] For example, each branch includes at least one node.

[0066] For example, step S201 may include: determining N current nodes, wherein the N current nodes are the start node of the computation graph or one or more adjacent nodes in the output direction of the end node of the branch to be processed in the previous branch search processing operation, and N is a positive integer; for each current node, traversing the computation graph according to the data flow direction with the current node as the starting node to obtain the branch corresponding to the current node; and taking the N branches corresponding to the N current nodes as at least one branch corresponding to the branch search processing operation.

[0067] For example, in this disclosure, a branch can be understood as a path in the computation graph along the data flow direction, consisting of one or more nodes. A branch starts from the current node and traverses along the data flow direction until it ends at a node that satisfies the endpoint condition. A branch can be a path along the data flow direction from the current node to a node that satisfies the endpoint condition, and all nodes on this path (excluding previously traversed nodes) constitute a branch. For example, a node that satisfies the endpoint condition can be an end node, a fork node, or a previously traversed node. Here, an end node is the node in the computation graph that outputs the final result externally, and a fork node is a node with multiple adjacent nodes in the output direction.

[0068] For example, for each current node, starting from the current node, the computation graph is traversed along the data flow direction to obtain the branch corresponding to the current node. This can include: in response to the current node satisfying the endpoint node condition, determining that the branch corresponding to the current node includes the current node; in response to the current node not satisfying the endpoint node condition, continuing to search along the data flow direction for one or more nodes that have a direct or indirect adjacency relationship with the current node and satisfying the endpoint node condition, determining that the branch corresponding to the current node includes nodes between the current node and the node satisfying the endpoint node condition, the current node, and nodes satisfying the endpoint node condition (excluding previously traversed nodes); wherein, nodes satisfying the endpoint node condition include the end node of the computation graph, a branch node, or a previously traversed node, and a branch node is a node with multiple adjacent nodes in the output direction.

[0069] For example, when performing the first branch search operation, the current node can be the starting node of the computation graph, such as the node that receives external input. Figure 2A For example, the starting node can be node 1. Of course, depending on the different network structures, the computation graph can also have multiple starting nodes, and these starting nodes can all serve as the current node when executing the first branch search operation.

[0070] For example, when executing the second and subsequent branch search processing operations, the current node can be the adjacent node in the output direction of the endpoint node of the branch to be processed in the previous branch search processing operation. For example, at least one branch search processing operation is executed sequentially. The branch to be processed in the current branch search processing operation may be a branch found in the previous branch search processing operation or a branch found in the current branch search processing operation. For example, after completing the processing of the branches to be processed in the current branch search processing operation, the adjacent node in the output direction of the endpoint node of these processed branches is taken as the current node, and the next branch search processing operation is executed.

[0071] For example, with Figure 2A For example, when performing the first branch search operation, the current node includes node 1. Node 1 is a branch node, meaning its output direction has two adjacent nodes, namely node 2 and node 7. Node 1 satisfies the endpoint node condition, therefore, the branch corresponding to node 1 includes node 1 itself, and node 1 is also the endpoint node of this branch. Furthermore, for the first branch search operation, since it only has one current node, this branch search operation corresponds to a branch that includes node 1.

[0072] For example, after processing the branch including node 1, the adjacent nodes in the output direction of node 1, namely node 2 and node 7, are used as the current nodes for the next branch search operation (e.g., the second branch search operation).

[0073] For example, when performing the second branch search operation, the current nodes include node 2 and node 7. Both node 2 and node 7 are branch nodes and satisfy the endpoint node condition. Therefore, for the second branch search operation, this branch search operation corresponds to a branch including node 2 and a branch including node 7.

[0074] For example, taking node 5 as the current node, node 5 does not meet the end node condition. We traverse the adjacent nodes in the output direction of node 5 along the data flow, which is node 6. Since node 6 also does not meet the end node condition, we continue to traverse the adjacent nodes in the output direction of node 6 along the data flow, which is node 12. Since node 12 is the end node, that is, the node that outputs the final result to the outside, it meets the end node condition. Therefore, we can get a branch including node 5, node 6 and node 12.

[0075] For example, after processing the branch including node 7, we determine the branch with nodes 8 and 10 as the current nodes. Node 9 is a branch node and satisfies the endpoint node condition, so we can determine that nodes 10 and 9 form a branch. Since node 9 has already been traversed, and node 8 is the adjacent node in the input direction of node 9, node 8 satisfies the endpoint node condition, so we determine a branch including node 8. Of course, we can also treat nodes 8 and 9 as one branch and node 10 as another branch, with the same logic as described above, which will not be repeated here.

[0076] For example, after obtaining at least one branch corresponding to the current branch search processing operation, these branches are added to the branch container.

[0077] In step S202, at least one branch is added to the branch container.

[0078] For example, a branch container may include a first container and a second container, which may be linear storage structures. For instance, the first container stores branches that do not include the sink node, and the second container stores branches that do include the sink node.

[0079] For example, step S202 may include: for each branch: determining that the branch does not include a sink node, and placing the branch into a first container; determining that the branch includes a sink node, and placing the branch into a second container.

[0080] For example, the first container can be a linear storage structure that follows the first-in-first-out principle, such as a queue; the second container can be a linear storage structure that follows the last-in-first-out principle, such as a stack. Of course, this disclosure is not limited to this, and it can also be a container with other structures.

[0081] For example, a branch can only be searched for in the stack after all branches in the queue have been processed and their corresponding execution sequences have been generated. For instance, branches in the queue can be processed sequentially in the order they were added; if the queue is empty, a branch is searched for in the stack.

[0082] For example, when the first container is a queue and the second container is a stack, branches that enter the first container first are processed first, so branches closer to the start node are processed earlier. This reduces the probability of a branch being unprocessable due to unprocessed input at the sink node when processing branches including the sink node. Meanwhile, branches that enter the second container first are processed later, so the input of branches that enter the second container first is more likely to have been processed. The space of the first and second containers will not grow too much. This configuration is the most efficient and has the lowest time complexity.

[0083] In step S203, M branches are obtained from the branch container as M branches to be processed.

[0084] For example, when obtaining M branches to be processed, the branch excluding the sink node in the branch container is selected as the branch to be processed first. The sink node is a node with multiple inputs, and M is a positive integer.

[0085] For example, step S203 may include: in response to the existence of a branch in the first container, obtaining M branches from the first container as M branches to be processed; in response to the absence of a branch in the first container, obtaining M branches from the second container as M branches to be processed.

[0086] For example, in response to the absence of a branch in the first container, obtaining M branches from the second container as M branches to be processed can include: selecting the last branch placed into the second container from all branches stored in the second container that satisfies the processing conditions of the sink node as the branch to be processed. For any branch among all branches, in response to the situation where, among the multiple adjacent nodes in the input direction of the sink node in any branch, only one adjacent node has been traversed but not processed, and all other adjacent nodes have been traversed and processed, it is determined that any branch satisfies the processing conditions of the sink node.

[0087] For example, if there is no branch in the first container, and none of the branches stored in the second container meet the processing conditions of the aggregation node, then the next branch search processing operation can be executed directly.

[0088] For example, M can be 1, which means that one branch is selected as the branch to be processed each time. If there is a branch in the first container, then one branch is selected as the branch to be processed. For example, the branch that entered the first container earliest is selected as the branch to be processed. If the first container is empty, then one branch is selected as the branch to be processed from the second container. For example, the branch that entered the second container latest is selected as the branch to be processed.

[0089] For example, M can also be greater than 1, and the specific value of M can be set as needed. For example, a processing threshold can be preset, that is, the upper limit of the number of branches to be processed in each branch search processing operation, for example, 2. If there are multiple branches in the first container, two branches are selected as branches to be processed, and both branches are selected as branches to be processed. Then, the adjacent node in the output direction of the endpoint node of the two branches is taken as the current node, and the next round of branch search processing operation continues. If there is only one branch in the first container, that branch is selected as the branch to be processed. Alternatively, a branch can be selected from the first container and a branch can be selected from the second container, and both branches can be selected as branches to be processed. This disclosure does not impose specific limitations in this regard.

[0090] For example, taking a branch including nodes 5, 6, and 12 as an example, node 6 is the sink node, and its adjacent nodes in the input direction are nodes 5 and 4. Since nodes 5 and 6 belong to the same branch, they must have been traversed but not processed. Therefore, if node 4 has been traversed and processed before, then the branch including nodes 5, 6, and 12 satisfies the sink node processing condition; otherwise, the branch does not satisfy the sink node processing condition.

[0091] For example, a first and second marker can be added to all nodes in the computation graph. The first marker indicates whether the node has been traversed; traversed nodes belong to a branch, while untraversed nodes do not yet belong to any branch, thus preventing the same node from being traversed repeatedly, i.e., preventing the same node from being assigned to multiple branches. The second marker indicates whether the node has been processed; processed nodes generate a corresponding execution sequence, while unprocessed nodes do not, preventing errors in the computation graph traversal order. Thus, by adding two markers to record the state of each node during traversal (whether it has been traversed and processed), an execution sequence can be generated with a single traversal.

[0092] For example, during traversal, in response to a node being traversed and belonging to any branch, the first marker of that node is modified from "never traversed" to "has been traversed," indicating that the node has been traversed and belongs to a branch, and there is no need to traverse the node again.

[0093] In step S204, the M branches to be processed are processed to generate the execution sequence parts corresponding to the M branches to be processed in the execution sequence corresponding to the computation graph.

[0094] For example, in some embodiments, operator fusion is not required, and step S204 may include: for each branch to be processed: removing the branch to be processed from the branch container; establishing the input-output relationship of the branch to be processed, generating the execution sequence part corresponding to the branch to be processed in the execution sequence; marking a second label of one or more nodes included in the branch to be processed as processed.

[0095] For example, in some other embodiments, operator fusion needs to be performed, and step S204 may include: for each branch to be processed: removing the branch to be processed from the branch container; performing operator fusion processing on the branch to be processed to obtain at least one fused execution unit; reconstructing the input-output relationship of the at least one fused execution unit to generate an execution sequence part corresponding to the branch to be processed in the execution sequence, wherein the number of execution sequence parts corresponding to the branch to be processed is at least one, and at least one execution sequence part corresponds one-to-one with at least one fused execution unit; marking a second mark of one or more nodes included in the branch to be processed as processed.

[0096] Of course, it should be noted that some operators may undergo operator fusion processing while others may not. For example, branches that require operator fusion processing may first undergo operator fusion processing to obtain fused execution units, and then the input-output relationship of the fused execution units may be reconstructed; branches that do not undergo operator fusion processing may directly reconstruct their input-output relationship. This disclosure does not impose specific restrictions on this.

[0097] For example, if the branch to be processed undergoes operator fusion processing, the operator fusion strategy can determine whether some or all nodes in the branch to be processed are merged into one or more fusion execution units. For example, if the branch to be processed includes 3 nodes, according to the operator fusion strategy, the 3 nodes can be merged into one fusion execution unit. The execution sequence part corresponding to the branch to be processed is obtained after processing such as reconstructing the input-output relationship of the fusion execution unit. For example, according to the operator fusion strategy, 2 of the 3 nodes can be merged into one fusion execution unit, and the other node can be used as a fusion execution unit. The number of execution sequence parts corresponding to the branch to be processed is 2. One execution sequence part corresponds to the fusion execution unit obtained by merging 2 nodes, which is generated after processing such as reconstructing the input-output relationship of the fusion execution unit. The other execution sequence part corresponds to the fusion execution unit obtained by merging 1 node, which is also generated after processing such as reconstructing the input-output relationship of the fusion execution unit. In this case, multiple execution sequence parts are sequentially adjacent and connected to each other, and are connected to other execution sequence parts according to the data flow direction.

[0098] Whether a branch is subject to operator fusion depends on the operator fusion strategy (algorithm). It is understood that the embodiments of this disclosure do not restrict the specific rules for operator fusion, and conventional methods in related technologies can be used.

[0099] Furthermore, depending on actual needs, the processing of generating the execution sequence portion may include other processing, and this disclosure does not impose specific limitations on this.

[0100] For example, step S20 may include: obtaining the execution sequence parts corresponding to all branches to be processed in each branch search processing operation; combining all execution sequence parts obtained from at least one branch search processing operation according to the generation order and input-output relationship to obtain the execution sequence corresponding to the computation graph.

[0101] For example, after each branch search operation is completed, M execution sequence parts corresponding to the M branches to be processed in that branch search operation can be obtained. These M execution sequence parts are combined with the execution sequence parts generated by the previous branch search operation according to the generation order and input-output relationship. This combination includes connecting the execution sequence parts obtained by the current branch search operation with the previously generated execution sequence parts according to the data flow direction. For example, pointers can be used to connect them in the program. Thus, after all branch search operations are completed, the execution sequence corresponding to the computation graph can be obtained.

[0102] It should be noted that the traversal process provided in this embodiment is a process of traversing and processing simultaneously. That is, the branch search processing operations are executed sequentially. Each branch search processing operation first determines one or more searched branches based on the current node of the branch search processing operation and puts them into a branch container. Then, it selects the branch to be processed from the branch container, generates the execution sequence part corresponding to the branch to be processed, and then takes the adjacent node in the output direction of the endpoint node of the branch to be processed as the current node to start the next branch search processing operation. Due to the complexity of the neural network structure, this method does not need to put all branches into the container, avoiding excessive growth of the container space, which can increase processing efficiency and reduce time complexity.

[0103] To better illustrate the complete data processing method execution process, the following will use... Figure 2A Taking the calculation graph shown as an example, the data processing method provided in this disclosure embodiment is explained in detail to obtain... Figure 2B The execution process of the execution sequence shown.

[0104] Figures 4A to 4I This is a schematic diagram illustrating the execution process of a data processing method provided in at least one embodiment of the present disclosure.

[0105] It should be noted that, Figures 4A to 4IThe gray nodes represent nodes that have been processed. For ease of explanation, the first container is a queue, and the second container is a stack. Data in the first container enters the queue from the "tail" and exits from the "head". Data in the second container is pushed onto the stack from the "top" and also popped from the stack.

[0106] like Figure 4A As shown, in the first branch search processing operation, the current node is node 1. Referring to the relevant content of step S201, branch 1, which includes node 1, is obtained. Since branch 1 does not include the convergence node, branch 1 enters the first container; simultaneously, the first marker of node 1 is modified from untraversed to traversed. Branch 1 is selected as the branch to be processed, and branch 1 is processed according to the relevant content of step S204; after processing branch 1, the second marker of node 1 is modified from unprocessed to processed, and the execution sequence part A corresponding to branch 1 is generated. For example, the processing may include establishing the input-output relationship of branch 1, etc.

[0107] like Figure 4B As shown, node 1 has been processed, branch 1 has been removed from the first container, and the execution sequence part A corresponding to branch 1 has been generated. Next, in the second branch search processing operation, the current node is the two adjacent nodes in the output direction of the endpoint node of branch 1, namely node 2 and node 7. Referring to the relevant content of step S201, branch 2 including node 2 and branch 3 including node 7 are obtained. Since neither branch 2 nor branch 3 includes a convergence node, both branch 2 and branch 3 enter the first container; simultaneously, the first marker of nodes 2 and 7 is modified from untraversed to traversed. Branch 2, which is closer to the head of the queue, is selected as the branch to be processed. Referring to the relevant content of step S204, branch 2 is processed. After branch 2 is processed, the second marker of node 2 is modified from unprocessed to processed, and the execution sequence part B corresponding to branch 2 is generated.

[0108] like Figure 4CAs shown, node 2 has been processed, branch 2 has been removed from the first container, and a branch 2 execution sequence part B has been generated. Execution sequence part B is connected to execution sequence part A according to the data flow direction (or input-output relationship). Then, in the third branch search processing operation, the current node is the two adjacent nodes in the output direction of the endpoint node of branch 2, namely node 3 and node 5. Referring to the relevant content of step S201, branch 4 including node 3 and node 4 and branch 5 including node 5, node 6, and node 12 are obtained. Since branch 4 includes the convergence node 4 and branch 5 includes the convergence node 6, both branch 4 and branch 5 enter the second container; simultaneously, the first markers of nodes 3, 4, 5, 6, and 12 are modified from untraversed to traversed. Since there are still branches unprocessed in the first container at this time, branch 3 is selected as the branch to be processed. After branch 3 is processed, the second marker of node 7 is modified from unprocessed to processed, and the execution sequence part C corresponding to branch 3 is generated.

[0109] like Figure 4D As shown, node 7 has been processed, branch 3 has been removed from the first container, and an execution sequence part C corresponding to branch 3 has been generated. Execution sequence part C is connected to execution sequence part A according to the data flow direction (or input-output relationship). Then, in the fourth branch search processing operation, the current node is the two adjacent nodes in the output direction of the endpoint node of branch 3, namely node 8 and node 10. Referring to the relevant content of step S201, branch 6 including node 8 and branch 7 including nodes 10 and 9 are obtained. Since branch 6 does not include the convergence node, it enters the first container; branch 7 includes the convergence node 9 and enters the second container. At the same time, the first markers of nodes 8, 9, and 10 are modified from untraversed to traversed. Since there are still branches unprocessed in the first container at this time, branch 6 is selected as the branch to be processed. After branch 6 is processed, the second marker of node 8 is modified from unprocessed to processed, and an execution sequence part D corresponding to branch 6 is generated.

[0110] like Figure 4EAs shown, node 8 has been processed, branch 6 has been removed from the first container, and an execution sequence part D corresponding to branch 6 has been generated. Execution sequence part D is connected to execution sequence part C according to the data flow direction (or input-output relationship). Then, in the fifth branch search processing operation, the current node is the adjacent node in the output direction of the endpoint node of branch 6, i.e., node 9. However, since node 9 has already been traversed, traversal is no longer continued, and the process of selecting a branch to be processed begins. Since the first container is empty at this time, a branch to be processed is selected from the second container. For the branch that entered last among all branches in the second container, i.e., branch 7, it is determined whether branch 7 meets the conditions for processing the convergence node. Since node 8 has been processed and node 10 has not been processed among the adjacent nodes in the input direction of node 9, branch 7 meets the conditions for processing the convergence node, and branch 7 is selected as the branch to be processed. After branch 7 is processed, the second markers of nodes 10 and 9 are modified from unprocessed to processed, and an execution sequence part E corresponding to branch 7 is generated.

[0111] like Figure 4F As shown, nodes 10 and 9 have been processed, branch 7 has been removed from the second container, and an execution sequence part E corresponding to branch 7 has been generated. Execution sequence part E is connected to execution sequence parts C and D according to the data flow direction (or input-output relationship). Then, in the sixth branch search processing operation, the current node is the adjacent node in the output direction of the endpoint node of branch 7, i.e., node 4. However, since node 4 has already been traversed, it is no longer traversed, and the process of selecting a branch to be processed begins. Since the first container is empty at this time, a branch to be processed is selected from the second container. For the branch that entered last among all branches in the second container, i.e., branch 4, it is determined whether branch 4 meets the conditions for processing the convergence node. Since node 9 has been processed and node 3 has not been processed among the adjacent nodes in the input direction of node 4, branch 4 meets the conditions for processing the convergence node, and branch 4 is selected as the branch to be processed. After processing branch 4, the second markers of nodes 3 and 4 are modified from unprocessed to processed, and an execution sequence part F corresponding to branch 4 is generated.

[0112] It should be noted that if branch 4 enters the second container first and branch 5 enters the second container later, then branch 5 does not meet the processing conditions of the convergence node. This is because among the adjacent nodes in the input direction of node 6, neither node 4 nor node 5 has been processed. Therefore, branch 3 is still selected as the branch to be processed.

[0113] like Figure 4GAs shown, nodes 3 and 4 have been processed, branch 4 has been removed from the second container, and an execution sequence part F corresponding to branch 4 has been generated. Execution sequence part F is connected to execution sequence parts E and B according to the data flow direction (or input-output relationship). Then, in the seventh branch search processing operation, the current node is the adjacent node in the output direction of the endpoint node of branch 4, namely nodes 6 and 11. However, since node 6 has already been traversed, its related adjacent nodes are no longer traversed. Node 11 itself is the end node of the computation graph, thus branch 8, including node 11, is obtained. Since branch 8 does not include the convergence node, it enters the first container; simultaneously, the first marker of node 11 is modified from untraversed to traversed. Since there are still branches unprocessed in the first container at this time, branch 8 is selected as the branch to be processed. After branch 8 is processed, the second marker of node 11 is modified from unprocessed to processed, and an execution sequence part G corresponding to branch 8 is generated.

[0114] like Figure 4H As shown, node 11 has been processed, branch 8 has been removed from the first container, and an execution sequence part G corresponding to branch 8 has been generated. Execution sequence part G is connected to execution sequence part F according to the data flow direction (or input-output relationship). Afterwards, in the eighth branch search processing operation, node 11 is the end node of the computation graph and traversal stops; the process of selecting a branch to be processed begins. Since the first container is empty at this time, a branch to be processed is selected from the second container. At this time, the second container only contains one branch, 5, which is selected as the branch to be processed. After branch 5 is processed, the second markers of nodes 5, 6, and 12 are modified from unprocessed to processed, and an execution sequence part H corresponding to branch 5 is generated.

[0115] like Figure 4I As shown, execution sequence part H is connected to execution sequence parts B and F according to the data flow (or input-output relationship), and all nodes have been processed. Therefore, the final output execution sequence can be generated after one traversal.

[0116] For example, when multiple branches are searched in a single branch search operation, the order in which these branches are placed into the branch container is not restricted. For instance, taking the second branch search operation as an example, the order in which branches 2 and 3 are placed into the first container is not restricted. Alternatively, branch 3 can enter the first container first, followed by branch 2. In this case, branch 3 can be processed first, and the subsequent processing order may be slightly different, but the principle is the same, and the result is also the same. This will not be elaborated further here.

[0117] For example, in some embodiments, operator fusion processing is also performed when generating the execution sequence, and whether a branch is subject to operator fusion processing is determined by the fusion strategy.

[0118] For example, branch 7 can undergo operator fusion processing. Branch 7 is first processed by operator fusion into a fused execution unit, and then its input-output relationship is established to obtain the execution sequence part E.

[0119] For example, branch 4 can undergo operator fusion processing. Branch 4 is first processed by operator fusion into a fused execution unit, and then its input-output relationship is established to obtain the execution sequence part F.

[0120] For example, branch 5 can undergo operator fusion processing. Branch 5 is first processed into a fused execution unit, and then its input-output relationship is established to obtain the execution sequence part H. For example, in some other examples, according to the fusion strategy, nodes 5 and 6 in branch 5 can be merged into a fused execution unit, while node 12 is a separate fused execution unit. In this case, after processing, branch 5 can obtain two sequentially adjacent execution sequence parts. That is to say, in the embodiments of this disclosure, each branch can correspond to at least one execution sequence part. The specific process will not be described in detail.

[0121] Furthermore, in Figures 4A-4I In the illustrated embodiment, one branch is selected as the branch to be processed each time. Of course, two or more branches can also be selected as the branches to be processed. For example, a threshold upper limit can be set, such as selecting a maximum of 2 branches as the branches to be processed. For example, if there are multiple branches in the first container, two branches are selected as the branches to be processed. Then, the adjacent node in the output direction of the endpoint node of the two branches is taken as the current node, and the next round of branch search processing operation continues. For example, if there is one branch in the first container, that branch is selected as the branch to be processed. Alternatively, the branch in the first container and one branch in the second container can be selected as the branches to be processed. The specific process will not be described in detail.

[0122] For example, for Figure 4I or Figure 2B The execution sequence shown can be divided into several parts, each of which can constitute a fusion layer in a neural network to perform computations. The execution sequence represents the data dependencies and execution order between multiple fusion layers in the neural network.

[0123] In the data processing method described above, a first and second marker are added to each node to help indicate the state of each node (whether it has been traversed and processed). During the traversal, branches that do not include the sink node are processed first to generate their corresponding execution sequence parts. Thus, the execution sequence can be generated in one traversal, and operator fusion can also be completed. Compared with methods such as topological sorting, this reduces the number of graph traversals and achieves more efficient traversal of the computation graph and generation of execution sequences.

[0124] Currently, the data processing method provided in this disclosure has been successfully applied to networks such as VGG16, ResNet50, YOLOv5, and BERT. In practice, it has also been proven that the computation graph processing method provided in this disclosure can efficiently handle complex network structures and quickly generate the corresponding execution sequence of the network.

[0125] At least one embodiment of this disclosure also provides a data processing apparatus. Figure 5 This is a schematic block diagram of a data processing apparatus provided for at least one embodiment of the present disclosure.

[0126] like Figure 5 As shown, the data processing device 100 may include an acquisition unit 101 and a processing unit 102.

[0127] For example, these units can be implemented through hardware (e.g., circuit) modules, software modules, or any combination of both, as is the case in the following embodiments, and will not be repeated here. For example, these units can be implemented through a central processing unit (CPU), a data processor (GPU), a tensor processor (TPU), a field-programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, along with corresponding computer instructions.

[0128] For example, acquisition unit 101 is configured to acquire a computation graph. For example, the computation graph may include multiple nodes. The concept of a computation graph can be found in the preceding content and will not be repeated here.

[0129] For example, processing unit 102 is configured to traverse the computation graph sequentially according to the data flow, branch by branch, to generate an execution sequence corresponding to the computation graph that is executed in chronological order and has data dependencies.

[0130] For example, the traversal process executed by processing unit 102 includes at least one branch search processing operation. When the processing unit executes each branch search processing operation, it includes the following operations: obtaining at least one branch corresponding to the branch search processing operation, wherein each branch includes at least one node; adding at least one branch to a branch container; obtaining M branches from the branch container as M branches to be processed, wherein when obtaining the M branches to be processed, branches in the branch container that do not include the sink node are preferentially selected as branches to be processed, the sink node is a node with multiple inputs, and M is a positive integer; processing the M branches to be processed to generate the parts in the execution sequence corresponding to the computation graph that correspond to the M branches to be processed respectively.

[0131] For example, the acquisition unit 101 and processing unit 102 may include code and programs stored in memory; the processor may execute the code and programs to implement some or all of the functions of the acquisition unit 101 and processing unit 102 as described above. For example, the acquisition unit 101 and processing unit 102 may be dedicated hardware devices used to implement some or all of the functions of the acquisition unit 101 and processing unit 102 as described above. For example, the acquisition unit 101 and processing unit 102 may be a circuit board or a combination of multiple circuit boards used to implement the functions described above. In the embodiments of this application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) processor-executable firmware stored in memory.

[0132] It should be noted that the acquisition unit 101 can be used to implement Figure 1 The processing unit 102 can be used to implement step S10 as shown. Figure 1 The step S20 is shown. Therefore, for a detailed description of the functions that the acquisition unit 101 and the processing unit 102 can achieve, please refer to the relevant descriptions of steps S10 to S20 in the embodiments of the above data processing method; repeated details will not be repeated here. Furthermore, the data processing device 100 can achieve similar technical effects to the aforementioned data processing method, which will not be described further here.

[0133] Furthermore, for a detailed description of the functions that the execution unit can perform when performing the branch search processing operation, please refer to the relevant descriptions of steps S201 to S204 in the embodiments of the above data processing method. Repeated descriptions will not be repeated here.

[0134] Regarding the specific implementation process of the data processing apparatus according to the embodiments of this disclosure, reference can be made to the above. Figures 1-4 The data processing methods described according to some embodiments of this disclosure will not be repeated here. The data processing apparatus using embodiments of this disclosure can perform similar data processing procedures and achieve similar technical effects.

[0135] It should be noted that, in the embodiments of this disclosure, the data processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited; it can be constructed from analog devices, digital chips, or other suitable methods according to circuit principles.

[0136] At least one embodiment of this disclosure also provides an electronic device. Figure 6 This is a schematic diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0137] For example, such as Figure 6 As shown, the electronic device includes a processor 201, a communication interface 202, a memory 203, and a communication bus 204. The processor 201, communication interface 202, and memory 203 communicate with each other via the communication bus 204. The processor 201, communication interface 202, and memory 203 can also communicate with each other via a network connection. This disclosure does not limit the type and function of the network. It should be noted that... Figure 6 The components of the electronic device shown are merely exemplary and not limiting; the electronic device may have other components depending on the actual application requirements.

[0138] For example, memory 203 is used to store computer-readable instructions non-transitory. When processor 201 executes the computer-readable instructions, it implements the data processing method according to any of the above embodiments. For specific implementations and explanations of the various steps of this data processing method, please refer to the embodiments of the data processing method described above, and they will not be repeated here.

[0139] For example, other implementations of the data processing method implemented by the processor 201 executing computer-readable instructions stored in the memory 203 are the same as those mentioned in the foregoing method embodiment section, and will not be repeated here.

[0140] For example, the communication bus 204 can be a Peripheral Component Interconnect Standard (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0141] For example, communication interface 202 is used to enable communication between electronic devices and other devices.

[0142] For example, processor 201 and memory 203 can be located on the server side (or in the cloud).

[0143] For example, processor 201 can control other components in an electronic device to perform desired functions. Processor 201 can be a central processing unit (CPU), network processor (NP), tensor processor (TPU), or graphics processing unit (GPU) with data processing and / or program execution capabilities; it can also be a digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The central processing unit (CPU) can be based on x86 or ARM architectures, etc.

[0144] For example, memory 203 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and processor 201 may execute the computer-readable instructions to implement various functions of the electronic device. Various application programs and various data may also be stored in the storage medium.

[0145] For example, a detailed description of the process by which an electronic device performs data processing can be found in the relevant descriptions in the embodiments of the data processing method, and repeated descriptions will not be repeated here.

[0146] Figure 7 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. For example, such as Figure 7 As shown, storage medium 300 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 301 can be stored non-transitory. For example, when the computer-readable instructions 301 are executed by a processor, one or more steps in the data processing method described above can be performed.

[0147] For example, the storage medium 300 can be used in the aforementioned electronic device, such as the storage medium 300 may include the memory in the electronic device.

[0148] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0149] For example, the description of storage medium 300 can be found in the description of memory in the embodiments of the electronic device, and repeated descriptions will not be repeated here.

[0150] In summary, some embodiments of this disclosure provide a data processing method, apparatus, electronic device, and storage medium for generating general execution sequences for data processing that can be represented by directed acyclic graphs (e.g., various types of neural network structures, especially neural networks with complex structures). These sequences can generate execution sequences with data dependencies and operator fusion capabilities with a single traversal, thereby achieving efficient computation of neural networks.

[0151] Those skilled in the art will understand that the contents disclosed herein can be varied and modified in many ways. For example, the various devices or components described above can be implemented in hardware, or in software, firmware, or a combination of some or all of the three.

[0152] Furthermore, while this disclosure makes various references to certain elements of systems according to embodiments of this disclosure, any number of different elements may be used and operated on clients and / or servers. Elements are merely illustrative, and different aspects of the system and method may use different elements.

[0153] This disclosure uses flowcharts to illustrate the steps of a method according to embodiments of this disclosure. It should be understood that the preceding or following steps are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes.

[0154] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This disclosure is not limited to any particular combination of hardware and software.

[0155] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should also be understood that terms such as those defined in a common dictionary should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0156] The foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it. While several exemplary embodiments of the present disclosure have been described, those skilled in the art will readily understand that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined by the claims. It should be understood that the foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A data processing method comprising: obtaining a computation graph, wherein the computation graph comprises a plurality of nodes; traversing the computation graph in a data flow direction in a sequence of branches to generate an execution sequence corresponding to the computation graph, the execution sequence being executed in a time sequence and having a data dependency relationship, wherein the traversing comprises at least one branch searching processing operation, each branch searching processing operation comprising: obtaining at least one branch corresponding to the branch searching processing operation, wherein each branch comprises at least one node; adding the at least one branch to a branch container; obtaining M branches from the branch container as M to-be-processed branches, wherein when obtaining the M to-be-processed branches, branches in the branch container that do not comprise a sink node are preferentially selected as to-be-processed branches, the sink node being a node having a plurality of inputs, and M being a positive integer; processing the M to-be-processed branches to generate an execution sequence part corresponding to the M to-be-processed branches in the execution sequence corresponding to the computation graph, wherein the obtaining the at least one branch corresponding to the branch searching processing operation comprises: determining N current nodes, wherein the N current nodes are start nodes of the computation graph or one or more adjacent nodes in an output direction of a terminal node of a to-be-processed branch processed in a previous branch searching processing operation, and N is a positive integer; for each current node, traversing the computation graph in the data flow direction with the current node as a starting node to obtain a branch corresponding to the current node; adding the N branches corresponding to the N current nodes as the at least one branch corresponding to the branch searching processing operation, wherein the traversing the computation graph in the data flow direction with the current node as the starting node to obtain the branch corresponding to the current node comprises: in response to the current node satisfying a terminal node condition, determining that the branch corresponding to the current node comprises the current node; in response to the current node not satisfying the terminal node condition, continuing to search for a node satisfying the terminal node condition from one or more nodes having a direct or indirect adjacency relationship with the current node in the data flow direction, and determining that the branch corresponding to the current node comprises nodes between the current node and the node satisfying the terminal node condition, the current node, and the node satisfying the terminal node condition; wherein the node satisfying the terminal node condition comprises an end node of the computation graph, a branch node, or an adjacent node in an input direction of a traversed node, the branch node being a node having a plurality of adjacent nodes in an output direction; wherein the branch container comprises a first container and a second container, the adding the at least one branch to the branch container comprises: for each branch: determining that the branch does not comprise a sink node, and placing the branch in the first container; determining that the branch comprises a sink node, and placing the branch in the second container.

2. The data processing method of claim 1, wherein, the obtaining the M branches from the branch container as the M to-be-processed branches comprises: In response to the presence of the branch in the first container, M branches in the first container are obtained as the M to-be-processed branches. In response to the absence of the branch in the first container, M branches in the second container are obtained as the M to-be-processed branches.

3. The data processing method of claim 2, wherein, In response to the absence of the branch in the first container, M branches in the second container are obtained as the M to-be-processed branches, including: From all the branches stored in the second container, one branch that is last put into the second container among one or more branches satisfying a sink node processing condition is selected as a to-be-processed branch; In response to only one adjacent node of the sink node input direction in any one of the branches being traversed but not processed, and all the other adjacent nodes being traversed and processed, it is determined that the any one branch satisfies the sink node processing condition.

4. The data processing method of claim 1, wherein, Each node has a first mark for indicating whether the node is traversed, In the process of the traversal, in response to the node being traversed and having belonged to any one branch, the first mark of the node is modified from not traversed to having been traversed.

5. The data processing method according to any one of claims 1 to 4, wherein, Each node has a second mark for indicating whether the node is processed; Processing the M to-be-processed branches to generate an execution sequence part corresponding to each of the M to-be-processed branches in an execution sequence corresponding to the computation graph, including: For each to-be-processed branch: The to-be-processed branch is removed from the branch container; An input-output relationship of the to-be-processed branch is established to generate an execution sequence part corresponding to the to-be-processed branch in the execution sequence; The second mark of one or more nodes included in the to-be-processed branch is marked as processed.

6. The data processing method according to any one of claims 1 to 4, wherein, Processing the M to-be-processed branches to generate an execution sequence part corresponding to each of the M to-be-processed branches in an execution sequence corresponding to the computation graph, including: For each to-be-processed branch: The to-be-processed branch is removed from the branch container; The to-be-processed branch is subjected to operator fusion processing to obtain at least one fused execution unit; An input-output relationship of the at least one fused execution unit is re-established to generate an execution sequence part corresponding to the to-be-processed branch in the execution sequence, where the number of the execution sequence parts corresponding to the to-be-processed branch is at least one, and the at least one execution sequence part corresponds to the at least one fused execution unit in a one-to-one manner; The second mark of one or more nodes included in the to-be-processed branch is marked as processed.

7. The data processing method according to any one of claims 1 to 4, wherein, In branch units, the computation graph is traversed in sequence according to a data flow direction to generate an execution sequence corresponding to the computation graph and having a time sequence execution and a data dependency relationship, including: An execution sequence part corresponding to each to-be-processed branch in the at least one branch search processing operation is obtained; All the execution sequence parts obtained by the at least one branch search processing operation are combined according to a generation order and an input-output relationship to obtain an execution sequence corresponding to the computation graph.

8. The data processing method according to any one of claims 1 to 3, wherein, The first container is a linear storage structure following the first-in first-out principle.

9. The data processing method according to any one of claims 1 to 4, wherein, The computation graph is a directed acyclic graph representing data relationships of the neural network. Each part of the execution sequence corresponding to a to-be-processed branch constitutes a fusion layer, The neural network is executed layer by layer based on the computation of the execution sequence in units of the fusion layer.

10. A data processing apparatus, comprising: an acquisition unit configured to acquire a computation graph, wherein the computation graph comprises a plurality of nodes; a processing unit configured to traverse the computation graph in a data flow direction in units of branches to generate an execution sequence corresponding to the computation graph and having a time sequence and a data dependency relationship, wherein the traversal process comprises at least one branch searching processing operation, and the processing unit performs each branch searching processing operation by performing the following operations: acquiring at least one branch corresponding to the branch searching processing operation, wherein each branch comprises at least one node; adding the at least one branch to a branch container; acquiring M branches from the branch container as M to-be-processed branches, wherein when the M to-be-processed branches are acquired, branches in the branch container that do not include a sink node are preferentially selected as to-be-processed branches, the sink node is a node having multiple inputs, and M is a positive integer; processing the M to-be-processed branches to generate parts of the execution sequence corresponding to the M to-be-processed branches, respectively, wherein acquiring the at least one branch corresponding to the branch searching processing operation comprises: determining N current nodes, wherein the N current nodes are start nodes of the computation graph or one or more adjacent nodes in an output direction of an end node of a to-be-processed branch processed in a previous branch searching processing operation, and N is a positive integer; for each current node, traversing the computation graph in a data flow direction with the current node as a starting node to obtain a branch corresponding to the current node; adding the N branches corresponding to the N current nodes as the at least one branch corresponding to the branch searching processing operation, wherein for each current node, traversing the computation graph in a data flow direction with the current node as a starting node to obtain a branch corresponding to the current node comprises: in response to the current node satisfying an end node condition, determining that the branch corresponding to the current node includes the current node; in response to the current node not satisfying the end node condition, continuing to search for a node satisfying the end node condition from one or more nodes having a direct or indirect adjacency relationship with the current node in the data flow direction, and determining that the branch corresponding to the current node includes nodes between the current node and the node satisfying the end node condition, the current node, and the node satisfying the end node condition; wherein the node satisfying the end node condition includes an end node of the computation graph, a branch node, or an adjacent node in an input direction of a traversed node, and the branch node is a node having multiple adjacent nodes in an output direction; wherein the branch container comprises a first container and a second container, adding the at least one branch to a branch container, including: for each branch: determining that the branch does not include a sink node, placing the branch in the first container; determining that the branch includes a sink node, placing the branch in the second container. 11.An electronic device, comprising: a memory that stores, non-transitorily, computer-executable instructions; a processor that is configured to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the data processing method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium, wherein, The non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the data processing method according to any one of claims 1-9.

Images