Operator fusion method, apparatus, device and computer readable storage medium

Abstract

The application relates to the technical field of artificial intelligence, in particular to an operator fusion method and device, equipment and a computer readable storage medium, the method comprising the following steps: determining whether there are to-be-split current nodes and upstream nodes which are directly connected and different in data arrangement type in a calculation graph; if there are, constructing a first subgraph containing a first associated node by unidirectional search starting from the current node, and constructing a second subgraph containing a second associated node by bidirectional search starting from the upstream node; taking the first subgraph and the second subgraph as a new calculation graph, and iteratively detecting until there are no directly connected nodes which are different in data arrangement type; if there are multiple first subgraphs, merging the first subgraphs with the same graph attribute to obtain a third subgraph to realize operator fusion. The application aims to improve the efficiency of operator fusion.

Description

Operator fusion method, apparatus, device and computer-readable storage medium Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an operator fusion method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] In the field of deep learning, computation graph optimization and operator fusion techniques are key directions for improving model computation performance and fully utilizing the efficiency of GPU (Graphics Processing Unit) hardware resources. With the continuous increase in the complexity of deep learning models, the number of computation nodes in the computation graph has grown significantly, and different nodes correspond to diverse operation types and data arrangement methods. How to achieve efficient operator fusion by optimizing the computation graph structure has become an important research topic in the industry.

[0003] Currently, operator fusion techniques commonly employ algorithms such as depth-first search to traverse the computation graph. However, this method has significant limitations: to ensure coverage of all fusion possibilities, the traversal process requires handling a massive number of computational paths, resulting in enormous computational load and low algorithm efficiency. This makes it difficult to meet the real-time and high-performance requirements of practical applications, particularly in the optimization of computational graphs for complex models.

[0004] The aforementioned problems make it difficult for those skilled in the art to efficiently achieve the optimization goal of operator fusion when faced with highly complex computational graphs, and there is an urgent need for better technical solutions to break through the bottlenecks of computational load and efficiency.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide an operator fusion method, apparatus, device, and storage medium, which aims to improve the efficiency of operator fusion in complex scenarios.

[0007] To achieve the above objectives, this application provides an operator fusion method, which includes:

[0008] Determine whether there are two first nodes in the computation graph that have different data arrangement types and are directly connected, where the two first nodes are the current node to be split and the upstream node of the current node;

[0009] If it exists, a downstream unidirectional search is performed in the computation graph starting from the current node to obtain the first associated node that has a relationship with the current node, and the first subgraph is constructed based on the current node and the first associated node.

[0010] In the computation graph, a bidirectional search is performed upstream and downstream starting from the upstream node to obtain the second associated node that has a relationship with the upstream node. The second subgraph is then constructed based on the upstream node and the second node.

[0011] Both the first and second subgraphs are treated as new computation graphs, and the steps of determining whether there are two first nodes with different data arrangement types and directly connected are performed according to the new computation graphs, until it is detected that there are no two first nodes with different data arrangement types and directly connected in the latest computation graph.

[0012] If there are multiple second subgraphs, all second subgraphs belonging to the same graph attribute will be merged to obtain a third subgraph, so that the operator fusion in the computation graph can be achieved through the third subgraph.

[0013] Furthermore, to achieve the above objectives, this application also provides an operator fusion apparatus, the operator fusion apparatus comprising:

[0014] The node detection module is used to determine whether there are two first nodes in the computation graph that have different data arrangement types and are directly connected. The two first nodes are the current node to be split and the upstream node of the current node.

[0015] The first subgraph construction module is used to perform a downstream unidirectional search in the computation graph starting from the current node if it exists, to obtain the first associated node that has a relationship with the current node, and to construct the first subgraph based on the current node and the first associated node.

[0016] The second subgraph construction module is used to perform a bidirectional search in the computation graph, starting from the upstream node, to obtain the second associated node that is related to the upstream node, and to construct the second subgraph based on the upstream node and the second node.

[0017] The iterative processing module is used to treat both the first subgraph and the second subgraph as new computation graphs, and to perform the step of determining whether there are two first nodes with different data arrangement types and direct connection in the computation graph according to the new computation graph, until it is detected that there are no two first nodes with different data arrangement types and direct connection in the latest computation graph.

[0018] The subgraph merging module is used to merge all second subgraphs belonging to the same graph attribute if multiple second subgraphs exist, to obtain a third subgraph, so as to achieve operator fusion in the computation graph through the third subgraph.

[0019] Each functional module of the operator fusion apparatus of this application implements the steps of the operator fusion method of this application as described above during operation.

[0020] In addition, to achieve the above objectives, this application also provides an operator fusion device, which includes a memory, a processor, and an operator fusion program stored in the memory and executable on the processor. When the operator fusion program is executed by the processor, it implements the steps of the above-described operator fusion method.

[0021] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, and stores an operator fusion program thereon, which, when executed by a processor, implements the steps of the above-described operator fusion method.

[0022] This application provides an operator fusion method. This method determines whether there are two directly connected first nodes with different data arrangement types in the computation graph. These two first nodes are the current node to be split and its upstream node, allowing for precise location of the subgraph boundary to be split. If such a node exists, a downstream unidirectional search is performed in the computation graph starting from the current node to obtain the first associated node, and a first subgraph is constructed based on the current node and the first associated node, ensuring that all nodes in the subgraph maintain a consistent data arrangement with the current node. A bidirectional upstream and downstream search is then performed in the computation graph starting from the upstream node to obtain the second associated node, and a second subgraph is constructed based on the upstream node and the second node, ensuring complete data fusion. The algorithm covers all associated nodes related to the upstream node to ensure the integrity of the upstream subgraph. It treats both the first and second subgraphs as new computation graphs and performs steps to determine if two directly connected first nodes with different data layout types exist in the new computation graph, continuing until no such nodes are found in the latest graph. This achieves full optimization of the computation graph. If multiple second subgraphs exist, all second subgraphs belonging to the same graph attribute are merged to obtain a third subgraph. This reduces the total number of subgraphs while ensuring fusion effectiveness, allowing operator fusion within the computation graph to be achieved through the third subgraph. This ensures independent subgraph fusion, reduces invalid computations and dependency conflicts, and improves overall efficiency. Attached Figure Description

[0023] Figure 1 is a flowchart of the first embodiment of the operator fusion method of this application;

[0024] Figure 2 is a flowchart illustrating the process of determining the absence of two first nodes using the operator fusion method of this application;

[0025] Figure 3 is a schematic diagram of the upstream and downstream bidirectional search involved in the embodiment of this application;

[0026] Figure 4 is a schematic diagram of the upper and lower partitions involved in the embodiment of this application;

[0027] Figure 5 is a schematic diagram of the construction of the first and second sub-graphs involved in the embodiments of this application;

[0028] Figure 6 is a schematic diagram of the existence of multiple second sub-diagrams in the embodiments of this application;

[0029] Figure 7 is a schematic diagram of the merged third sub-diagram involved in the embodiment of this application;

[0030] Figure 8 is a schematic diagram of the operator fusion device involved in the embodiment of this application;

[0031] Figure 9 is a schematic diagram of the operator fusion device involved in the embodiment of this application.

[0032] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0033] This application provides an operator fusion method. Referring to FIG1, FIG1 is a flowchart of the first embodiment of the operator fusion method of this application.

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0035] Existing operator fusion algorithms generally employ depth-first search to traverse all possible computational graph structures, then achieve node fusion through subgraph partitioning. While this full-search strategy covers all fusion possibilities, the computational cost increases exponentially with the number of nodes, leading to low algorithm efficiency and making it difficult to meet the real-time requirements of industrial scenarios. Furthermore, existing solutions lack a dynamic adjustment mechanism for subgraph partitioning, often resulting in suboptimal node configurations after fusion. Additionally, the partitioning process does not adequately verify the acyclicity of the graph structure, potentially creating cyclically dependent subgraphs that violate the topological constraints of operator fusion. Moreover, the lack of dynamic limits on the total number of nodes in the computational graph leads to redundant paths in the search process, further exacerbating the waste of computational resources.

[0036] It is worth noting that existing technologies have significant shortcomings in judging data arrangement suitability. The data storage format of different nodes in the computation graph (such as row-major or column-major) directly affects the feasibility of fusion. Traditional algorithms either ignore the differences in data arrangement and force fusion, leading to a surge in data conversion overhead, or miss potential fusion opportunities due to crude judgment logic. This neglect of data arrangement characteristics makes the actual benefits of operator fusion far lower than theoretical expectations.

[0037] Therefore, in order to solve the technical problems of low search efficiency, unreasonable subgraph partitioning, easy formation of loop structure and lack of node number control in existing operator fusion algorithms, this application provides an operator fusion method, apparatus, device and storage medium.

[0038] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, database system, etc., or a device capable of performing the above functions, such as an operator fusion device. The following description uses an operator fusion device as an example to illustrate this embodiment and the subsequent embodiments.

[0039] The operator fusion method of this application includes the following implementation steps S10 to S60.

[0040] Step S10: Determine whether there are two first nodes in the computation graph that have different data arrangement types and are directly connected, wherein the two first nodes are the current node to be split and the upstream node of the current node; if they exist, proceed to step S20, otherwise proceed to step S30, wherein step S20: perform a downstream unidirectional search in the computation graph starting from the current node to obtain the first associated node that has a relationship with the current node, and construct the first subgraph based on the current node and the first associated node; step S30: perform operator fusion processing on all nodes in the computation graph to obtain the fused computation graph.

[0041] Optionally, referring to Figure 2, which is a flowchart illustrating the process of determining the absence of two first nodes using the operator fusion method of this application, in step S10, by detecting the current node and its upstream node that are directly connected and have different data arrangement types in the computation graph, triggering conditions are provided for subsequent subgraph splitting. It should be noted that the computation graph refers to a computational model composed of nodes and directed edges, used to describe the data processing flow. Here, a node represents a computational unit, such as an operator, and a directed edge represents the data flow between nodes. It should be noted that data arrangement type refers to the way data is organized in storage or computation, including but not limited to array dimension order (such as NHWC (N-Batch, H-Height, W-Width, C-Channels: batch-height-width-channel) format, NCHW (N-Batch, C-Channels, H-Height, W-Width: batch-channel-height-width) format, memory alignment (such as 32-byte alignment), data precision (such as FP32 (32-bit FloatingPoint: 32-bit floating-point number), FP16 (16-bit FloatingPoint: 16-bit floating-point number)), etc.). Direct connection means that there is a direct data dependency between two nodes, and there are no other intermediate nodes.

[0042] Furthermore, operator fusion must be based on nodes with consistent data arrangement types; otherwise, data conversion overhead will be introduced. Therefore, it is necessary to first locate node pairs that do not meet this condition. Detecting only directly connected nodes can reduce invalid calculations. Since there are intermediate nodes for indirectly connected nodes, their data arrangement differences can be handled through the intermediate nodes. By focusing on the current node and its upstream node, the starting point difference of data flow can be accurately located, ensuring that the data arrangement within the split subgraph is consistent.

[0043] Optionally, it can be determined whether there are current nodes and upstream nodes with different data arrangement types that are directly connected in the computation graph. The computation graph refers to a graph that represents the structure of a deep learning model, where nodes represent operations, edges represent data flows, and data arrangement type refers to the way data is stored in memory. The storage method includes, but is not limited to, row-first storage, column-first storage, and channel-first storage. Through diverse judgment strategies, it comprehensively covers the judgment needs in different scenarios, can efficiently and accurately identify target node pairs, provide a reliable basis for subsequent subgraph splitting, thereby reducing invalid fusion in the computation graph, reducing the amount of computation, optimizing the utilization of hardware resources such as GPUs, and ultimately improving the inference performance of deep learning models.

[0044] Optionally, in step S20, the computation graph is a graph representing the structure of a deep learning model, where nodes represent operations, edges represent data flows, the first associated node refers to a node that has a downstream association with the current node, and the first subgraph is composed of the current node and the first associated node. This reduces search complexity through targeted search, ordered filtering, and structured construction, provides a clear local structure for subgraph splitting and fusion, and helps improve GPU resource utilization and model inference performance.

[0045] Optionally, it should be noted that the criteria for determining relationships prioritize retaining nodes with direct data flow dependencies on the current node, extending to indirectly dependent nodes when necessary. These relationships specifically include, but are not limited to, retaining only directly downstream nodes, suitable for lightweight quantum graph scenarios, and recursively traversing to the point where no new downstream nodes are found, suitable for scenarios requiring complete link analysis. By accurately selecting related nodes and avoiding the inclusion of irrelevant nodes, the relevance and compactness of the subgraph are ensured.

[0046] Optionally, the construction rules of the first subgraph include the current node, the first associated node, and the edges between nodes, and the subgraph structure must support subsequent fusion. It should be noted that the subgraph must fully preserve the data flow relationships between nodes to provide a basis for subsequent splitting, so that the constructed subgraph can accurately reflect the local computational logic and lay the foundation for the effectiveness of operator fusion.

[0047] Step S40: Perform a bidirectional search in the computation graph, starting from the upstream node, to obtain the second associated node that is related to the upstream node. Construct the second subgraph based on the upstream node and the second node.

[0048] In this implementation, through bidirectional search and precise filtering, the relationship between upstream nodes is fully captured. The constructed second subgraph complements the first subgraph, jointly supporting the integrity and accuracy of subgraph splitting.

[0049] Specifically, optionally, a depth-first search (DFS) should be used, following the topological order, starting from the upstream node and allowing both upward and downward searches. The search stops at the entry node of the computation graph when searching upwards and stops when the search overlaps with a node in the first subgraph, avoiding duplication and ensuring a closed loop with the downstream unidirectional search. Topological order constraints prevent the generation of loops, while bidirectional search covers all associated nodes in the entire link, solving the problem of missing upstream dependencies in traditional unidirectional search and improving the completeness of relationship capture.

[0050] Optionally, based on data dependencies, only nodes that have direct or indirect data flow interactions with upstream nodes can be retained. Specifically, directly related nodes can be prioritized for lightweight splitting scenarios; or recursively traversing to the point where no new related nodes are found can be used for scenarios requiring complete link analysis. This approach accurately filters effective related nodes, avoids the inclusion of irrelevant nodes, ensures the compactness and relevance of the second subgraph, and provides a reliable foundation for subsequent subgraph merging.

[0051] Optionally, the construction rules for the second subgraph include upstream nodes, second associated nodes, and edges between nodes. The subgraph must be acyclic and have compatible data arrangements to prepare for subsequent merging. During construction, the node data arrangement type must be recorded synchronously to provide a basis for subsequent splitting verification with the first subgraph. The constructed second subgraph can fully reflect the association logic of upstream nodes and satisfy fusion constraints, providing structural guarantees for the effectiveness of subgraph splitting. This solves the problem of traditional unidirectional search missing upstream dependencies, improves the comprehensiveness of associated node identification, reduces the inclusion of invalid nodes, and reduces the computational load of subgraph processing.

[0052] Step S50: Treat both the first subgraph and the second subgraph as new computation graphs, and perform the step of determining whether there are two first nodes with different data arrangement types and directly connected in the computation graph according to the new computation graph, until it is detected that there are no two first nodes with different data arrangement types and directly connected in the latest computation graph.

[0053] In this implementation, the first and second subgraphs obtained from the splitting are used as new computational graphs. The step of detecting first nodes with different data arrangement types and direct connections is repeatedly executed to form an iterative process: if such node pairs still exist in the new computational graph, the splitting continues; until all subgraphs have no such node pairs, the iteration terminates. Its core purpose is to obtain subgraphs with consistent data arrangement that can be directly fused through layer-by-layer splitting and detection, laying the foundation for subsequent fusion operations.

[0054] Optionally, when the first and second subgraphs are used as new computation graphs, the core features of the original computation graph must be fully preserved, including nodes, edges, and data arrangement types. For example, the new computation graph must clearly record the data arrangement identifier of each node to ensure that the detection steps can directly reuse the original logic, thereby ensuring the feature consistency between the new and original computation graphs, making the iterative detection results reliable, and avoiding misjudgments due to missing information.

[0055] Optionally, each iteration follows the logic of "detection, splitting, and new graph generation," with each input being the subgraph split from the previous round and the output being the detection result or termination signal for the next round. For example, the first round of splitting yields subgraphs A and B; detecting subgraph A reveals incompatible node pairs, resulting in splitting A1 and A2; then, A1 and A2 are used as new computational graphs for repeated detection until no incompatible node pairs remain. By gradually eliminating node pairs that do not meet the fusion conditions through closed-loop iteration, the final subgraph satisfies the fusion prerequisite of consistent data arrangement, thus solving the problem of low node efficiency after fusion in traditional algorithms.

[0056] Optionally, the latest computation graph does not have two directly connected first nodes with different data arrangement types as a termination condition. For example, a counter or threshold can be set to assist in the judgment, avoiding infinite iteration. A clear termination condition ensures that the algorithm can be completed in a finite number of steps, balancing the optimization effect and computational cost, and improving engineering practicality.

[0057] In other words, by recursively detecting and splitting, a subgraph that meets the fusion conditions is finally obtained, which solves the problem of low efficiency caused by differences in data arrangement of nodes after fusion in traditional algorithms. By recursively eliminating non-compliant node pairs, efficient data interaction within the subgraph is ensured. The iterative process gradually simplifies the subgraph, reduces the amount of computation, and improves the utilization of GPU (Graphics Processing Unit) resources, thereby supporting the performance improvement goal of the entire algorithm.

[0058] Step S60: If there are multiple second subgraphs, all second subgraphs belonging to the same graph attribute are merged to obtain a third subgraph, so as to achieve operator fusion in the computation graph through the third subgraph.

[0059] Optionally, when multiple second subgraphs are obtained from the split, the same graph attribute refers to the common characteristics that allow for merging (such as consistent data arrangement type, compatible node operations, and no cycles after merging). By merging these subgraphs to form a third subgraph, the number of independent subgraphs in the computation graph can be reduced, avoiding the low fusion efficiency caused by fragmented subgraphs. Ultimately, centralized operator fusion is achieved through the third subgraph, improving computational performance.

[0060] Optionally, the specific criteria for determining the same graph attribute include, but are not limited to, the same data arrangement type (e.g., both are stored in row-major order), the subgraph structure is acyclic, and the node operations belong to the same computation category (e.g., both are convolution operations). This clear attribute standard ensures the accuracy of the merged objects, avoids data interaction conflicts introduced after merging subgraphs with different attributes, and guarantees the structural validity of the third subgraph.

[0061] Optionally, the merging process follows the principle of preserving effective features and eliminating redundant structures. This principle includes, but is not limited to, integrating the nodes and edges of the subgraph, deleting duplicate connections, and verifying whether a cycle is generated after merging. For example, when merging subgraphs A and B, if the output nodes of A and the input nodes of B have the same data arrangement, they are directly connected to form a third subgraph. At the same time, it is checked whether a cycle exists. This ensures that the merging process preserves the core computational logic of the subgraph, simplifies the overall structure, reduces the interaction overhead between nodes, and provides a more efficient basic unit for operator fusion.

[0062] Optionally, after merging, it is necessary to verify whether the third subgraph meets the fusion conditions to ensure that the third subgraph can be directly used for subsequent operator fusion. For example, by reusing the step of detecting differences in data arrangement, it can be verified that the merged subgraph has no non-compliant node pairs, so that the verification mechanism ensures the feasibility of fusion of the third subgraph, avoids the need for secondary splitting after merging, and improves the coherence of the algorithm flow.

[0063] In other words, by merging second subgraphs with the same attributes to generate a third subgraph, the structured integration of the computation graph is achieved, the problem of subgraph fragmentation is solved, the number of independent subgraphs is reduced, and the complexity of operator fusion is reduced; the third subgraph is ensured to have unified graph attributes (such as data arrangement and acyclicity), the execution efficiency of the fused operators is improved, and reliable support is provided for subsequent fusion steps.

[0064] In this embodiment, by determining whether there are two first nodes with different data arrangement types and directly connected in the computation graph, where the two first nodes are the current node to be split and the upstream node of the current node, the boundary of the subgraph to be split can be accurately located. If they exist, a downstream unidirectional search is performed in the computation graph starting from the current node to obtain the first associated node that is related to the current node. The first subgraph is then constructed based on the current node and the first associated node, ensuring that all nodes in the subgraph maintain the same data arrangement as the current node. A bidirectional upstream and downstream search is then performed in the computation graph starting from the upstream node to obtain the second associated node that is related to the upstream node. The second subgraph is then constructed based on the upstream node and the second node, which can completely cover the nodes related to the upstream node. All related nodes are checked to ensure the integrity of the upstream subgraph. The first and second subgraphs are treated as new computation graphs, and the process of determining whether there are two directly connected first nodes with different data layout types is performed according to the new computation graphs until the latest computation graph does not contain two directly connected first nodes with different data layout types. This achieves full optimization of the computation graph. If there are multiple second subgraphs, all second subgraphs belonging to the same graph attribute are merged to obtain a third subgraph. This reduces the total number of subgraphs while ensuring the effectiveness of fusion, so that operators in the computation graph can be fused through the third subgraph. This ensures independent fusion of subgraphs, reduces invalid computation and dependency conflicts, and improves overall efficiency.

[0065] Furthermore, based on the above, a second embodiment of the operator fusion method of this application is proposed. In some feasible embodiments, step S10 above: determining whether there are two first nodes with different data arrangement types and directly connected in the computation graph, further includes the following implementation steps B101~S105.

[0066] Step B101: Traverse each node in the computation graph based on the preset computation graph topological dependencies;

[0067] In this embodiment, all nodes are traversed according to a preset topological dependency relationship. The preset topological dependency relationship includes, but is not limited to, topological sorting. This ensures that the traversal order conforms to the data flow logic between nodes. The upstream nodes are traversed first, and then the downstream nodes that depend on their output are traversed. This provides an orderly processing basis for subsequent steps such as detecting the node data arrangement type and determining whether splitting is necessary.

[0068] Optionally, the predefined topological dependencies must be explicit and executable. For example, the node traversal order can be predefined using an adjacency list or a topological sorting algorithm (such as Kahn's algorithm) to ensure that each node is processed only after all its upstream dependent nodes have been traversed, avoiding traversal chaos caused by ambiguous dependencies. The predefined topological order ensures that the traversal logic is consistent with the data flow of the computation graph, preventing downstream nodes from being processed before upstream nodes, providing the correct node association for subsequent detection of the data arrangement type of upstream nodes, and reducing logical errors.

[0069] Optionally, it is necessary to ensure that the traversal covers all nodes in the computation graph, and that each node is traversed only once. For example, marking a node as "traversed" avoids redundant processing, ensures that no node is missed, and guarantees that subsequent detection covers the entire graph. It is evident that a full traversal guarantees that no node is missed, avoiding fusion obstacles caused by unprocessed nodes, such as undetected nodes with inconsistent data arrangements, and ensuring that the algorithm's optimization coverage of the computation graph is complete.

[0070] Optionally, the traversal results directly provide input for subsequent steps, such as determining the data layout type of a node and its upstream nodes. That is, only by traversing in topological order can the upstream nodes of the current node be accurately obtained, thus clarifying the dependencies. For example, if downstream nodes are traversed first, and their upstream nodes have not yet been processed, the data layout type of the upstream nodes cannot be obtained. Therefore, by associating traversal with topological dependencies and subsequent steps, it is ensured that operations such as data layout detection and subgraph splitting are based on correct node dependencies, improving the overall logical coherence and reliability of the algorithm.

[0071] Step B102: If the traversed node has an upstream node, determine the first data arrangement type identifier of the traversed node and the second data arrangement type identifier of the upstream node corresponding to the traversed node.

[0072] In this embodiment, when the traversed node has an upstream node, there is a direct data flow dependency. By obtaining the first data arrangement type identifier of the current node and the second data arrangement type identifier of the upstream node, a specific basis is provided for subsequent detection of whether the two are consistent. This is a prerequisite for determining whether subgraph splitting is necessary and the effectiveness of direct association operator fusion.

[0073] Optionally, the rules for generating the identifiers can be explicitly defined to ensure that they uniquely and accurately reflect the data arrangement type. For example, standardized identifiers can be preset to avoid vague descriptions; or hash values ​​can be dynamically generated as identifiers based on data arrangement characteristics (such as dimension order and alignment) to ensure that custom data arrangement types can also be accurately represented. Thus, explicit identifier rules guarantee the comparability of the first and second identifiers, avoid misjudgments due to inconsistent representations, and provide reliable input for subsequent detection steps.

[0074] Optionally, the "corresponding upstream node" can be defined as a "direct upstream node," meaning a node with a direct data flow connection to the current node, rather than an indirect upstream node. For example, if the input to node B comes directly from node A, then node A is a direct upstream node of node B, and only the identifier of node A needs to be obtained. If the input to node B comes indirectly from node A via node C, then node A is not a "corresponding upstream node," and its identifier does not need to be obtained. Therefore, by focusing on directly related nodes and reducing the identification operations of irrelevant nodes, the computational load is reduced, and the efficiency of the steps is improved.

[0075] Optionally, by traversing nodes based on topological dependencies, the traversal order is ensured to conform to the data flow logic, and a data foundation is provided for subsequent detection of consistent identifiers, forming a closed loop of "traversal, identifier acquisition, and difference detection." For example, only by traversing in topological order, upstream first and downstream second, can the upstream node be accurately found and its identifier acquired when traversing the current node; otherwise, the upstream node may not be traversed and its identifier cannot be acquired. It is evident that the logically coherent step design ensures the smoothness of the entire process, avoids step failures due to disordered order, and provides reliable preconditions for subsequent subgraph splitting or merging.

[0076] Step B103: Check whether the first data layout type identifier and the second data layout type identifier are consistent;

[0077] If they do not match, proceed to step B104; if they match, proceed to step B105.

[0078] In step B104: Determine that the data arrangement types between the traversed node and the upstream node corresponding to the traversed node are different, and take the traversed node as the current node to be split, and take the upstream node corresponding to the traversed node as the upstream node of the current node; Step B105: Keep the association relationship between the traversed node and the upstream node corresponding to the traversed node unchanged, and continue to traverse the next node in the computation graph until the traversal of all nodes in the computation graph is completed.

[0079] In this embodiment, node pairs are differentiated by detecting whether the data arrangement type identifier of the current node and the upstream node are consistent. The data arrangement type identifier is a marker representing the storage method. If they are inconsistent, they are marked as objects to be split, in preparation for subsequent subgraph splitting. If they are consistent, the relationship is preserved, in preparation for subsequent fusion, while continuing to traverse to cover all nodes in the graph to ensure no omissions.

[0080] Specifically, optionally, the detection of whether the first data arrangement type identifier and the second data arrangement type identifier are consistent needs to be based on preset comparison rules. These preset comparison rules include, but are not limited to, direct string matching and hash value similarity calculation. Therefore, clear detection rules ensure accurate difference judgment, avoid misjudgment due to ambiguity in comparison logic, and provide a reliable basis for subsequent processing.

[0081] Optionally, when the identifiers are inconsistent, the current node and its upstream node are marked as a "node pair to be split". This step requires specifying the attributes of the "to be split", such as recording the node ID and association relationship. If the marking information is missing, subsequent subgraph splitting will have no clear object. Therefore, accurately locating the node pairs that need to be split avoids computational redundancy caused by indiscriminate processing, provides a clear target for subsequent subgraph splitting, and improves the algorithm's targeting.

[0082] Optionally, when the identifiers match, the association is maintained and traversal continues. It should be noted that, on the one hand, node pairs with consistent data arrangement are potential fusion targets, and preserving the association avoids redundant connection construction later; on the other hand, continuing traversal ensures coverage of all nodes in the computation graph, leaving no fusionable node pairs untouched. Therefore, ensuring the integrity of the process—preserving valid associations to prepare for fusion while ensuring no nodes are missed through full graph traversal—improves the algorithm's comprehensiveness.

[0083] In this embodiment, each node in the computation graph is traversed based on a preset computation graph topology dependency relationship. If the traversed node has an upstream node, the first data arrangement type identifier of the traversed node is determined, and the second data arrangement type identifier of the upstream node corresponding to the traversed node is determined. It is then checked whether the first data arrangement type identifier and the second data arrangement type identifier are consistent. If they are inconsistent, it is determined that the data arrangement types between the traversed node and the upstream node corresponding to the traversed node are different, and the traversed node is taken as the current node to be split, and the upstream node corresponding to the traversed node is taken as the upstream node of the current node. Thus, through ordered traversal, accurate detection, and directional marking, efficient identification of operator fusion obstacles is achieved, providing a core basis for subsequent subgraph splitting and computation graph optimization, thereby improving operator fusion efficiency and optimizing GPU resource utilization.

[0084] Furthermore, based on the above content, in some feasible embodiments, the above step S20: performing a downstream unidirectional search in the computation graph starting from the current node to obtain the first associated node that has a relationship with the current node, and constructing a first subgraph based on the current node and the first associated node, may also include the following implementation steps B201 to B204.

[0085] Step B201: Determine the first branch in the computation graph that contains the current node and the first downstream node of the current node;

[0086] In this embodiment, the current node is the node to be split as marked above. Unlike the upstream node, the data arrangement type is different. The first downstream node refers to the direct downstream node of the current node, that is, the node whose output is directly used as its input. The first branch refers to the local data flow link formed by the current node, the first downstream node and the connecting edge between the two, reflecting the direct data interaction relationship between the two. This enables the accurate extraction of the local branch formed by the current node and its direct downstream node from the complex computation graph, providing a clear starting link for the subsequent downstream unidirectional search to construct the first subgraph, and avoiding the problem of excessive range and computational redundancy caused by full graph traversal.

[0087] Specifically, optionally, the first branch includes only the current node, its direct downstream nodes, and the direct connection edges between them, excluding indirect downstream nodes or nodes without direct association. For example, if the current node is A and its direct downstream node is B, then the first branch is "A→B", where the arrow indicates the direction of data flow between nodes in the computation graph, that is, the output data of node A serves as the input data of node B, reflecting the direct data flow dependency between the two nodes; if the downstream node of B is C, then C does not belong to this branch. It is evident that a clearly defined branch scope ensures that subsequent searches focus on directly related local links, avoiding the inclusion of irrelevant nodes, reducing computational load, and improving processing specificity.

[0088] Optionally, it should be noted that determining the first branch step follows the previously marked current node to be split, and provides a specific starting point for subsequent downstream unidirectional searches to obtain the first associated node and construct the first subgraph. In other words, by connecting the preceding and following steps through branch positioning, it ensures that the starting point of subgraph construction is clear and the scope is controllable, avoids confusion in the search direction, and guarantees the orderly splitting of subsequent subgraphs.

[0089] Step B202: In the first branch, perform a one-way downstream search starting from the current node until the last downstream node in the first branch is found;

[0090] In this embodiment, following the data flow direction of the first branch, all downstream nodes are traversed from the current node until the end of the branch, i.e., the last downstream node. The purpose is to fully capture all downstream nodes within the branch that have direct or indirect data flow dependencies with the current node, providing clear operation objects for subsequent node movement to the next partition, and ensuring that no related nodes within the branch are missed when splitting the subgraph.

[0091] Specifically, optionally, the search is limited to the first branch and does not extend to other branches, such as other downstream branches of the current node. This focuses on the target branch, avoids including irrelevant nodes, reduces computational redundancy, and improves search targeting.

[0092] Optionally, a downstream unidirectional search is adopted, which only moves along the direction of data flow and does not backtrack upstream. This conforms to the downward unidirectional search logic of the subgraph splitting algorithm and follows the topological dependency relationship of the computation graph, proceeding upstream first and then downstream. This avoids loops or repeated traversals caused by reverse search, thereby ensuring the orderliness of the search, avoiding infinite loops or repeated calculations, and improving search efficiency.

[0093] Specifically, optionally, the search is terminated when the last downstream node in the first branch is found. A clear termination condition avoids infinite searching, balances search completeness with computational cost, and enhances the practicality of the project.

[0094] Step B203: Move all downstream nodes found in the downstream unidirectional search as the first associated node to the preset lower partition containing the current node;

[0095] In this embodiment, all downstream nodes obtained through downstream unidirectional search in step B202 are moved to the lower partition that already contains the current node, so that the current node and the first associated node form an independent local subgraph, thereby achieving separation from the partition where the upstream node is located, laying the structural foundation for subsequent subgraph fusion without data arrangement differences.

[0096] Optionally, the first associated node is limited to all downstream nodes found in the downstream unidirectional search, that is, all downstream nodes within the first branch traversed in step B202, which must be completely included and not exceed this range. Therefore, ensuring the association and completeness of the nodes in the lower partition is crucial to avoid incomplete subgraph structure due to missing nodes, which would affect the effectiveness of subsequent fusion.

[0097] Optionally, the sub-partitions need to be pre-defined, such as through logical tags or memory partitioning, and by default include the current node. A move operation only needs to include the first associated node in this partition. This pre-defined partition ensures clear partition boundaries, avoiding confusion with the partitions containing upstream nodes. In other words, pre-defined partitions reduce the computational overhead of dynamic partitioning, while clearly defining the affiliation of the current node and the first associated node, providing a clear scope for independent processing of the subgraph, such as subsequent iterative detection.

[0098] Optionally, all downstream nodes need to be completely moved to the next partition to ensure that the data flow link between the current node and its first associated node is fully preserved within the partition. For example, the connection relationship A→B→C should remain unchanged within the next partition. If any node is omitted, the subgraph link will be broken, rendering the splitting meaningless. Therefore, ensuring the integrity of the data flow within the nodes of the next partition makes the partition a subgraph unit that can be processed independently, providing a complete object for subsequent detection of data arrangement differences within the subgraph.

[0099] In other words, by creating a unified sub-partition for the current node and its downstream associated nodes, and separating it from the upstream nodes, the problem of "low fusion efficiency caused by the mixing of nodes with different data layouts" is solved. The data flow links between nodes are fully preserved, providing reliable objects for subsequent iterative optimization within the subgraph. Through preset partitioning and complete movement operations, the computational redundancy of subgraph partitioning is reduced, which helps to improve the efficiency of the overall operator fusion process.

[0100] Step B204: Connect at least one first associated node in the lower partition to the current node according to the preset computation graph topology dependency relationship to obtain the first subgraph.

[0101] In this embodiment, the preset computation graph topological dependencies refer to the inherent data flow order between nodes, such as the dependency relationship where the output of an upstream node is the input of a downstream node; the lower partition already contains the current node and the first associated node. Based on the topological dependencies of the original computation graph, the current node in the lower partition is connected to at least one first associated node according to the original data flow logic to form a first subgraph containing nodes and connection relationships. This subgraph can fully reflect the local computation link between the current node and its downstream associated nodes, providing a structural basis for subsequent subgraph optimization and fusion.

[0102] Furthermore, in one embodiment, connecting at least one first associated node in the lower partition to the current node must adhere to a preset computational graph topology dependency relationship. This means maintaining the original data flow direction between nodes, such as the output of current node A → the input of first associated node B, and the output of B → the input of first associated node C. The connection order or direction cannot be arbitrarily changed. For example, if A→B→C is a given topology in the original computational graph, the upstream and downstream relationships between A and B, and B and C, must be preserved during connection. Connections between A→C or B→A cannot be constructed. This ensures that the structure of the first subgraph is consistent with the local logic of the original computational graph, preventing the subgraph from failing to reflect the actual computational process due to connection errors, and guaranteeing the effectiveness of subsequent processing.

[0103] Furthermore, in one embodiment, at least one first associated node must cover key nodes within the lower partition that have direct or indirect dependencies on the current node. All first associated nodes are connected preferentially to preserve the complete downstream link. For example, the current node A must be connected to both first associated nodes B and C, ensuring no necessary connections are omitted. In other words, this guarantees that the first subgraph contains complete downstream associated links, preventing logical breaks in the subgraph due to incomplete connections, which would hinder subsequent operator fusion for full-link optimization.

[0104] Furthermore, in one embodiment, by connecting nodes within a lower partition based on topological dependencies, following step B203, a first subgraph is formed through these connections. This provides a clear object for subsequent iterative detection using the first subgraph as a new computational graph. For example, only after connecting A, B, and C through topological dependencies can "A→B→C" be used as the first subgraph to detect whether there are differences in data arrangement within the subgraph. It is evident that the connection operation transforms isolated nodes within a lower partition into an organic whole, realizing the transformation from a "node set" to a valid subgraph and ensuring a coherent closed loop in the entire subgraph splitting process.

[0105] In this embodiment, by forming a local subgraph that reflects the real data flow between the current node and the first associated node, the problem of logical confusion caused by the lack of connection after node movement is solved; by following the original topological dependency, it is ensured that the first subgraph can accurately map the downstream links of the original computation graph, providing a reliable structural foundation for subsequent iterative detection; the complete subgraph structure enables subsequent fusion operations to be optimized for the entire link, helping to reduce data interaction overhead and improve GPU resource utilization.

[0106] Furthermore, based on the above, in some feasible embodiments, the upstream and downstream bidirectional search includes searching in the upstream direction and searching in the downstream direction. The above step B40: performing a bidirectional search in the computation graph starting from the upstream node to obtain a second associated node that has a relationship with the upstream node, and constructing a second subgraph based on the upstream node and the second node, also includes the following implementation steps B401-B410.

[0107] Step B401: Determine the second branch of the first upstream node in the computation graph that contains upstream nodes and upstream nodes, and determine whether the second branch satisfies the preset upstream search conditions;

[0108] If satisfied, proceed to step B402; otherwise, proceed to step B403.

[0109] Step B402: In the second branch, start from the upstream node and search upstream. When the upstream node found in the upstream search is the source node of the second branch, stop searching upstream.

[0110] Step B403: Stop the upstream search in the second branch, starting from the upstream node;

[0111] In this embodiment, the upstream node is a node whose data arrangement differs from the current node. It is necessary to capture its complete associated link by searching upstream to construct the second subgraph. The preset upstream search conditions include that the upstream node is not the source node of the second branch and that the upstream node's first upstream node does not belong to the lower partition. Specifically, the local data flow link formed by the upstream node and its first upstream node is locked, such as "upstream node U ← first upstream node P", where the arrow indicates the data flow direction, serving as the starting unit for the upstream search. Based on the two conditions—the upstream node not being the source node and the first upstream node not belonging to the lower partition—it is determined whether to initiate an upstream search. If the conditions are met, step B402 is executed; otherwise, step B403 is executed, forming a closed-loop logic of judgment and execution.

[0112] Specifically, optionally, the second branch is limited to the direct link between the upstream node and its first upstream node, excluding indirect upstream nodes, such as the upstream node of the first upstream node. For example, if the direct upstream of upstream node U is P, then the second branch only refers to the connection between U and P, avoiding including P's upstream node Q in the branch scope, preventing the search boundary from getting out of control, thereby accurately defining the starting boundary of the upstream search, ensuring that subsequent searches start from directly related nodes, and avoiding redundant calculations caused by ambiguous scope.

[0113] Optionally, two preset conditions restrict the search initiation from different dimensions. Specifically, the upstream node is not the source node of the second branch: if the upstream node is the source node, i.e., there is no node further upstream, then searching upwards is meaningless; this condition ensures the feasibility of the search. The first upstream node of the upstream node does not belong to the lower partition, i.e., the lower partition is the region where the current node and downstream nodes are located. If the first upstream node belongs to this region, it means it is already included in the first subgraph, and there is no need to search it again; this condition ensures the independence of the search. The combination of these two conditions forms a complete filtering logic, avoiding invalid searches due to missing conditions. Therefore, through complementary conditions, only scenarios with searchable nodes that have not yet been processed are retained, reducing the waste of computing resources.

[0114] Optionally, the judgment result can be used to directly connect steps B402 and B403, providing valid input for subsequent step B404. For example, if the second branch meets the condition, the upstream search in step B402 can obtain the upstream node Q of P, supplementing the associated nodes for the second subgraph; if it does not meet the condition, terminating the search in step B403 can avoid invalid traversal of the source node or the lower partition node. Therefore, by using a logical closed loop, the upstream search is ensured to "search when it should be searched and stop when it should be stopped," guaranteeing the completeness of the captured associated nodes while avoiding exceeding the necessary range.

[0115] In other words, by filtering meaningless search scenarios through conditional judgment, the problem of excessive computation caused by traditional full-graph upstream search is solved; it ensures that the nodes searched upstream are all within the range not covered by the lower partition, providing a clean set of nodes for the construction of the second subgraph and avoiding overlap with the first subgraph; it fully captures upstream related nodes, providing a structural foundation for the subsequent merging and fusion of the second subgraph, and helping to improve the integrity of operator fusion and GPU resource utilization.

[0116] Step B404: Determine the node found in the upstream search that matches the second data arrangement type identifier, and obtain the upstream second node that is associated with the upstream node;

[0117] In this embodiment, the second data arrangement type identifier is the data arrangement type mark of the upstream node, such as representing the way data is stored in memory. The key premise of operator fusion is that the data arrangement between nodes is consistent, which can reduce the time consumption of data conversion. Specifically, from all the nodes searched upstream in step B402, nodes that are consistent with the second data arrangement type identifier of the upstream node are selected. These nodes are defined as upstream second nodes, that is, they have a direct or indirect data flow relationship with the upstream node and their data arrangement is compatible. They are the core components of constructing the second subgraph and provide a set of nodes that meet the conditions for subsequent subgraph fusion.

[0118] Optionally, the selection criterion can be consistent with the identifier of the second data arrangement type, and the identifier comparison rules can be clearly defined, such as string matching, feature value consistency, etc. For example, if the second identifier of the upstream node is "row priority - R", then only upstream nodes with the same identifier "row priority - R" will be included in the upstream second node, and nodes with inconsistent identifiers such as "column priority - C" will be excluded. This ensures the accuracy of the selection results, avoids nodes with different data arrangements from being mixed into the second subgraph, and provides structural guarantees for the efficiency of subsequent fusion.

[0119] Optionally, the second upstream node is a node found during the upstream search, meaning it is a related node with a data flow dependency on the upstream node, and cannot include unrelated nodes. Therefore, it is essential to ensure the logical relationship between the second upstream node and the upstream node to avoid including irrelevant nodes and causing redundancy in the subgraph structure.

[0120] Step B405: Determine the third branch of the first downstream node in the computation graph that contains the upstream node and the upstream node, and determine whether the third branch satisfies the preset downstream search conditions;

[0121] If satisfied, proceed to step B406; otherwise, proceed to step B407.

[0122] Step B406: In the third branch, start from the upstream node and search downstream. When the downstream node found in the downstream search is a node of the lower partition, stop the downstream search.

[0123] Step B407: Stop the search in the downstream direction starting from the upstream node in the third branch;

[0124] In this embodiment, the upstream node needs to expand its associated links by searching downstream to construct a complete second subgraph, but it needs to avoid duplication with nodes that have been assigned to the lower partition (i.e., the area where the current node and downstream nodes are located). Specifically, the local data flow link formed by the upstream node and its first downstream node is locked as the starting unit for the downstream search. The preset downstream search conditions include that the first downstream node of the upstream node does not belong to the lower partition. Based on the preset condition that the first downstream node does not belong to the lower partition, it is determined whether to start the downstream search. If the condition is met, step B406 is executed; otherwise, step B407 is executed to ensure that the search range is limited to the area not covered by the lower partition.

[0125] Optionally, the third branch is the direct link between the upstream node and its first downstream node, excluding indirect downstream nodes. Therefore, accurately defining the starting boundary of the downstream search ensures that subsequent searches begin from directly related nodes, avoiding redundant calculations caused by ambiguous scope.

[0126] Optionally, the condition that the first downstream node of an upstream node does not belong to the lower partition is essentially to avoid redundant processing through region partitioning: the lower partition has already included the current node and its downstream nodes through steps B203-B204. If the first downstream node belongs to this region, it means that it has already been included in the first subgraph and does not need to be included in the upstream node's association link. Therefore, by filtering based on the condition, the search is only initiated for downstream nodes not covered by the lower partition, reducing the waste of computing resources and ensuring the independence of subgraph splitting, that is, there are no overlapping nodes between the upper and lower partitions.

[0127] Optionally, the result of determining whether the third branch meets the preset downstream search conditions directly connects to steps B406 and B407, and provides valid input for the subsequent step B408. For example, if the third branch meets the conditions, the downstream search in step B406 can obtain the downstream node E of D, supplementing the associated nodes for the second subgraph; if it does not meet the conditions, terminating the search in step B407 can avoid invalid traversal of the lower partition nodes. Therefore, by using a logical closed loop, the downstream search is ensured to "expand as needed," guaranteeing both the completeness of the associated node capture and control within the non-lower partition range, thus avoiding subgraph overlap.

[0128] In other words, by clearly defining branches, filtering conditions, and connecting logic, it is possible to filter nodes already in the lower partition through conditional judgment, thus solving the problem of excessive computation caused by traditional full-graph downstream search; it ensures that all nodes searched downstream belong to the range not covered by the lower partition, making the boundary between the second subgraph and the first subgraph clear and non-overlapping; it fully captures downstream related nodes, providing a structural foundation for the subsequent merging and fusion of the second subgraph, and helping to improve the integrity of operator fusion and GPU resource utilization.

[0129] Step B408: Determine the node found in the downstream search that matches the second data arrangement type identifier, and obtain the downstream second node that is related to the upstream node;

[0130] In this embodiment, the second data arrangement type identifier is the data arrangement type marker of the upstream node, used to characterize the data storage method in memory, such as row-major or column-major order. A key prerequisite for operator fusion is consistent data arrangement between nodes, which reduces the additional overhead of data conversion. Specifically, from all nodes searched downstream in step B406, nodes with the same second data arrangement type identifier as the upstream node are selected and defined as "downstream second nodes." These nodes have direct or indirect data flow connections with the upstream node, are linked downstream through the third branch, and have compatible data arrangements. They are an important component in constructing the second subgraph, providing a structural foundation for operator fusion within the subsequent subgraph.

[0131] Specifically, optionally, the screening is based on the core criterion of "consistency with the second data arrangement type identifier", and the identification comparison rules are clearly defined, such as complete string matching, consistent feature values, etc., so as to ensure the accuracy of the screening results, avoid nodes with different data arrangements from being mixed into the second subgraph, eliminate the fusion obstacles in the subgraph from the source, and ensure the efficiency of subsequent fusion.

[0132] Optionally, the downstream second node is a node found during the downstream search, meaning a node that has a direct or indirect data flow dependency with the upstream node, such as the first downstream node D of upstream node U, and the downstream node E of D, etc. It cannot include unrelated nodes. For example, if nodes D and E are found downstream, and both have the same identifier as U, then D and E are both downstream second nodes; if node F is found that has no data flow relationship with U, then F is not a downstream second node. In other words, the logical relationship between the downstream second node and the upstream node is guaranteed, avoiding the inclusion of irrelevant nodes that would lead to redundancy in the subgraph structure.

[0133] It should be noted that the step of accurately selecting downstream related nodes with consistent data distribution with upstream nodes follows the search results from step B406 and provides a clear target for step B409, forming a closed loop of "search, filtering, and aggregation". Thus, by connecting the search and subgraph construction through filtering, the quality of nodes in the second subgraph is ensured, giving the subgraph the characteristics of "consistent data distribution and close correlation," thereby guaranteeing the effectiveness of operator fusion.

[0134] Step B409: Move the upstream second node and the downstream second node as second associated nodes to the preset upper partition containing the upstream node, and move the other nodes in the second branch and the third branch that have not been moved to the upper partition to the lower partition;

[0135] In this embodiment, the upper partition is a pre-defined logical partition used to centrally store upstream nodes and their associated nodes with consistent data arrangement, while the lower partition is the partition where the current node and its downstream nodes reside. The upstream second node determined in step B404 and the downstream second node determined in step B408 are designated as second associated nodes and moved uniformly to the upper partition, which already contains the upstream node, ensuring data arrangement compatibility within the upper partition. The remaining nodes in the second and third branches that are not included in the upper partition are moved to the lower partition, achieving isolation from the upper partition. Through this operation, the upper and lower partitions each form independent subgraphs with consistent data arrangement, laying the structural foundation for subsequent subgraph fusion and optimization.

[0136] Specifically, optionally, the "second associated node" is limited to "upstream second node and downstream second node," and both must satisfy the requirement of "being consistent with the second data arrangement type identifier of the upstream node." For example, if the second identifier of upstream node U is "row priority - R," then only upstream second node P and downstream second node D, identified as "row priority - R," will be moved to the upper partition, excluding node Q, identified as "column priority - C." This ensures that the data arrangement of nodes in the upper partition is completely consistent, avoiding subgraph fusion obstacles caused by the mixing of inconsistent nodes and guaranteeing the effectiveness of subsequent fusion.

[0137] Optionally, the upper partition is pre-defined to include upstream nodes, and the move operation only needs to include the second associated node in this partition; the lower partition receives the remaining nodes, and the nodes in the two types of partitions do not overlap. For example, the upstream node Q that was not selected in the second branch and the downstream node E that was not selected in the third branch are both moved to the lower partition, separating them from U, P, and D in the upper partition. It can be seen that by clearly defining the partition boundaries, the independence of the upper and lower partitions is ensured, and the computational logic confusion caused by subgraph intersections is avoided.

[0138] Step B410: Connect at least one second associated node and the upstream node in the upper partition according to the preset computation graph topology dependencies to obtain the second subgraph. Step B410 includes: Step B4101: Connect at least one first associated node, the current node, and other nodes in the lower partition according to the preset computation graph topology dependencies to obtain the first subgraph.

[0139] Optionally, the lower partition includes the current node to be split, the first associated node selected by downstream unidirectional search, and other nodes. The preset computation graph topology dependency relationship is the inherent data flow order between nodes, such as the output of the current node serving as the input of the first associated node, and the output of the first associated node serving as the input of other nodes. This direction must be strictly preserved when connecting, without changing the original data transmission logic.

[0140] Optionally, "at least one first associated node" must cover key nodes within the lower partition that are directly dependent on the current node. All associated nodes should be connected first to preserve the complete downstream link and avoid logical breaks in the subgraph due to incomplete connections. Other nodes that have data flow dependencies on the current node and the first associated node and are arranged in the same way should also be included in the connection according to the original topology to ensure that the subgraph completely reflects the local computing link.

[0141] Optionally, step B4101 follows the step of "moving nodes to the lower partition" and forms an organic whole from isolated nodes through topological dependencies, so that the first subgraph has the characteristics of "consistent data arrangement and close association", providing a reliable structural foundation for subsequent iterative detection and operator fusion, and avoiding the subgraph from failing to reflect the real computation logic due to connection errors.

[0142] In this embodiment, the upper partition has already incorporated upstream nodes and second associated nodes through step B409. Both the upstream and downstream second nodes have the same data arrangement type as the upstream nodes. The preset computation graph topology dependency refers to the inherent data flow order between nodes, such as the upstream and downstream data transmission relationship between upstream nodes and associated nodes, with the arrow direction reflecting the data flow direction. Specifically, based on the topology dependency of the original computation graph, the upstream nodes in the upper partition are connected to at least one second associated node according to the original data flow direction, forming a second subgraph containing nodes and connection relationships. This process ensures that the second subgraph can fully reflect the local computation links of the upstream nodes and their associated nodes with the same data arrangement, providing a structural foundation for subsequent subgraph fusion and optimization.

[0143] Specifically, optionally, the connections in step B410 must be based on the preset computation graph topology dependencies, that is, maintaining the original data flow direction between nodes, such as the output of the upstream second node P → the input of the upstream node U, and the output of the upstream node U → the input of the downstream second node D, without arbitrarily changing the connection order or direction. For example, if P→U→D is a given topology in the original computation graph, then the upstream and downstream relationships between P and U, and U and D are preserved during connection, ensuring that the subgraph structure is consistent with the local logic of the original computation graph. Therefore, it ensures that the second subgraph can accurately map the local data processing flow of the original computation graph, avoiding the subgraph failing to reflect the real computation logic due to connection errors, and providing a reliable object for subsequent fusion operations.

[0144] Optionally, "at least one second associated node" needs to cover key nodes in the upper partition that are directly dependent on the upstream node. All second associated nodes should be connected first to preserve the complete downstream link. For example, upstream node U and upstream second node P and downstream second node D should be connected to ensure that the second subgraph contains a complete associated link. This avoids the subgraph logic being broken due to incomplete connections, which would prevent the subgraph from being unable to support the "full link optimization" requirement of subsequent operator fusion.

[0145] Optionally, step B410 follows step B409, forming a second subgraph through connections, providing a clear object for subsequent "subgraph fusion" or "iterative optimization". For example, only after connecting P, U, and D through topological dependencies can "P→U→D" be used as the second subgraph to check whether the fusion conditions are met within the subgraph. It is evident that logical associations achieve the transformation from a "set of nodes" to a "valid subgraph," ensuring a coherent and closed-loop subgraph splitting process and preventing subsequent steps from failing due to missing node connections.

[0146] As shown in Figure 3, Figure 3 is a schematic diagram of the upstream and downstream bidirectional search involved in the embodiment of this application. In the figure, the nodes are arranged sequentially from upstream to downstream according to the data flow, including nodes such as Q, P, U, D, E, F, and X. Specifically, Q is the source node, with data arranged in row-first-R, and serves as the endpoint of the upstream search of the second branch; P is the first upstream node of U, with data arranged in row-first-R, and serves as the core node of the second branch; U is the starting point of the bidirectional search, with data arranged in row-first-R, and serves as the core node that triggers the subgraph splitting; D is the current node to be split, with data arranged in column-first-C, and serves as the starting point of the first branch; E is the first downstream node of D, with data arranged in row-first-R, and is not included in the first subgraph because its data arrangement is inconsistent with that of D; F is the first associated node of D, with data arranged in column-first-C, consistent with the data arrangement of D, and is included in the first subgraph; X is the last downstream node of D, with data arranged in row-first-R, and is not included in the first subgraph because its data arrangement is inconsistent with that of D. Solid arrows correspond to the original data flow direction of the computation graph (such as Q→P, P→U, U→D, etc.), representing the static topological dependency of data output from the starting node and input to the ending node; dashed arrows correspond to the traversal direction of DFS search. Specifically, when searching upstream (such as the dashed line in the U→P→Q direction), the solid data flow is reversed, marking the trajectory of "traversing from upstream node U to its first upstream P and source Q"; when searching downstream (such as the dashed line in the D→E→F→X direction), the solid data flow is followed, marking the trajectory of "traversing from the current node D to its first downstream E and associated node F".

[0147] From the perspective of branching and search processes, the first branch starts with D and covers the downstream link D→E→F→X. A one-way downstream search is performed from D, selecting only F nodes with the same data arrangement as D. These two nodes are moved to the lower partition and connected according to the original data flow (D→F), forming the first subgraph. The second branch starts with upstream node U and covers the upstream link U←P. Since U is not the source node of the second branch and P does not belong to the lower partition, the upstream search condition is met. The search proceeds upstream from U to P, then to the source node Q, and stops there, selecting P and Q nodes with the same data arrangement as U as the second upstream nodes. The third branch also starts with upstream node U and covers the downstream link U→D. Since D belongs to the lower partition, the downstream search condition is not met, and the downstream search stops.

[0148] As shown in Figure 4, which is a schematic diagram of the upper and lower partitions involved in the embodiment of this application, specifically, since the second branch containing U and its first upstream node P is determined, U is not the source of the second branch (P has a higher upstream node Q), and P does not belong to the lower partition (the lower partition is the area where the current node D and downstream nodes are located), satisfying the preset upstream search conditions, the search proceeds upstream from U in reverse data flow until the source node Q of the second branch is found. P and Q, which are found to be consistent with the data arrangement type identifier (row priority - R) of U, are selected as the upstream second nodes and moved into the preset upper partition together with U. Therefore, the upper partition contains the source node Q, the first upstream node P of the upstream node, and the upstream node U of the starting point of the second and third branches.

[0149] By identifying the first branch containing D and its first downstream node E, a one-way search is performed from D downstream along the data flow. F, which matches the data arrangement type identifier (column priority - C) of D, is selected as the first associated node and moved into the preset lower partition along with D. Although E and X have a data arrangement of row priority - R, which does not match D, they are naturally assigned to the lower partition because they belong to the downstream area of ​​D. As boundary nodes in the lower partition, the lower partition contains the current node D at the starting point of the first branch, the first downstream node E of the current node, the first associated node F of the current node, and the last downstream node X of the first branch.

[0150] As shown in Figure 5, which is a schematic diagram of the construction of the first and second subgraphs involved in the embodiment of this application, based on the aforementioned content of Figure 4, in the lower partition, the current node D and the first associated node F, which is selected by the downstream unidirectional search and has the same data arrangement type identifier (column priority - C) as D, are connected according to the preset computation graph topology dependency relationship (i.e., the data flow order from D output to F input in the original computation graph), forming the first subgraph. In the upper partition, the upstream node U and the upstream second nodes P and Q, which are selected by the upstream search and have the same data arrangement type identifier (row priority - R) as U, are connected according to the preset computation graph topology dependency relationship (i.e., the data flow order from Q output to P input and from P output to U input in the original computation graph), forming the second subgraph. By retaining the topology dependency relationship of the original computation graph to connect the nodes in the partition, the upper and lower partitions are transformed into the first and second subgraphs with consistent data arrangement and complete association links, respectively, providing an independent and adaptable structural foundation for subsequent operator fusion.

[0151] In this embodiment, a second branch containing an upstream node and a first upstream node in the computation graph is determined, and it is determined whether the second branch satisfies a preset upstream search condition. The preset upstream search condition includes that the upstream node is not the source node of the second branch and that the first upstream node of the upstream node does not belong to a lower partition. If satisfied, an upstream search is performed in the second branch starting from the upstream node. When the upstream node found in the upstream search is the source node of the second branch, the upstream search stops. Nodes found in the upstream search that match the second data arrangement type identifier are determined, resulting in an upstream second node associated with the upstream node. A third branch containing an upstream node and a first downstream node in the computation graph is determined, and it is determined whether the third branch satisfies a preset downstream search condition. The preset downstream search condition includes... The first downstream node of the upstream node does not belong to the lower partition; if this condition is met, a search is performed in the third branch starting from the upstream node in the downstream direction. When the downstream node found in the downstream search is a node of the lower partition, the search in the downstream direction is stopped; the node found in the downstream search that matches the second data arrangement type identifier is determined, and the downstream second node that is associated with the upstream node is obtained; the upstream second node and the downstream second node are moved as second associated nodes to the preset upper partition containing the upstream node, and other nodes in the second and third branches that have not been moved to the upper partition are moved to the lower partition; at least one second associated node in the upper partition and the upstream node are connected according to the preset computation graph topology dependency relationship to obtain the second subgraph, thereby realizing the construction of a second subgraph with consistent data arrangement and clear boundaries, reducing computational complexity, and improving operator fusion efficiency and GPU resource utilization.

[0152] Furthermore, based on the above, in some feasible embodiments, the subgraph merging process includes the merging of nodes in the subgraph. The above step S60: merge all second subgraphs belonging to the same graph attribute to obtain a third subgraph, and also includes the following implementation steps B601 to B602.

[0153] Step B601: For each second subgraph, use the data arrangement type of the upstream nodes in the second subgraph as the graph attribute of the second subgraph;

[0154] In this embodiment, through bidirectional upstream and downstream search and filtering, it is ensured that the data layout type of all nodes in the second subgraph is completely consistent with that of the upstream nodes. Therefore, the data layout type of the upstream nodes can serve as a feature identifier for the entire subgraph, accurately reflecting the core attributes of the subgraph and ensuring that subsequent merging only occurs between subgraphs with consistent data layouts.

[0155] Optionally, by extracting the arrangement type of upstream nodes as graph attributes, the structural features of the subgraph are transformed into identifiable attribute information, realizing the transformation from "subgraph entity" to "attribute label" and providing standardized input for subsequent processing. Among them, graph attributes include, but are not limited to, row priority and column priority.

[0156] Step B602: Merge nodes in all second subgraphs belonging to the same graph attribute to obtain a third subgraph;

[0157] In this embodiment, specifically, the "same graph attribute" in step B602 needs to be defined as a common constraint that allows subgraphs to be merged, including but not limited to consistent data arrangement (such as memory storage format, tensor dimension matching), acyclic topology (no circular dependencies after merging), and compatible operator operation types (such as convolution or fully connected operators), ensuring that when different second subgraphs are merged, the data flow between nodes is conflict-free and the structure is reasonable. For example, if the data arrangement of two second subgraphs is NHWC and NCHW respectively, they cannot be merged due to attribute incompatibility, avoiding memory access errors or computational logic disorder after merging.

[0158] Optionally, "node fusion" is not simply merging nodes; it must maintain the topological dependencies of the original nodes within the second subgraph, i.e., preserve the upstream and downstream connection directions (such as the connection X→Y in second subgraph A and the connection Y→Z in second subgraph B, maintaining the X→Y→Z data flow after merging), ensuring that the structure of the third subgraph is consistent with the local logic of the original computation graph. Arbitrarily changing the connection order may lead to a break in data dependencies, causing subsequent operator fusion to fail due to logical errors.

[0159] Optionally, there may be multiple second subgraphs belonging to the same graph attribute. Priority should be given to merging all second subgraphs that satisfy the attribute constraints to maximize the size of the third subgraph. For example, when upstream nodes generate three second subgraphs, all of which satisfy the attributes of "NHWC data layout and acyclic topology," they should all be merged into a single third subgraph, rather than just partially merged. This provides a larger optimization unit for operator fusion, improving computational efficiency after fusion (e.g., reducing the number of operator calls and lowering data transport overhead).

[0160] Optionally, this step follows step S50 (severing upstream and downstream connections and splitting the computation graph into independent subgraphs). By merging second subgraphs with the same attribute, the scattered "intermediate subgraphs" are reconstructed into "fusionable units." If this step is skipped, multiple isolated second subgraphs will be too small to allow operator fusion to be performed within a single, small scope, preventing cross-subgraph batch optimization and weakening the overall performance improvement of the computation graph. Therefore, node fusion connects the "subgraph splitting" and "operator fusion" stages by "aggregating compatible subgraphs," forming a closed-loop logic of "split → filter → aggregate → optimize," ensuring the effectiveness of subsequent fusion operations.

[0161] In this embodiment, for each second subgraph, the data arrangement type of the current node to be split in the second subgraph is taken as the graph attribute of the second subgraph; nodes belonging to the same graph attribute in all second subgraphs are merged to obtain the third subgraph. Thus, by combining technical logic and specification requirements, the efficiency and feasibility of the operator fusion algorithm are effectively supported.

[0162] Furthermore, based on the content of any of the above embodiments, in some feasible embodiments, step B602: performing node fusion on nodes in all second subgraphs belonging to the same graph attribute, further includes the following implementation steps B6021 to B6023.

[0163] Step B6021: Perform directional analysis on any second subgraph belonging to the same graph attribute to determine whether there is a closed loop path in the second subgraph;

[0164] Step B6022: If a closed-loop path exists, stop merging nodes in all second subgraphs belonging to the same graph attribute;

[0165] Step B6023: If none of the second subgraphs belonging to the same graph attribute have a closed loop path, then perform node fusion on the nodes in all the second subgraphs belonging to the same graph attribute.

[0166] Specifically, the "directedness analysis" in step B6021 requires, for each second subgraph under the same graph attribute, to traverse the directed connections of nodes within the subgraph using algorithms such as depth-first search (DFS) or topological sorting, and to detect whether there is a closed-loop path that "starts from a certain node, goes through several edges, and returns to itself".

[0167] Optionally, the "stop fusion" logic in step B6022 is based on the characteristic that "a single closed loop can pollute the entire fusion result": if any second subgraph under the same graph attribute has a closed loop, even if other subgraphs are acyclic, the merged third subgraph will still inherit that closed loop. For example, if second subgraph A contains a loop and B does not, after fusion, the loop of A will be embedded in the third subgraph, causing the computation graph to fail to terminate. Therefore, as soon as a single closed loop is detected, fusion is stopped immediately to avoid generating illegal subgraphs and reduce subsequent debugging costs.

[0168] Optionally, the "fusion prerequisite" in step B6023 requires that all second subgraphs with the same graph attribute be acyclic, ensuring the topology of the third subgraph is valid while maintaining the compatibility logic of "graph attribute filtering". For example, if all three second subgraphs satisfy "NHWC data arrangement, convolution operator, and acyclic topology", the merged third subgraph can safely perform operator fusion, avoiding fusion failure due to topology issues.

[0169] From a process flow perspective, B6021-6023 follows the "graph attribute grouping" step, supplementing it with a secondary topology validity check: even if the "graph attributes" implicitly contain the constraint of "topological acyclicity," the construction process of the second subgraph may unexpectedly introduce cycles due to the complex connections between associated nodes. Therefore, by using loop detection, the "potential topology risks in the subgraph construction stage" are addressed, ensuring the dual constraints of "graph attribute screening and loop verification," providing a set of structurally valid and attribute-compatible subgraphs for node fusion, ultimately guaranteeing the stability and performance improvement of operator fusion.

[0170] As shown in Figure 6, Figure 6 is a schematic diagram of multiple second subgraphs involved in the embodiment of this application. As shown in Figure 7, Figure 7 is a schematic diagram of the merged third subgraph involved in the embodiment of this application. In Figure 5, there are two second subgraphs: the upper second subgraph consists of the source node Q, the first upstream node P of the upstream node, and the upstream node U; the lower second subgraph consists of node K and node J (the data arrangement is row-major -R). According to step S601, the data arrangement type of the upstream nodes (U, K) in each second subgraph is taken as the graph attribute. Since the graph attributes are consistent, the two belong to the same graph attribute of the second subgraph. Step S6021 is executed to perform directional analysis on the two second subgraphs, and it is found that the link Q→P→U in the upper subgraph and the link K→J in the lower subgraph have no closed loop path. After satisfying the condition of step S6023, the nodes of the second subgraphs with the same graph attribute are merged. During fusion, based on the topological dependencies of the original computation graph, an implicit direct data flow relationship is found between upstream node U and node K. Since the branch search boundary was not connected during the previous splitting stage, the link U→K is connected to form the third subgraph Q→P→U→K→J as shown in Figure 7. The original node data flow direction is preserved, and the node fusion of the second subgraph with the same attribute is realized, providing a more complete structural unit for subsequent operator fusion.

[0171] In this embodiment, directional analysis is performed on any second subgraph belonging to the same graph attribute to determine whether a closed-loop path exists in the second subgraph. If a closed-loop path exists, node fusion is stopped for all nodes in all second subgraphs belonging to the same graph attribute. If no closed-loop path exists in any second subgraph belonging to the same graph attribute, node fusion is performed for all nodes in all second subgraphs belonging to the same graph attribute. This achieves a combination of loop detection to ensure the legality of the subgraph structure, same-attribute filtering to ensure data compatibility, and steps forming a coherent logical link, effectively supporting the efficiency and feasibility of the operator fusion algorithm.

[0172] Furthermore, based on the content of any of the above embodiments, in some feasible embodiments, the above operator fusion method further includes the following implementation steps B70 to B80.

[0173] Step B70: Count the total number of subgraphs in real time and determine whether the total number of subgraphs is greater than the preset subgraph number threshold. The total number of subgraphs includes the number of subgraphs in the first and second subgraphs and the number of subgraphs in the third subgraph after subgraph merging.

[0174] In this embodiment, one of the core objectives of operator fusion is to reduce the complexity of the computation graph. If there are too many subgraphs, such as the first and second subgraphs not being effectively merged after splitting, or the third subgraph still being out of control in size after merging, it will increase the burden of computational resource scheduling and reduce efficiency. Specifically, the total number of all current effective subgraphs is counted in real time, and by comparing it with a preset threshold, it is determined whether the size of the subgraph is within a reasonable range, providing a basis for decision-making on whether to continue splitting, merging, or terminating the process.

[0175] Optionally, the total number of subgraphs is limited to the sum of "first subgraph, second subgraph, and third subgraph," ensuring that no valid subgraphs generated in previous steps are omitted. For example, if there are currently 3 first subgraphs, 2 second subgraphs, and 1 third subgraph, the total number is counted as 6. This ensures that all subgraph units affecting computational complexity are covered, avoiding inaccurate threshold judgments due to ambiguity in the statistical range, and ensuring comprehensive and effective control over the size of the subgraphs.

[0176] Optionally, the preset subgraph number threshold is a reasonable upper limit set based on computing resources and model requirements, used to trigger the adjustment of subgraph size. For example, if the threshold is set to 10, when the total number reaches 11, it is judged to be out of range, and subgraph merging optimization needs to be initiated. Therefore, the threshold mechanism prevents the number of subgraphs from growing indefinitely, avoiding the efficiency decline caused by a surge in computing power.

[0177] Step B80: If the total number of subgraphs exceeds the preset subgraph number threshold, then perform the preset pruning operation based on the computation graph to stop generating new subgraphs.

[0178] In this embodiment, operator fusion needs to improve efficiency while reducing the complexity of the computation graph. If the first and second subgraphs are over-generated, the total number of subgraphs will surge, increasing the burden of computational resource scheduling and ultimately reducing overall performance. The new subgraphs include both the first and second subgraphs. Specifically, when step B70 determines that the total number of subgraphs exceeds a preset threshold, the generation of new first and second subgraphs is terminated through a "preset pruning operation." These two types of subgraphs are the basic units for subsequent fusion, but their unrestricted generation will directly lead to uncontrolled scale. The pruning operation is performed based on the topology of the computation graph, ensuring that the integrity of existing subgraphs is not destroyed while stopping the generation of new subgraphs, thereby controlling the subgraph size within a reasonable range.

[0179] Optionally, the pruning operation only targets the "new subgraph" and does not affect the already generated first, second, and third subgraphs. For example, if four first subgraphs, two second subgraphs, and one third subgraph have already been generated, exceeding the threshold, the process of building new first and second subgraphs will not be initiated after pruning. However, existing subgraphs can continue to participate in subsequent operations such as fusion. This avoids disrupting the processing chain of existing subgraphs due to pruning, ensuring that scale control only affects the incremental part, thus controlling the total amount while preserving the results of previous processing.

[0180] Optionally, the pruning operation is performed "based on the computation graph," meaning that the boundary for stopping the generation of new subgraphs is determined based on the node topological dependencies in the computation graph. For example, subgraph construction is no longer initiated for unsplit nodes on non-core links, rather than terminating randomly. For instance, priority is given to preserving the node splitting permissions for core paths in the computation graph, stopping the generation of new subgraphs for edge path nodes, ensuring that core computational logic is not affected. This guarantees that the pruning operation does not disrupt the core functionality of the computation graph, maintaining the effectiveness of the algorithm while controlling its scale.

[0181] It should be noted that step B80 uses the threshold judgment result of performing pruning operation when the number of subgraphs exceeds the limit to form a response mechanism of "exceeding the limit, starting pruning, and stopping the generation of new subgraphs". Subsequently, fusion optimization can continue to be performed based on existing subgraphs to form a complete closed loop of "scale detection, over-limit control, and stock optimization".

[0182] In this embodiment, the total number of subgraphs is counted in real time to determine whether it exceeds a preset subgraph number threshold. The total number of subgraphs includes the number of subgraphs in the constructed first and second subgraphs, as well as the number of subgraphs in the merged third subgraph. If the total number of subgraphs exceeds the preset subgraph number threshold, a preset pruning operation is performed based on the computation graph to stop generating new subgraphs. The new subgraphs include the first and second subgraphs. This achieves a combination of dynamically controlling the subgraph size to avoid computational overload and forming a coherent control chain of steps. While ensuring the fusion effect, the computational complexity is controlled within a reasonable range, effectively supporting the efficiency and feasibility of the operator fusion algorithm.

[0183] In summary, this application provides an operator fusion method. By determining whether there are two directly connected first nodes with different data arrangement types in the computation graph, where the two first nodes are the current node to be split and its upstream node, the boundary of the subgraph to be split can be accurately located. If such a method exists, a downstream unidirectional search is performed in the computation graph starting from the current node to obtain the first associated node that is related to the current node. A first subgraph is then constructed based on the current node and the first associated node, ensuring that all nodes in the subgraph maintain the same data arrangement as the current node. A bidirectional upstream and downstream search is then performed in the computation graph starting from the upstream node to obtain the second associated node that is related to the upstream node. A second subgraph is then constructed based on the upstream node and the second node, ensuring a complete subgraph. The algorithm covers all associated nodes related to the upstream node to ensure the integrity of the upstream subgraph. It treats both the first and second subgraphs as new computation graphs and performs steps to determine if two directly connected first nodes with different data layout types exist in the new computation graph, continuing until no such nodes are found in the latest graph. This achieves full optimization of the computation graph. If multiple second subgraphs exist, all second subgraphs belonging to the same graph attribute are merged to obtain a third subgraph. This reduces the total number of subgraphs while ensuring fusion effectiveness, allowing operator fusion within the computation graph to be achieved through the third subgraph. This ensures independent subgraph fusion, reduces invalid computations and dependency conflicts, and improves overall efficiency.

[0184] Furthermore, this application also provides an operator fusion device. Please refer to Figure 8, which is a schematic diagram of the operator fusion device involved in the embodiment of this application. The operator fusion device provided in this application includes:

[0185] The node detection module H01 is used to determine whether there are two first nodes in the computation graph that have different data arrangement types and are directly connected. The two first nodes are the current node to be split and the upstream node of the current node.

[0186] The first subgraph construction module H02 is used to perform a downstream unidirectional search in the computation graph starting from the current node if it exists, to obtain the first associated node that has a relationship with the current node, and to construct the first subgraph based on the current node and the first associated node.

[0187] The second subgraph construction module H03 is used to perform a bidirectional search of upstream and downstream nodes in the computation graph, starting from the upstream node, to obtain the second associated node that is related to the upstream node, and to construct the second subgraph based on the upstream node and the second node.

[0188] The iterative processing module H04 is used to treat both the first subgraph and the second subgraph as new computation graphs, and to perform the step of determining whether there are two first nodes with different data arrangement types and direct connection in the computation graph according to the new computation graph, until it is detected that there are no two first nodes with different data arrangement types and direct connection in the latest computation graph.

[0189] The subgraph merging module H05 is used to merge all second subgraphs belonging to the same graph attribute if multiple second subgraphs exist, to obtain a third subgraph, so as to achieve operator fusion in the computation graph through the third subgraph.

[0190] The operator fusion apparatus provided in this application, employing the operator fusion method in the above embodiments, can solve the technical problem of low efficiency in operator fusion apparatuses. Compared with the prior art, the beneficial effects of the operator fusion apparatus provided in this application are the same as those of the operator fusion apparatus method provided in the above embodiments, and other technical features in this operator fusion apparatus are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0191] Furthermore, this application also provides an operator fusion device. Please refer to Figure 9, which is a schematic diagram of the structure of the operator fusion device involved in the embodiment of this application. Specifically, the device in this embodiment can be a device for locally running the operator fusion method.

[0192] This application provides an operator fusion device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the operator fusion device method in the first embodiment described above.

[0193] Referring now to Figure 9, a schematic diagram of a suitable operator fusion device for implementing embodiments of this application is shown. The operator fusion device in embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The operator fusion device shown in Figure 9 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of this application.

[0194] As shown in Figure 9, the operator fusion device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the operator fusion device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following devices can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the operator fusion device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows an operator fusion device with various devices, it should be understood that it is not required to implement or possess all of the shown devices. More or fewer devices may be implemented alternatively.

[0195] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0196] The operator fusion device provided in this application, employing the operator fusion method in the above embodiments, can solve the technical problem of low efficiency in operator fusion devices. Compared with the prior art, the beneficial effects of the operator fusion device provided in this application are the same as those of the operator fusion method provided in the above embodiments, and other technical features in this operator fusion device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0197] Furthermore, this application provides a computer-readable storage medium. This computer-readable storage medium stores an operator fusion program, which, when executed by a processor, implements the steps of the aforementioned operator fusion method.

[0198] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0199] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0200] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0201] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An operator fusion method, characterized in that, The operator fusion method is applied to a GPU computing environment. The method includes: determining whether there are two directly connected first nodes with different data arrangement types in the computation graph, wherein the two first nodes are the current node to be split and its upstream node; wherein the step of determining whether there are two directly connected first nodes with different data arrangement types in the computation graph includes: traversing each node in the computation graph based on a preset computation graph topology dependency; if the traversed node has an upstream node, determining the first data arrangement type identifier of the traversed node and determining the second data arrangement of the upstream node corresponding to the traversed node. Type identifier; detect whether the first data arrangement type identifier and the second data arrangement type identifier are consistent; if they are inconsistent, determine that the data arrangement type between the traversed node and the upstream node corresponding to the traversed node is different, and take the traversed node as the current node to be split, and take the upstream node corresponding to the traversed node as the upstream node of the current node; if they exist, perform a downstream unidirectional search in the computation graph starting from the current node to obtain the first associated node that has an association relationship with the current node, and construct a first subgraph based on the current node and the first associated node; in the computation graph, take the upstream node as the starting point A bidirectional search is performed upstream and downstream to obtain a second associated node that is related to the upstream node. A second subgraph is constructed based on the upstream node and the second associated node. Both the first and second subgraphs are used as new computation graphs, and the step of determining whether there are two directly connected first nodes with different data arrangement types in the computation graph is performed on each of the new computation graphs until two directly connected first nodes with different data arrangement types are detected in the latest computation graph. If multiple second subgraphs exist, all second subgraphs belonging to the same graph attribute are merged to obtain a third subgraph, which is then used for computation. The algorithm implements operator fusion in the computation graph; it continuously counts the total number of subgraphs and determines whether the total number of subgraphs exceeds a preset subgraph number threshold. The total number of subgraphs includes the number of subgraphs in the constructed first and second subgraphs, as well as the number of subgraphs in the merged third subgraph. If the total number of subgraphs exceeds the preset subgraph number threshold, a preset pruning operation is performed based on the computation graph to stop generating new subgraphs. The computation graph represents the structure of a deep learning model, where nodes represent operations and edges represent data flows. The data arrangement type refers to the way data is stored in memory; different storage formats can cause additional overhead during computation due to data conversion.

2. The operator fusion method as described in claim 1, characterized in that, The step of performing a downstream unidirectional search in the computation graph starting from the current node to obtain a first associated node that is related to the current node, and constructing a first subgraph based on the current node and the first associated node, includes: determining a first branch in the computation graph that contains the current node and a first downstream node of the current node; performing a downstream unidirectional search in the first branch starting from the current node until the last downstream node in the first branch is found; moving all downstream nodes found by the downstream unidirectional search as the first associated nodes to a preset lower partition containing the current node; and connecting at least one first associated node and the current node in the lower partition according to a preset computation graph topology dependency to obtain a first subgraph.

3. The operator fusion method as described in claim 2, characterized in that, The bidirectional search includes upstream and downstream searches. The step of performing a bidirectional search in the computation graph starting from the upstream node to obtain a second associated node related to the upstream node, and constructing a second subgraph based on the upstream node and the second associated node, includes: determining a second branch in the computation graph containing the upstream node and a first upstream node of the upstream node, and determining whether the second branch satisfies a preset upstream search condition; if satisfied, performing an upstream search in the second branch starting from the upstream node, stopping the upstream search when the upstream node found in the upstream search is the source node of the second branch; determining a node found in the upstream search that matches the second data arrangement type identifier, obtaining an upstream second node related to the upstream node; determining a third branch in the computation graph containing the upstream node and a first downstream node of the upstream node, and determining whether the third branch satisfies a preset downstream search condition; if satisfied, performing a bidirectional search in the third branch starting from the upstream node, stopping the upstream search when the upstream node found in the upstream search is the source node of the second branch; determining a node found in the upstream search that matches the second data arrangement type identifier, obtaining an upstream second node related to the upstream node; determining a third branch in the computation graph containing the upstream node and a first downstream node of the upstream node, and determining whether the third branch satisfies a preset downstream search condition; if satisfied, performing a bidirectional search in the third branch... The branch searches downstream from the upstream node. When the downstream node found in the downstream search is a node of the lower partition, the downstream search stops. The node found in the downstream search that matches the second data arrangement type identifier is determined, and a downstream second node with an association with the upstream node is obtained. The upstream second node and the downstream second node are moved as the second associated nodes to a preset upper partition containing the upstream node, and other nodes in the second branch and the third branch that were not moved to the upper partition are moved to the lower partition. At least one second associated node and the upstream node in the upper partition are connected according to a preset computation graph topology dependency to obtain a second subgraph. Connecting at least one first associated node and the current node in the lower partition according to the preset computation graph topology dependency to obtain a first subgraph includes: connecting at least one first associated node, the current node, and other nodes in the lower partition according to the preset computation graph topology dependency to obtain a first subgraph.

4. The operator fusion method as described in claim 3, characterized in that, The subgraph merging process includes node fusion of subgraphs. The step of merging all second subgraphs belonging to the same graph attribute to obtain a third subgraph includes: for each second subgraph, taking the data arrangement type of the upstream node in the second subgraph as the graph attribute of the second subgraph; and merging the nodes in all second subgraphs belonging to the same graph attribute to obtain a third subgraph.

5. The operator fusion method as described in claim 4, characterized in that, The node fusion process for nodes in all second subgraphs belonging to the same graph attribute includes: performing directional analysis on any second subgraph belonging to the same graph attribute to determine whether a closed loop path exists in the second subgraph; if a closed loop path exists, then stop fusion of nodes in all second subgraphs belonging to the same graph attribute; if no closed loop path exists in any of the second subgraphs belonging to the same graph attribute, then perform node fusion of nodes in all second subgraphs belonging to the same graph attribute.

6. An operator fusion device, characterized in that, The operator fusion method is applied to a GPU computing environment. The operator fusion device includes: a node detection module, used to determine whether there are two first nodes with different data arrangement types and directly connected in the computation graph, wherein the two first nodes are the current node to be split and the upstream node of the current node; the node detection module is specifically used to traverse each node in the computation graph based on a preset computation graph topology dependency relationship; if the traversed node has an upstream node, then determine the first data arrangement type identifier of the traversed node, and determine the second data arrangement type identifier of the upstream node corresponding to the traversed node; detect the first data arrangement type identifier and the second data arrangement type identifier. If the type identifiers are inconsistent, it is determined that the data arrangement types between the traversed node and the upstream node corresponding to the traversed node are different, and the traversed node is taken as the current node to be split, and the upstream node corresponding to the traversed node is taken as the upstream node of the current node; the first subgraph construction module is used to perform a downstream unidirectional search in the computation graph starting from the current node if it exists, to obtain a first associated node that has an association relationship with the current node, and to construct a first subgraph based on the current node and the first associated node; the second subgraph construction module is used to perform a bidirectional search in the computation graph starting from the upstream node, to obtain a first associated node that has an association relationship with the current node. The first subgraph is constructed based on the upstream node and the second associated node, which have a relationship with the upstream node. An iterative processing module treats both the first and second subgraphs as new computational graphs and performs the step of determining whether two directly connected first nodes with different data arrangement types exist in each new computational graph, until two directly connected first nodes with different data arrangement types are detected in the latest computational graph. A subgraph merging module merges all second subgraphs belonging to the same graph attribute if multiple second subgraphs exist, obtaining a third subgraph, which is used to achieve the desired result. The operator fusion device in the computation graph further includes: real-time statistics of the current total number of subgraphs, determining whether the total number of subgraphs is greater than a preset subgraph number threshold, wherein the total number of subgraphs includes the number of subgraphs in the constructed first subgraph, second subgraph, and the third subgraph after subgraph merging; if the total number of subgraphs is greater than the preset subgraph number threshold, then performing a preset pruning operation based on the computation graph to stop generating new subgraphs, wherein the computation graph refers to a graph representing the structure of a deep learning model, where nodes represent operations and edges represent data flows; the data arrangement type refers to the way data is stored in memory, and different storage formats will cause additional overhead due to data conversion during the computation process.

7. An operator fusion device, characterized in that, The operator fusion device includes a memory, a processor, and an operator fusion program stored in the memory and executable on the processor. When the processor executes the operator fusion program, it implements the steps of the operator fusion method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an operator fusion program, which, when executed by a processor, implements the steps of the operator fusion method as described in any one of claims 1 to 5.

Images