Operator fusion method, device and equipment and computer readable storage medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an operator fusion method, device and equipment and a computer readable storage medium, and the method comprises the steps: determining whether a to-be-split current node and an upstream node which are different in data arrangement type and are directly connected exist in a computational graph or not; if yes, performing downstream one-way search by taking the current node as a starting point to construct a first sub-graph containing a first associated node, and performing two-way search by taking the upstream node as a starting point to construct a second sub-graph containing a second associated node; taking the first sub-graph and the second sub-graph as a new calculation graph, and performing iterative detection until no direct connection node with different data arrangement exists; and if a plurality of first sub-graphs exist, combining the first sub-graphs with the same graph attributes to obtain a third sub-graph so as to realize operator fusion. The invention aims to improve the efficiency of operator fusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an operator fusion method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] In the field of deep learning, the computation graph optimization and operator fusion technology are the key directions to improve the model computing performance and fully utilize the GPU (Graphics Processing Unit) hardware resource efficiency. With the continuous increase in the complexity of deep learning models, the number of computing nodes contained in the computation graph increases significantly, and different nodes correspond to diversified operation types and data arrangement modes. How to realize efficient operator fusion through optimizing the computation graph structure has become an important research topic in the industry.

[0003] At present, the operator fusion technology generally uses algorithms such as depth-first search to traverse the computation graph. However, this method has significant limitations: in order to cover all fusion possibilities, the traversal process needs to handle a large number of computing paths, resulting in a huge amount of computation, low algorithm running efficiency, and difficulty in meeting the real-time and high-performance requirements in actual application scenarios, especially in the computation graph optimization of complex models.

[0004] The above problems make it difficult for those skilled in the art to efficiently achieve the optimization goal of operator fusion when facing high-complexity computation graphs, and there is an urgent need for a more optimal technical solution to break through the bottleneck of computation amount and efficiency.

[0005] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0006] The main purpose of the present application is to provide an operator fusion method, device, equipment and storage medium, which aims to improve the efficiency of operator fusion in complex scenarios.

[0007] To achieve the above purpose, the present application provides an operator fusion method, which comprises: determining whether there are two first nodes with different data arrangement types and directly connected in the computation graph, wherein the two first nodes are a current node to be split and an upstream node of the current node; If so, perform a downstream one-way search in the computation graph starting from the current node to obtain a first associated node having an association relationship with the current node, and construct a first subgraph according to the current node and the first associated node; performing an upstream and downstream bidirectional search in the computation graph starting from the upstream node to obtain a second associated node having an association relationship with the upstream node, and constructing a second subgraph according to the upstream node and the second node; The first subgraph and the second subgraph are both taken as new calculation graphs, and the step of determining whether there are two first nodes with different data arrangement types and directly connected in the calculation graph is executed according to the new calculation graphs, until it is detected that there are no two first nodes with different data arrangement types and directly connected in the latest calculation graph. If there are multiple second subgraphs, all the second subgraphs belonging to the same graph attribute are subjected to subgraph merging processing to obtain a third subgraph, so as to realize operator fusion in the calculation graph through the third subgraph.

[0008] In addition, to achieve the above object, the present application also provides an operator fusion device, which comprises: A node detection module is configured to determine whether there are two first nodes with different data arrangement types and directly connected in the calculation graph, wherein the two first nodes are a current node to be split and an upstream node of the current node. A first subgraph construction module is configured to, if there are, perform downstream one-way search in the calculation graph starting from the current node to obtain first associated nodes having an association relationship with the current node, and construct a first subgraph according to the current node and the first associated nodes. A second subgraph construction module is configured to perform upstream and downstream bidirectional search in the calculation graph starting from the upstream node to obtain second associated nodes having an association relationship with the upstream node, and construct a second subgraph according to the upstream node and the second nodes. An iteration processing module is configured to take the first subgraph and the second subgraph as new calculation graphs, and execute the step of determining whether there are two first nodes with different data arrangement types and directly connected in the calculation graph according to the new calculation graphs, until it is detected that there are no two first nodes with different data arrangement types and directly connected in the latest calculation graph. A subgraph merging module is configured to, if there are multiple second subgraphs, perform subgraph merging processing on all the second subgraphs belonging to the same graph attribute to obtain a third subgraph, so as to realize operator fusion in the calculation graph through the third subgraph.

[0009] The various functional modules of the operator fusion device of the present application realize the steps of the operator fusion method of the present application when running.

[0010] In addition, to achieve the above object, the present application also provides an operator fusion device, which comprises a memory, a processor, and an operator fusion program stored in the memory and executable on the processor, and the operator fusion program realizes the steps of the above operator fusion method when executed by the processor.

[0011] In addition, to achieve the above object, the application further provides a storage medium, which is a computer readable storage medium, and an operator fusion program is stored on the computer readable storage medium, and the operator fusion program realizes the steps of the above operator fusion method when executed by a processor.

[0012] The application provides an operator fusion method. The application can accurately locate the subgraph boundary to be split by determining whether two first nodes with different data arrangement types and directly connected exist in a computation graph, wherein the two first nodes are a current node to be split and an upstream node of the current node; if the two first nodes exist, performing downstream one-way search in the computation graph from the current node as a starting point to obtain a first associated node having an association relationship with the current node, and constructing a first subgraph according to the current node and the first associated node, so that all nodes in the subgraph can maintain the same data arrangement as the current node; performing upstream and downstream bidirectional search in the computation graph from the upstream node as a starting point to obtain a second associated node having an association relationship with the upstream node, and constructing a second subgraph according to the upstream node and the second node, so that all associated nodes related to the upstream node can be completely covered, and the integrity of the upstream subgraph is ensured; taking the first subgraph and the second subgraph as new computation graphs, and performing the step of determining whether two first nodes with different data arrangement types and directly connected exist in the computation graph according to the new computation graphs, until it is detected that no two first nodes with different data arrangement types and directly connected exist in the latest computation graph, so that full optimization of the computation graph can be realized; if multiple second subgraphs exist, performing subgraph merging processing on all second subgraphs belonging to the same graph attribute to obtain a third subgraph, so that the total number of subgraphs can be reduced under the premise of ensuring fusion effectiveness, so that operator fusion in the computation graph can be realized through the third subgraph, thereby ensuring independent fusion of the subgraph, reducing invalid calculation and dependency conflicts, and improving overall efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a flowchart of the first embodiment of the operator fusion method of the application; Figure 2 is a flowchart of the determination that no two first nodes exist in the operator fusion method of the application; Figure 3 is a schematic diagram of upstream and downstream bidirectional search involved in the embodiment of the application; Figure 4 is a schematic diagram of upstream partitioning and downstream partitioning involved in the embodiment of the application; Figure 5 is a schematic diagram of construction of the first subgraph and the second subgraph involved in the embodiment of the application; Figure 6 is a schematic diagram of multiple second subgraphs involved in the embodiment of the application; Figure 7is a merged third subgraph diagram involved in the embodiment scheme of the present application; Figure 8 is a structural diagram of an operator fusion device involved in the embodiment scheme of the present application; Figure 9 is a structural diagram of an operator fusion device involved in the embodiment scheme of the present application.

[0014] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0015] The embodiment of the present application provides an operator fusion method, referring to Figure 1 , as shown in Figure 1 is a flow diagram of the first embodiment of the operator fusion method of the present application.

[0016] The exemplary embodiments will be described in detail below, and their examples are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementation described in the following exemplary embodiments does not represent all the implementations consistent with the present application.

[0017] The existing operator fusion algorithm generally adopts depth-first search to traverse all possible computation graph structures, and then realizes node fusion through subgraph division. Although this full-quantity search strategy can cover all fusion possibilities, the calculation amount increases in geometric progression with the number of nodes, resulting in low algorithm efficiency and difficulty in meeting the real-time requirements of high industrial scenarios. At the same time, the existing scheme lacks a dynamic adjustment mechanism for subgraph division of the computation graph, and often has the problem that the number of nodes after fusion does not reach the optimal configuration; and in the division process, the acyclic nature of the graph structure is not fully checked, which may form a circularly dependent subgraph, violating the topological constraints of operator fusion. In addition, the existing technology lacks a dynamic limiting strategy for the total number of nodes of the computation graph, resulting in the search process falling into redundant paths, further exacerbating the waste of computing resources.

[0018] It is worth noting that the existing technology has obvious defects in data arrangement adaptability judgment - the data storage format (such as row-major or column-major) of different nodes in the computation graph directly affects the fusion feasibility, and the traditional algorithm either ignores the data arrangement difference and forcibly fuses, resulting in a dramatic increase in data conversion overhead, or misses potential fusion opportunities due to rough judgment logic. This neglect of data arrangement characteristics makes the actual benefit of operator fusion much lower than the theoretical expectation.

[0019] Therefore, in order to solve the technical problems of low search efficiency, unreasonable subgraph division, easy formation of ring structure and lack of node number control of the existing operator fusion algorithm, the present application provides an operator fusion method, device, equipment and storage medium.

[0020] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, a database system, or a device capable of realizing the above functions, such as an operator fusion device. The operator fusion device is taken as an example to describe the embodiment and the following embodiments.

[0021] The operator fusion method includes the following implementation steps S10 to S60.

[0022] Step S10: Determine whether there are two first nodes with different data arrangement types and direct connections in the computation graph, wherein the two first nodes are the current node to be split and the upstream node of the current node; if there are, execute step S20, if not, execute step S30, wherein step S20: Perform downstream one-way search in the computation graph starting from the current node to obtain first associated nodes associated with the current node, and construct a first subgraph according to the current node and the first associated nodes, step S30: Perform operator fusion processing on all nodes in the computation graph to obtain a fused computation graph.

[0023] Optionally, referring to Figure 2 , Figure 2 A flowchart for determining that there are no two first nodes in the operator fusion method is provided in step S10, by detecting the current node and the upstream node of the current node with different data arrangement types and direct connections in the computation graph, to provide a trigger condition for subsequent subgraph splitting. It should be noted that the computation graph refers to an operation model composed of nodes and directed edges, used to describe the processing flow of data, wherein the node represents an operation unit such as an operator, and the directed edge represents the data flow between nodes. It should be noted that the data arrangement type refers to the organization form of data in storage or operation, including but not limited to array dimension order (such as NHWC (N-Batch, H-Height, W-Width, C-Channels: Batch-Height-Width-Channel) format, NCHW (N-Batch, C-Channels, H-Height, W-Width: Batch-Channel-Height-Width) format, memory alignment mode (such as 32-byte alignment), data precision (such as FP32 (32-bit Floating Point: 32-bit floating point number), FP16 (16-bit Floating Point: 16-bit floating point number), etc. Direct connection refers to the existence of direct data dependency between two nodes without other intermediate nodes.

[0024] Further, operator fusion needs to be based on nodes with consistent data arrangement types, otherwise data conversion overhead will be introduced, so it is necessary to locate node pairs that do not meet this condition first; only detecting directly connected nodes can reduce invalid calculations, and indirectly connected nodes can have their data arrangement differences handled by intermediate nodes. Focusing on the current node and its upstream nodes can accurately locate the differences in the starting point of data flow and ensure that the internal data arrangement of the split subgraph is consistent.

[0025] Optionally, it is determined whether there are current nodes and upstream nodes with different data arrangement types and direct connections in the computation graph, and the computation graph refers to a graph representing the structure of a deep learning model, wherein nodes represent operations and edges represent data flow. The data arrangement type refers to the storage method of data in memory, including but not limited to row-major storage, column-major storage, channel-major storage, etc. Through diversified judgment strategies, the judgment requirements in different scenarios are comprehensively covered, which can efficiently and accurately identify the target node pair, provide a reliable basis for subsequent subgraph splitting, reduce invalid fusion in the computation graph, reduce the amount of calculation, optimize the utilization rate of hardware resources such as GPUs, and ultimately improve the inference performance of the deep learning model.

[0026] Optionally, in step S20, the computation graph is a graph representing the structure of a deep learning model, wherein nodes represent operations and edges represent data flow. The first associated node refers to a node associated with the current node downstream, and the first subgraph is composed of the current node and the first associated node, so as to reduce the search complexity through directional search, ordered filtering and structured construction, provide a clear local structure for subgraph splitting and fusion, and help improve the utilization rate of GPU resources and the inference performance of the model.

[0027] Optionally, it should be noted that the determination standard of the association relationship is to preferentially retain nodes that have direct data flow dependence on the current node, and if necessary, to extend to indirectly dependent nodes. The association relationship specifically includes but is not limited to retaining only direct downstream nodes, which is suitable for lightweight subgraph scenarios, and recursively traversing to no new downstream nodes, which is suitable for scenarios that require complete link analysis. By accurately filtering associated nodes, irrelevant nodes are avoided, ensuring the association and compactness of the subgraph.

[0028] Optionally, the construction rule of the first subgraph includes the current node, the first associated node and the edges between the nodes, and the subgraph structure needs to support subsequent fusion. It should be noted that the subgraph needs to completely retain the data flow relationship between the nodes to provide a basis for subsequent splitting, and the constructed subgraph can accurately reflect the local calculation logic, laying a foundation for the effectiveness of operator fusion.

[0029] Step S40: Bidirectional search is performed in the computation graph with the upstream node as the starting point to obtain second associated nodes that have an association relationship with the upstream node, and a second subgraph is constructed according to the upstream node and the second node.

[0030] In the embodiment, the comprehensive capture of the association relationship of the upstream node is realized through bidirectional search and accurate screening, and the second subgraph and the first subgraph form a complement, which jointly supports the integrity and accuracy of the subgraph splitting.

[0031] Specifically, optionally, a depth-first search (DFS) is adopted and the topological order is followed, starting from the upstream node, allowing upward and downward search. The upward search stops at the entry node of the computation graph, and the downward search stops when it overlaps with the first subgraph node, avoiding duplication and ensuring the formation of a closed loop with the downstream unidirectional search. The generation of a loop is avoided through topological order constraints, and the bidirectional search covers all link associated nodes, solving the problem of missing upstream dependencies in traditional unidirectional search and improving the completeness of the association relationship capture.

[0032] Optionally, based on data dependency relationship screening, only nodes that have direct or indirect data flow interaction with the upstream node are retained. Specifically, the directly associated nodes are retained first, which is suitable for lightweight splitting scenarios; or through recursive traversal to no new associated nodes, which is suitable for scenarios that require complete link analysis, to accurately screen effective associated nodes and avoid irrelevant nodes from being included, ensuring the compactness and relevance of the second subgraph and providing a reliable foundation for subsequent subgraph merging.

[0033] Optionally, the construction rules of the second subgraph include the upstream node, the second associated node, and the edges between the nodes, and the subgraph needs to satisfy the acyclic and data arrangement compatible, to prepare for subsequent merging. The node data arrangement type needs to be recorded synchronously during construction, to provide a basis for subsequent splitting verification with the first subgraph, so that the constructed second subgraph can not only fully reflect the association logic of the upstream node, but also meet the fusion constraints, providing structural protection for the effectiveness of subgraph splitting, and thus solving the problem of missing upstream dependencies in traditional unidirectional search, improving the comprehensiveness of associated node identification, reducing the inclusion of invalid nodes, and reducing the computational complexity of subgraph processing.

[0034] Step S50: Take both the first subgraph and the second subgraph as new computation graphs, and perform the step of determining whether there are two first nodes with different data arrangement types and direct connections in the new computation graph, until it is detected that there are no two first nodes with different data arrangement types and direct connections in the latest computation graph; In the embodiment, by taking the first subgraph and the second subgraph obtained by splitting as new computation graphs, the step of detecting first nodes with different data arrangement types and direct connections is repeatedly executed, forming an iterative process: if there are still such node pairs in the new computation graph, continue splitting; until all subgraphs have no such node pairs, terminate the iteration. The core purpose is to ultimately obtain subgraphs with consistent data arrangement and direct fusion through layer-by-layer splitting and detection, laying a foundation for subsequent fusion operations.

[0035] Optionally, the first subgraph and the second subgraph, as a new calculation graph, need to completely retain the core features of the original calculation graph, including nodes, edges, data arrangement types, etc. For example, the new calculation graph needs to clearly record the data arrangement identifier of each node to ensure that the detection step can directly reuse the original logic to ensure the feature consistency of the new calculation graph and the original calculation graph, making the iterative detection result reliable and avoiding misjudgment due to information loss.

[0036] Optionally, each iteration follows the logic of "detection, splitting, and new graph generation", and each input is the subgraph of the previous round of splitting, and the output is the detection result of the next round or a termination signal. For example, the first round of splitting obtains subgraphs A and B; detecting subgraph A finds that there are still inconsistent node pairs, and splitting obtains A1 and A2; and then A1 and A2 are used as new calculation graphs for repeated detection until there are no inconsistent node pairs. Through closed-loop iteration, inconsistent node pairs that do not meet the fusion condition are gradually eliminated to ensure that the final subgraph meets the fusion prerequisite of consistent data arrangement, solving the problem of low efficiency of nodes after fusion in traditional algorithms.

[0037] Optionally, the latest calculation graph does not have two first nodes with different data arrangement types and direct connections as a termination condition. For example, a counter or threshold can be set to assist in judgment to avoid infinite iteration, and a clear termination condition ensures that the algorithm can be completed within a limited number of steps, balancing optimization effect and computing cost, and improving engineering practicality.

[0038] That is, through recursive detection and splitting, subgraphs that meet the fusion condition are finally obtained, solving the problem of low efficiency of nodes after fusion in traditional algorithms due to data arrangement differences, and through recursive elimination of inconsistent node pairs, the data interaction in the subgraph is ensured to be efficient; the iteration process gradually simplifies the subgraph, reduces the calculation amount, and improves the GPU (Graphics Processing Unit: graphics processor) resource utilization, thereby supporting the performance improvement goal of the entire algorithm.

[0039] Step S60: If there are multiple second subgraphs, all second subgraphs belonging to the same graph attribute are subjected to subgraph merging processing to obtain third subgraphs, so as to realize operator fusion in the calculation graph through the third subgraphs.

[0040] Optionally, when multiple second subgraphs are obtained by splitting, the same graph attribute refers to the common characteristics of the subgraphs that can be merged (such as consistent data arrangement types, compatible node operations, and no loop after merging). By merging these subgraphs to form third subgraphs, the number of independent subgraphs in the calculation graph can be reduced, avoiding low fusion efficiency caused by fragmented subgraphs, and finally realizing centralized operator fusion through the third subgraphs to improve the calculation performance.

[0041] Optionally, the specific determination criteria of the same graph attribute include, but are not limited to, the same data arrangement type (such as both being stored in row-major order), the acyclic subgraph structure, and the node operation belonging to the same calculation category (such as both being convolution operations), so that the explicit attribute criteria ensure the accuracy of the merging object and avoid introducing data interaction conflicts after merging subgraphs with different attributes, thereby ensuring the effectiveness of the structure of the third subgraph.

[0042] Optionally, the merging process follows the principle of retaining effective features and eliminating redundant structures, which includes, but is not limited to, integrating the nodes and edges of the subgraphs, deleting duplicate connections, and verifying whether a loop is generated after merging. For example, when merging subgraphs A and B, if the data arrangement of the output node of A is consistent with that of the input node of B, a third subgraph is directly connected, and it is checked whether a loop exists, so that the merging process not only retains the core calculation logic of the subgraph, but also simplifies the overall structure and reduces the interaction overhead between nodes, thereby providing a more efficient basic unit for operator fusion.

[0043] Optionally, after merging, it is necessary to verify whether the third subgraph meets the fusion conditions to ensure that the third subgraph can be directly used for subsequent operator fusion. For example, by detecting the data arrangement difference, it is verified that the merged subgraph does not have a non-conforming node pair, so that the verification mechanism ensures the feasibility of the fusion of the third subgraph, avoids the need for secondary splitting after merging, and improves the coherence of the algorithm process.

[0044] That is, by merging the second subgraphs with the same attribute to generate a third subgraph, the structured integration of the computation graph is achieved, the subgraph fragmentation problem is solved, the number of independent subgraphs is reduced, and the complexity of operator fusion is reduced; the third subgraph has a unified graph attribute (such as data arrangement and acyclic), which improves the execution efficiency of the fused operator and provides reliable support for subsequent fusion steps.

[0045] In the embodiment, by determining whether there are two first nodes with different data arrangement types and direct connection in the calculation graph, wherein the two first nodes are the current node to be split and the upstream node of the current node, the subgraph boundary to be split can be accurately located; if there are, a downstream unidirectional search is performed in the calculation graph starting from the current node to obtain a first associated node having an association relationship with the current node, and a first subgraph is constructed according to the current node and the first associated node, which can ensure that all nodes in the subgraph maintain the same data arrangement as the current node; a bidirectional search is performed in the calculation graph starting from the upstream node to obtain a second associated node having an association relationship with the upstream node, and a second subgraph is constructed according to the upstream node and the second node, which can completely cover all associated nodes related to the upstream node and ensure the integrity of the upstream subgraph; the first subgraph and the second subgraph are both taken as new calculation graphs, and the step of determining whether there are two first nodes with different data arrangement types and direct connection in the calculation graph is performed according to the new calculation graphs, respectively, until it is detected that there are no two first nodes with different data arrangement types and direct connection in the latest calculation graph, which can realize full optimization of the calculation graph; if there are multiple second subgraphs, all second subgraphs belonging to the same graph attribute are subjected to subgraph merging processing to obtain a third subgraph, which can reduce the total number of subgraphs on the premise of ensuring fusion effectiveness, so as to realize operator fusion in the calculation graph through the third subgraph, thereby ensuring independent fusion of the subgraph, reducing invalid calculation and dependency conflicts, and improving overall efficiency.

[0046] Further, based on the above content, the second embodiment of the operator fusion method of the present application is proposed. In some feasible embodiments, the above step S10: determining whether there are two first nodes with different data arrangement types and direct connection in the calculation graph, further comprises the following implementation steps B101~S105.

[0047] Step B101: traversing each node in the calculation graph based on a preset calculation graph topology dependency relationship; In the embodiment, all nodes are traversed according to the preset topology dependency relationship, wherein the preset topology dependency relationship includes but is not limited to topological sorting, which ensures that the traversal order conforms to the data flow logic between nodes, and upstream nodes are traversed first, and then downstream nodes dependent on the output are traversed, thereby providing an ordered processing basis for subsequent steps of detecting node data arrangement type, judging whether to split, etc.

[0048] Optionally, the preset topological dependency relationship needs to be clear and executable, for example, the traversal order of nodes is predefined by an adjacency list or a topological sorting algorithm (such as Kahn algorithm), to ensure that each node is processed only after all its upstream dependency nodes are traversed, avoiding traversal confusion caused by ambiguous dependency relationship. The preset topological order ensures that the traversal logic is consistent with the data flow of the computation graph, avoiding downstream nodes being processed before upstream nodes, providing correct node association for subsequent detection of the data arrangement type of the node and upstream nodes, and reducing logical errors.

[0049] Optionally, it needs to be ensured that the traversal covers all nodes in the computation graph, and each node is traversed only once. For example, by marking the "traversed" state to avoid repeated processing and not to miss any node, it is ensured that the subsequent detection covers the entire graph. As can be seen, comprehensive traversal ensures that no node is missed, avoiding fusion obstacles caused by unprocessed nodes, such as hidden data arrangement inconsistency nodes that are not detected, ensuring that the algorithm optimizes the complete computation graph.

[0050] Optionally, the traversal result directly provides input for subsequent steps, such as determining the data arrangement type of the node and its upstream nodes, that is, only by traversing in topological order can the upstream nodes of the current node be accurately obtained, so that the dependency relationship is clear. For example, if the downstream nodes are traversed first, the upstream nodes have not been processed, and the data arrangement type of the upstream nodes cannot be obtained. Therefore, by traversing and subsequent steps based on topological dependency association, it is ensured that data arrangement detection, subgraph splitting, and other operations are based on correct node dependency relationship, improving the coherence and reliability of the overall logic of the algorithm.

[0051] Step B102: If the traversed node has upstream nodes, determining a first data arrangement type identifier of the traversed node, and determining a second data arrangement type identifier of the upstream node corresponding to the traversed node; In this embodiment, when the traversed node has upstream nodes, that is, there is a direct data flow dependency, by obtaining the first data arrangement type identifier of the current node and the second data arrangement type identifier of the upstream node, specific basis is provided for subsequent detection of whether they are consistent, which is a prerequisite for judging whether subgraph splitting is needed, and directly relates to the effectiveness of operator fusion.

[0052] Optionally, the generation rule of the identifier is clearly defined to ensure that it can uniquely and accurately reflect the data arrangement type. For example, a preset standardized identifier is used to avoid ambiguous expressions, or a hash value is dynamically generated based on data arrangement characteristics (such as dimension order and alignment method) as an identifier to ensure that a custom data arrangement type can also be accurately represented. The clear identification rule ensures the comparability of the first identifier and the second identifier, avoiding misjudgment caused by inconsistent representation, and providing reliable input for subsequent detection steps.

[0053] Optionally, the "corresponding upstream node" is defined as the "direct upstream node", i.e. the node directly connected to the current node in data flow, rather than the indirect upstream node. For example: if the input of node B comes directly from node A, then node A is the direct upstream node of node B, and only the identification of node A needs to be obtained; if the input of node B comes indirectly from node A through node C, then node A does not belong to the "corresponding upstream node" and its identification does not need to be obtained. Therefore, focusing on direct association nodes reduces the identification acquisition operation of irrelevant nodes, reduces the calculation amount, and improves the efficiency of the steps.

[0054] Optionally, by accepting the traversal of the nodes based on the topological dependency relationship, it is ensured that the traversal order conforms to the data flow logic and provides a data basis for subsequent detection of whether the identification is consistent, forming a closed loop of traversal, identification acquisition and difference detection. For example: only by traversing in topological order, from upstream to downstream, can the upstream nodes of the current node be accurately found and the identification be obtained when traversing the current node, otherwise the upstream nodes may not be traversed and the identification cannot be obtained. It can be seen that the logically coherent step design ensures the smoothness of the entire process, avoids step failure caused by chaotic order, and provides a reliable precondition for subsequent subgraph splitting or fusion.

[0055] Step B103: detecting whether the first data arrangement type identification and the second data arrangement type identification are consistent; If not consistent, step B104 is executed, and if consistent, step B105 is executed; Step B104: determining that the data arrangement types between the traversed node and the upstream node corresponding to the traversed node are different, taking the traversed node as the current node to be split, and taking the upstream node corresponding to the traversed node as the upstream node of the current node; step B105: keeping the association relationship between the traversed node and the upstream node corresponding to the traversed node unchanged, and continuing to traverse the next node in the computation graph until the traversal of all nodes in the computation graph is completed.

[0056] In this embodiment, the nodes are differentially processed by detecting whether the data arrangement type identification of the current node and the upstream node is consistent, wherein the data arrangement type identification is a mark representing the storage mode. If not consistent, the mark is a split object, which prepares for subsequent subgraph splitting; if consistent, the association relationship is preserved, which prepares for subsequent fusion, and at the same time the traversal is continued to cover all graph nodes to ensure that there is no omission.

[0057] Specifically, the detection of whether the first data arrangement type identification and the second data arrangement type identification are consistent needs to be based on a preset comparison rule, which includes but is not limited to string direct matching, hash value similarity calculation, etc. Therefore, the clear detection rule ensures accurate difference judgment and avoids misjudgment due to fuzzy comparison logic, providing a reliable basis for subsequent processing.

[0058] Optionally, when the identities are inconsistent, the current node and the upstream node are marked as a "to-be-split node pair". This step needs to specify the attributes of the "to-be-split", such as recording the node ID, the association relationship. If the marking information is missing, the subsequent subgraph splitting will have no clear object. Therefore, accurate positioning of the node pair to be split can avoid computational redundancy caused by indiscriminate processing, provide a clear target for subsequent subgraph splitting, and improve the algorithm's pertinence.

[0059] Optionally, when the identities are consistent, the association relationship is maintained and the traversal is continued. It should be noted that, on the one hand, the node pair with consistent data arrangement is a potential fusion object, and maintaining the association relationship can avoid repeated construction of connections in the subsequent; on the other hand, continuing the traversal ensures that all nodes in the computation graph are covered, and no node pair is missed. Therefore, the integrity of the process is guaranteed, which not only retains the effective association for fusion preparation, but also ensures that no node is missed through full graph traversal, improving the comprehensiveness of the algorithm.

[0060] In the embodiment, each node in the computation graph is traversed based on the preset computation graph topology dependency relationship; if the traversed node has an upstream node, the first data arrangement type identifier of the traversed node is determined, and the second data arrangement type identifier of the upstream node corresponding to the traversed node is determined; it is detected whether the first data arrangement type identifier and the second data arrangement type identifier are consistent; if not, it is determined that the data arrangement types between the traversed node and the upstream node corresponding to the traversed node are different, and the traversed node is taken as a current node to be split, and the upstream node corresponding to the traversed node is taken as an upstream node of the current node, thereby realizing efficient identification of operator fusion obstacles through ordered traversal, accurate detection and directional marking, providing a core basis for subsequent subgraph splitting and computation graph optimization, and further improving operator fusion efficiency and optimizing GPU resource utilization.

[0061] Further, based on the above content, in some feasible embodiments, the above step S20: performing a downstream one-way search in the computation graph starting from the current node to obtain a first associated node having an association relationship with the current node, and constructing a first subgraph according to the current node and the first associated node, can further include the following implementation steps B201 to B204.

[0062] Step B201: determining a first branch in the computation graph containing the current node and a first downstream node of the current node; In the embodiment, the current node is the preceding marked node to be split, the first downstream node refers to the direct downstream node of the current node, i.e., the node whose input is directly the output of the current node, and the first branch refers to the local data flow link formed by the current node, the first downstream node and the connection edge therebetween, reflecting the direct data interaction relationship therebetween, and realizing accurate extraction of the local branch formed by the current node and the direct downstream node thereof from the complex calculation graph, providing an explicit starting link for building the first subgraph for subsequent downstream unidirectional search, and avoiding the problems of too large range and calculation redundancy caused by full graph traversal.

[0063] Specifically, optionally, the first branch only includes the current node, the direct downstream node thereof and the direct connection edge therebetween, excluding indirect downstream nodes or nodes without direct association. For example, if the current node is A and the direct downstream node thereof is B, the first branch is "A→B", wherein the arrow represents the data flow direction between nodes in the calculation graph, i.e., the output data of node A is the input data of node B, embodying the direct data flow dependency relationship between the two nodes; if the downstream node of B is C, C does not belong to the branch. It can be seen that the explicit branch range ensures that the subsequent search focuses on the local link with direct association, avoids irrelevant nodes, reduces the calculation amount and improves the processing pertinence.

[0064] Optionally, it should be noted that the step of determining the first branch connects the preceding marked current node to be split, and provides a specific starting point for obtaining the first associated node and building the first subgraph for subsequent downstream unidirectional search. That is, through the branch positioning to connect the preceding and subsequent steps, the starting point of the subgraph construction is ensured to be explicit and the range is controllable, avoiding confusion of the search direction and providing a guarantee for the orderliness of subsequent subgraph splitting.

[0065] Step B202: performing downstream unidirectional search in the first branch with the current node as the starting point until the last downstream node in the first branch is searched; In the embodiment, all downstream nodes are traversed from the current node along the data flow direction of the first branch until the last downstream node at the end of the branch, and the purpose is to completely capture all downstream nodes in the branch that have direct or indirect data flow dependency with the current node, provide an explicit operation object for moving the node to the lower partition in the subsequent step, and ensure that no associated nodes in the branch are missed during subgraph splitting.

[0066] Specifically, optionally, the search is limited in the first branch and does not extend to other branches, such as other downstream branches of the current node. Thus, the target branch is focused on, irrelevant nodes are avoided, calculation redundancy is reduced, and the search pertinence is improved.

[0067] Optionally, the downstream unidirectional search is adopted, only proceeding in the data flow direction without backtracking upstream, conforming to the downward unidirectional search logic of the split subgraph algorithm and following the topological dependency of the computation graph, proceeding upstream first and downstream second, avoiding loops or repeated traversal caused by reverse search, thereby ensuring the search order, avoiding dead loops or repeated calculation, and improving the search efficiency.

[0068] Specifically, optionally, the last downstream node in the first branch is taken as the termination condition of the search, and the explicit termination condition avoids infinite search, balances the search integrity and the calculation cost, and enhances the engineering practicability.

[0069] Step B203: moving all the downstream nodes searched by the downstream unidirectional search to a preset lower partition containing the current node as the first associated node; In this embodiment, all the downstream nodes obtained by the downstream unidirectional search in step B202 are moved to the lower partition containing the current node, so that the current node and the first associated node form an independent local subgraph, thereby realizing the separation from the partition containing the upstream node and laying a structural foundation for subsequent subgraph fusion without data arrangement difference.

[0070] Optionally, the first associated node is limited to all the downstream nodes searched by the downstream unidirectional search, i.e., all the downstream nodes in the first branch traversed in step B202, which needs to be completely contained and not exceed the range. Therefore, the relevance and integrity of the lower partition nodes are ensured, and the incomplete subgraph structure caused by missing nodes is avoided, thereby affecting the effectiveness of subsequent fusion.

[0071] Optionally, the lower partition needs to be preset, such as being divided by logical markers or memory partitioning, and by default containing the current node, and the moving operation only needs to include the first associated node in the partition. This preset ensures that the partition boundary is clear and avoids confusion with the partition containing the upstream node. That is, the preset partition reduces the calculation overhead of dynamic division, and clearly defines the ownership of the current node and the first associated node, thereby providing a clear range definition for the independent processing of the subgraph, such as subsequent iterative detection.

[0072] Optionally, all the downstream nodes need to be completely moved to the lower partition, so as to ensure the data flow link between the current node and the first associated node to be completely retained in the partition, such as the connection relationship of A→B→C being unchanged in the lower partition. If a node is missed, the subgraph link will be broken, losing the significance of splitting. Therefore, the data flow integrity of the nodes in the lower partition is ensured, so that the partition forms a subgraph unit that can be independently processed, thereby providing a complete object for detecting whether there is a data arrangement difference in the subgraph.

[0073] That is, the current node is unified with the downstream associated node to form a lower partition, realizes separation from the upstream node, solves the problem of "low fusion efficiency caused by mixed data arrangement difference nodes"; the data flow link between nodes is completely retained to provide reliable objects for subsequent iteration optimization in the subgraph; through the preset partition and complete moving operation, the calculation redundancy of subgraph division is reduced, which helps to improve the efficiency of the overall operator fusion process.

[0074] Step B204: connecting at least one first associated node and the current node in the lower partition according to the preset calculation graph topology dependency relationship to obtain a first subgraph.

[0075] In the embodiment, the preset calculation graph topology dependency relationship refers to the inherent data flow order between nodes, such as the dependency relationship that the output of the upstream node is the input of the downstream node; the lower partition already contains the current node and the first associated node. Based on the topology dependency relationship of the original calculation graph, the current node and at least one first associated node in the lower partition are connected according to the original data flow logic to form a first subgraph containing nodes and connection relationships, so that the subgraph can completely reflect the local calculation link of the current node and its downstream associated nodes, and provide a structural basis for subsequent subgraph optimization and fusion.

[0076] Further, in an embodiment, the connection of at least one first associated node and the current node in the lower partition must follow the preset calculation graph topology dependency relationship, that is, the original data flow direction between nodes is maintained, such as the output of the current node A→the input of the first associated node B, the output of B→the input of the first associated node C, and the connection order or direction cannot be changed at will. For example: if A→B→C is the established topology in the original calculation graph, the upstream and downstream relationships of A and B and B and C need to be maintained when connecting, and connections such as A→C or B→A cannot be constructed. Thus, the structure of the first subgraph is consistent with the local logic of the original calculation graph, avoiding the situation that the subgraph cannot reflect the real calculation process due to connection errors, and ensuring the effectiveness of subsequent processing.

[0077] Further, in an embodiment, at least one first associated node needs to cover the key nodes in the lower partition that have direct or indirect dependency with the current node, and all first associated nodes are connected in priority to retain the complete downstream link, such as the current node A and the first associated node B, C all need to be connected, and no necessary connection relationship is omitted. That is, the first subgraph contains complete downstream associated links, avoiding the situation that the subgraph logic is broken due to incomplete connection, which cannot support the demand of subsequent operator fusion for full link optimization.

[0078] Further, in an embodiment, by connecting the nodes in the lower partition based on the topological dependency relationship, step B203 is followed by connecting to form a first subgraph, providing a clear object for subsequent iteration detection of the first subgraph as a new calculation graph. For example: only by connecting A, B and C through topological dependency, "A→B→C" can be taken as the first subgraph for detecting whether there is a data arrangement difference in the subgraph. It can be seen that the connection operation makes the isolated nodes in the lower partition form an organic whole, realizes the transformation from "node set to effective subgraph, and ensures the entire subgraph splitting process to be coherent and closed loop.

[0079] In the present embodiment, by forming a local subgraph reflecting the real data flow with the current node and the first associated node, the problem of logical confusion caused by the lack of connection after node movement is solved; following the original topological dependency, it is ensured that the first subgraph can accurately map the downstream link of the original calculation graph, providing a reliable structural basis for subsequent iteration detection; the complete subgraph structure enables the subsequent fusion operation to be optimized for the entire link, helping to reduce data interaction overhead and improve GPU resource utilization.

[0080] Further, based on the above, in some feasible embodiments, the upstream and downstream bidirectional search includes upstream direction search and downstream direction search, and the above step B40: performing upstream and downstream bidirectional search in the calculation graph starting from the upstream node to obtain a second associated node having an association relationship with the upstream node, and constructing a second subgraph according to the upstream node and the second node, further includes the following implementation steps B401-B410.

[0081] Step B401: determining a second branch containing the upstream node and a first upstream node of the upstream node in the calculation graph, and judging whether the second branch satisfies a preset upstream direction search condition; If it is satisfied, step B402 is executed, and if it is not satisfied, step B403 is executed; Step B402: performing upstream direction search in the second branch starting from the upstream node, and stopping the upstream direction search when the upstream node searched in the upstream direction is the source node of the second branch; Step B403: stopping the upstream direction search in the second branch starting from the upstream node; In the embodiment, the upstream node is a node that has a data arrangement difference with the current node, and needs to capture its complete associated link by searching upstream to construct a second subgraph, wherein the preset upstream searching condition includes that the upstream node is not a source node of the second branch and the first upstream node of the upstream node does not belong to the lower partition. Specifically, the local data flow link formed by the upstream node and the first upstream node of the upstream node is locked, such as "upstream node U←first upstream node P", the arrow indicates the data flow direction, and is used as the starting unit of the upstream search; whether the upstream search needs to be started is determined through the two conditions that the upstream node is not a source node and the first upstream node of the upstream node does not belong to the lower partition; if the conditions are met, step B402 is executed, otherwise step B403 is executed, forming a closed-loop logic of judgment and execution.

[0082] Specifically, optionally, the second branch limits the direct associated link of the "upstream node and its first upstream node", and does not include indirect upstream nodes such as the upstream node of the first upstream node. For example: the direct upstream of the upstream node U is P, then the second branch only refers to the "connection relationship between U and P", avoiding the inclusion of the upstream node Q of P in the branch range, preventing the search boundary from being out of control, thereby accurately defining the starting boundary of the upstream search, ensuring that the subsequent search starts from the directly associated node, and avoiding redundant calculation caused by ambiguous range.

[0083] Optionally, the two preset conditions limit the search start from different dimensions. Specifically, the upstream node is not a source node of the second branch: if the upstream node is a source node, i.e. there is no more upstream node, then the upstream search is meaningless, and this condition ensures the feasibility of the search. The first upstream node of the upstream node does not belong to the lower partition, i.e. the lower partition is the area where the current node and the downstream node are located. If the first upstream node belongs to this area, it means that it has been included in the first subgraph and does not need to be searched repeatedly. This condition ensures the independence of the search. The combination of the two forms a complete screening logic, avoiding invalid search caused by missing conditions. Therefore, through condition complementation, only the scenarios with searchable nodes and not processed are retained, reducing the waste of computing resources.

[0084] Optionally, the judgment results are directly connected to step B402 and step B403, and provide effective input for subsequent step B404. For example: if the second branch meets the conditions, the upstream search of step B402 can obtain the upstream node Q of P to supplement the associated nodes of the second subgraph; if it does not meet the conditions, the search of step B403 can avoid invalid traversal of the source node or the lower partition node. Therefore, through the logic loop, the upstream search is ensured to be "searched if it needs to be searched, and stopped if it needs to be stopped", which guarantees the completeness of the associated node capture and avoids exceeding the necessary range.

[0085] That is, the meaningless search scene is filtered through conditional judgment, the problem of excessive calculation caused by traditional upstream search on the whole graph is solved; it is ensured that the nodes for upstream search all belong to the range not covered by the lower partition, a pure node set is provided for the construction of the second subgraph, and overlap with the first subgraph is avoided; the upstream associated nodes are completely captured, a structural basis is provided for the merging and fusion of the second subgraph, and the integrity of operator fusion and the GPU resource utilization rate are improved.

[0086] Step B404: determining the nodes searched in the upstream direction search and consistent with the second data arrangement type identifier, to obtain the upstream second nodes having an association relationship with the upstream nodes; In this embodiment, the second data arrangement type identifier is a data arrangement type mark of the upstream node, such as representing the storage mode of data in the memory, and the key premise of operator fusion is that the data arrangement of nodes is consistent, which can reduce the time-consuming of data conversion. Specifically, from all the nodes searched in the upstream direction in step B402, the nodes consistent with the second data arrangement type identifier of the upstream node are screened out, and these nodes are defined as upstream second nodes, that is, they have a direct or indirect data flow association with the upstream nodes, and the data arrangement is compatible, which is the core component for constructing the second subgraph, and provides a qualified node set for subsequent subgraph fusion.

[0087] Optionally, the consistent second data arrangement type identifier is used as the screening standard, and the identification comparison rule is defined, such as string matching, consistent characteristic value, etc. For example, if the second identifier of the upstream node is “row-major-R”, only the upstream nodes searched with the same identifier “row-major-R” are included in the upstream second nodes, and the inconsistent nodes such as “column-major-C” are excluded, so as to ensure the accuracy of the screening result, avoid the mixing of nodes with different data arrangements into the second subgraph, and provide structural guarantee for the efficiency of subsequent fusion.

[0088] Optionally, the upstream second node is a node searched in the upstream direction, that is, an associated node having a data flow dependency with the upstream node, and cannot include irrelevant nodes. Therefore, the logical association between the upstream second node and the upstream node is ensured, and the inclusion of irrelevant nodes is avoided to cause subgraph structure redundancy.

[0089] Step B405: determining a third branch containing the upstream node and the first downstream node of the upstream node in the computational graph, and judging whether the third branch meets a preset downstream direction search condition; If yes, step B406 is performed, and if no, step B407 is performed; Step B406: performing a downstream direction search with the upstream node as a starting point in the third branch, and stopping the downstream direction search when a downstream node searched in the downstream direction is a node of the lower partition; Step B407: stop the search in the third branch in the upstream node as the starting point in the downstream direction; In this embodiment, the upstream node needs to expand its associated link by searching downstream to build a complete second subgraph, but needs to avoid duplication with nodes that have been divided into the lower partition (i.e. the region where the current node and the downstream node are located). Specifically, the local data flow link formed by the upstream node and its first downstream node is locked as the starting unit for the downstream search, and the preset downstream search condition includes that the first downstream node of the upstream node does not belong to the lower partition. By the preset condition that the first downstream node does not belong to the lower partition, it is judged whether the downstream search needs to be started. If the condition is met, step B406 is executed, otherwise step B407 is executed, ensuring that the search range is limited to the area not covered by the lower partition.

[0090] Optionally, the third branch is the direct associated link of the "upstream node and its first downstream node", and does not include indirect downstream nodes. Therefore, the starting boundary of the downstream search is accurately defined to ensure that the subsequent search starts from the directly associated node, avoiding redundant calculation caused by ambiguous range.

[0091] Optionally, the condition that the first downstream node of the upstream node does not belong to the lower partition is essentially to avoid repeated processing through regional division: the lower partition has been included in the current node and the downstream node through steps B203-B204, and if the first downstream node belongs to the region, it means that it has been included in the first subgraph, and there is no need to include the associated link of the upstream node. Therefore, by condition filtering, only the downstream nodes not covered by the lower partition are searched to reduce the waste of computing resources and ensure the independence of subgraph splitting, i.e. no overlapping nodes between the upper partition and the lower partition.

[0092] Optionally, the judgment result of whether the third branch meets the preset downstream search condition directly links steps B406 and B407, and provides effective input for subsequent step B408. For example: if the third branch meets the condition, the downstream search of step B406 can obtain the downstream node E of D to supplement the associated nodes of the second subgraph; if it does not meet the condition, the search is terminated in step B407 to avoid invalid traversal of the nodes in the lower partition. Therefore, through logical closed loop, the downstream search is "extended on demand", which not only guarantees the integrity of the associated nodes captured, but also controls the range to be non-lower partition to avoid subgraph overlap.

[0093] That is, by defining the branch, condition filtering and logical connection, the nodes in the lower partition are filtered through condition judgment, solving the problem of excessive calculation caused by traditional downstream search of the whole graph; ensuring that the nodes searched in the downstream direction all belong to the range not covered by the lower partition, making the boundary of the second subgraph and the first subgraph clear and non-overlapping; complete capture of downstream associated nodes provides a structural basis for subsequent merging and fusion of the second subgraph, helping to improve the integrity of operator fusion and GPU resource utilization.

[0094] Step B408: determining the nodes searched in the downstream direction search that are consistent with the second data arrangement type identifier, to obtain downstream second nodes that have an association relationship with the upstream node; In this embodiment, the second data arrangement type identifier is a data arrangement type mark of the upstream node, used to represent the storage mode of data in the memory, such as row priority, column priority, and the like, and the key prerequisite for operator fusion is that the data arrangement of nodes is consistent, which can reduce the additional overhead of data conversion. Specifically, from all the nodes searched in the downstream direction in step B406, the nodes that are consistent with the second data arrangement type identifier of the upstream node are screened out, and these nodes are defined as “downstream second nodes”. These nodes have a direct or indirect data flow association with the upstream node, pass through the downstream link of the third branch, and have compatible data arrangement, which are important components of the second subgraph and provide a structural basis for subsequent operator fusion in the subgraph.

[0095] Specifically, optionally, the comparison rule of the identifier is defined with “consistency with the second data arrangement type identifier” as the core standard, such as complete string matching, consistent characteristic value, and the like, so as to ensure the accuracy of the screening result, avoid mixing nodes with different data arrangements into the second subgraph, eliminate the fusion obstacles in the subgraph from the source, and guarantee the efficiency of subsequent fusion.

[0096] Optionally, the downstream second node is a “node searched in the downstream direction search”, that is, an associated node that has a direct or indirect data flow dependence with the upstream node, such as the first downstream node D of the upstream node U, the downstream node E of D, and the like, and cannot include irrelevant nodes. For example, if nodes D and E are searched downstream, and the identifiers of both are consistent with U, then D and E are both downstream second nodes; if a node F that has no data flow association with U is searched, then F does not belong to the downstream second node. That is, the logical association of the downstream second node with the upstream node is guaranteed, and irrelevant nodes are avoided from being included to cause subgraph structure redundancy.

[0097] It should be noted that the step of accurately screening the downstream associated nodes that are consistent with the data arrangement of the upstream node connects the search result of step B406 and provides a clear object for step B409, forming a closed loop of “searching, screening, and collecting”. As can be seen, the search is connected with the subgraph construction through screening, the quality of the nodes of the second subgraph is ensured, the subgraph has the characteristics of “consistent data arrangement and close association”, and the effectiveness of operator fusion is guaranteed.

[0098] Step B409: moving the upstream second nodes and the downstream second nodes as second associated nodes to a preset upper partition containing the upstream node, and moving other nodes in the second branch and the third branch that are not moved to the upper partition to a lower partition; In this embodiment, the upper partition is a preset logical partition for storing the upstream node and its associated nodes with consistent data arrangement, and the lower partition is a partition where the current node and its downstream nodes are located. The upstream second node determined in step B404 and the downstream second node determined in step B408 are taken as the second associated nodes, and are uniformly moved to the upper partition containing the upstream node to ensure that the nodes in the upper partition have compatible data arrangement. The remaining nodes in the second branch and the third branch that are not included in the upper partition are moved to the lower partition to achieve isolation from the upper partition. Through this operation, the upper partition and the lower partition form independent subgraphs with consistent data arrangement, laying a structural foundation for subsequent subgraph fusion and optimization.

[0099] Specifically, the "second associated node" is defined as the "upstream second node and the downstream second node", and both of them need to meet the "consistent with the second data arrangement type identifier of the upstream node". For example: the second identifier of the upstream node U is "row-major - R", only the upstream second node P and the downstream second node D with the identifier "row-major - R" are moved to the upper partition, excluding the node Q with the identifier "column-major - C". This ensures that the data arrangement of the nodes in the upper partition is completely consistent, avoids the fusion obstacle of the subgraph caused by the mixed inconsistent nodes, and guarantees the effectiveness of the subsequent fusion.

[0100] Optionally, the upper partition is preset to contain the upstream node, and the moving operation only needs to include the second associated node in this partition; the lower partition receives the remaining nodes, and there is no overlap between the nodes of the two types of partitions. For example: the upstream node Q not selected in the second branch and the downstream node E not selected in the third branch are both moved to the lower partition and separated from U, P and D in the upper partition. It can be seen that by clearly defining the partition boundary, the independence of the upper partition and the lower partition is guaranteed, and the calculation logic confusion caused by the intersection of subgraphs is avoided.

[0101] Step B410: connecting at least one second associated node in the upper partition and the upstream node according to the preset computational graph topology dependency relationship to obtain a second subgraph. Wherein, step B410 includes: step B4101: connecting at least one first associated node, the current node and other nodes in the lower partition according to the preset computational graph topology dependency relationship to obtain a first subgraph.

[0102] Optionally, the lower partition contains the current node to be split, the first associated node selected by the downstream unidirectional search, and other nodes. The preset computational graph topology dependency relationship is the inherent data flow order between nodes, such as the output of the current node as the input of the first associated node, the output of the first associated node as the input of other nodes, etc. When connecting, this direction needs to be strictly preserved, and the original data transmission logic should not be changed.

[0103] Optionally, the "at least one first associated node" needs to cover the key nodes in the lower partition that are directly dependent on the current node, and all associated nodes are connected in priority to retain the complete downstream link, avoiding the logical breakage of the subgraph due to incomplete connection. If other nodes have data flow dependencies with the current node and the first associated node and are arranged consistently, they also need to be included in the connection according to the original topological relationship to ensure that the subgraph fully reflects the local computing link.

[0104] Optionally, the step B4101 is connected to the step of "moving the node to the lower partition", and the isolated nodes are formed into an organic whole through topological dependence, so that the first subgraph has the characteristics of "consistent data arrangement and close association", providing a reliable structural basis for subsequent iteration detection and operator fusion, and avoiding the situation that the subgraph cannot reflect the real computing logic due to connection errors.

[0105] In this embodiment, the upper partition has been included in the upstream node and the second associated node through step B409, and the upstream second node and the downstream second node are consistent with the upstream node in data arrangement type; the preset computing graph topological dependence relationship refers to the inherent data flow order between nodes, such as the upstream and downstream data transmission relationship between the upstream node and the associated node, and the arrow direction reflects the data flow direction. Specifically, based on the topological dependence relationship of the original computing graph, the upstream node and the at least one second associated node in the upper partition are connected in the original data flow direction, forming a second subgraph containing nodes and connection relationships. This process ensures that the second subgraph can fully reflect the local computing link of the upstream node and its associated nodes with consistent data arrangement, providing a structural basis for subsequent subgraph fusion and optimization.

[0106] Specifically, optionally, the connection in step B410 needs to be based on the preset computing graph topological dependence relationship, that is, the original data flow direction between nodes is maintained, such as the output of the upstream second node P→the input of the upstream node U, the output of the upstream node U→the input of the downstream second node D, and the upstream and downstream relationship between P and U, U and D is not changed arbitrarily. For example: if P→U→D is the established topology in the original computing graph, the upstream and downstream relationship between P and U and U and D is retained during connection, ensuring that the subgraph structure is consistent with the local logic of the original computing graph. Therefore, it is ensured that the second subgraph can accurately map the local data processing flow of the original computing graph, avoiding the situation that the subgraph cannot reflect the real computing logic due to connection errors, and providing a reliable object for subsequent fusion operations.

[0107] Optionally, the "at least one second associated node" needs to cover the key nodes in the upper partition that are directly dependent on the upstream node, and all second associated nodes are connected in priority to retain the complete downstream link, such as the upstream node U and the upstream second node P, and the downstream second node D, which need to be connected to ensure that the second subgraph contains complete associated links, avoiding the logical breakage of the subgraph due to incomplete connection, which cannot support the demand for "full link optimization" of subsequent operator fusion.

[0108] Optionally, the step B410 receives the step B409, forms the second subgraph by connection, and provides a clear object for subsequent "subgraph fusion" or "iterative optimization". For example, only by connecting P, U, and D in topological dependence, "P→U→D" can be taken as the second subgraph for detecting whether the fusion condition in the subgraph is met. It can be seen that the transformation from "node set" to "effective subgraph" is realized through logical association, ensuring the coherence of the subgraph splitting process in a closed loop and avoiding the failure of subsequent steps due to the absence of node connection.

[0109] As shown in Figure 3 , Figure 3 The upstream and downstream bidirectional search schematic diagram involved in the embodiment of the present application is shown in the figure, wherein the nodes are arranged in sequence from upstream to downstream according to data flow, including nodes Q, P, U, D, E, F, and X. Specifically, Q is the source node, the data arrangement is row-major-R, and it is the end point of the upstream search of the second branch; P is the first upstream node of U, the data arrangement is row-major-R, and it is the core node of the second branch; U is the starting point of bidirectional search, the data arrangement is row-major-R, and it is the core node triggering subgraph splitting; D is the current node to be split, the data arrangement is column-major-C, and it is the starting point of the first branch; E is the first downstream node of D, the data arrangement is row-major-R, and it is not included in the first subgraph because it does not match the data arrangement of D; F is the first associated node of D, the data arrangement is column-major-C, and it is included in the first subgraph because it matches the data arrangement of D; X is the last downstream node of D, the data arrangement is row-major-R, and it is not included in the first subgraph because it does not match the data arrangement of D. Among them, the solid arrows correspond to the original data flow direction of the calculation graph (such as Q→P, P→U, U→D, etc.), representing the static topological dependence that data is output from the starting node and input to the end node; the dashed arrows correspond to the traversal direction of DFS search. Specifically, when searching upstream (such as the dashed line in the direction of U→P→Q), the data flow is reversed, marking the trajectory of "traversing from the upstream node U to its first upstream P and the source Q"; when searching downstream (such as the dashed line in the direction of D→E→F→X), the data flow is in order, marking the trajectory of "traversing from the current node D to its first downstream E and the associated node F".

[0110] From the perspective of branching and search processes, the first branch starts with D and covers the downstream link D→E→F→X. A one-way downstream search is performed from D, selecting only F nodes with the same data arrangement as D. These two nodes are moved to the lower partition and connected according to the original data flow (D→F), forming the first subgraph. The second branch starts with upstream node U and covers the upstream link U←P. Since U is not the source node of the second branch and P does not belong to the lower partition, the upstream search condition is met. The search proceeds upstream from U to P, then to the source node Q, and stops there, selecting P and Q nodes with the same data arrangement as U as the second upstream nodes. The third branch also starts with upstream node U and covers the downstream link U→D. Since D belongs to the lower partition, the downstream search condition is not met, and the downstream search stops.

[0111] like Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the upper and lower partitioning involved in the embodiments of this application. Specifically, by determining the second branch containing U and its first upstream node P, U is not the source of the second branch (P has a higher upstream node Q), and P does not belong to the lower partition (the lower partition is the area where the current node D and downstream nodes are located), satisfying the preset upstream search conditions, the search proceeds upstream from U in reverse data flow until the source node Q of the second branch is found. P and Q, which are found to be consistent with the data arrangement type identifier (row priority - R) of U, are selected as the upstream second nodes and moved into the preset upper partition together with U. Therefore, the upper partition contains the source node Q, the first upstream node P of the upstream node, and the upstream node U at the starting point of the second and third branches.

[0112] By identifying the first branch containing D and its first downstream node E, a one-way search is performed from D downstream along the data flow. F, which matches the data arrangement type identifier (column priority - C) of D, is selected as the first associated node and moved into the preset lower partition along with D. Although E and X have a data arrangement of row priority - R, which does not match D, they are naturally assigned to the lower partition because they belong to the downstream area of ​​D. As boundary nodes in the lower partition, the lower partition contains the current node D at the starting point of the first branch, the first downstream node E of the current node, the first associated node F of the current node, and the last downstream node X of the first branch.

[0113] like Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the construction of the first and second sub-graphs involved in the embodiments of this application, based on the foregoing Figure 4It can be seen that in the lower partition, the current node D is connected with the first associated node F screened out by the downstream one-way search and having the same data arrangement type identifier (column first-C) as D, to form a first subgraph according to the preset computing graph topology dependency (i.e. the data flow order in the original computing graph in which D outputs flow to F inputs). In the upper partition, the upstream node U is connected with the upstream second nodes P, Q screened out by the upstream search and having the same data arrangement type identifier (row first-R) as U, to form a second subgraph according to the preset computing graph topology dependency (i.e. the data flow order in the original computing graph in which Q outputs flow to P inputs and P outputs flow to U inputs). The nodes in the partition are connected by retaining the topology dependency of the original computing graph, so that the upper and lower partitions are respectively transformed into the first subgraph and the second subgraph having the same data arrangement and complete associated link, thereby providing an independent and adaptive structural basis for subsequent operator fusion.

[0114] In the embodiment, a second branch containing the upstream node and a first upstream node of the upstream node in the computing graph is determined, and it is judged whether the second branch satisfies a preset upstream direction search condition. The preset upstream direction search condition includes that the upstream node is not a source node of the second branch and the first upstream node of the upstream node does not belong to the lower partition. If the condition is satisfied, the upstream direction search is performed from the upstream node as a starting point in the second branch, and the upstream direction search is stopped when the upstream node searched in the upstream direction is the source node of the second branch. The node consistent with the second data arrangement type identifier searched in the upstream direction search is determined to obtain an upstream second node having an association relationship with the upstream node. A third branch containing the upstream node and a first downstream node of the upstream node in the computing graph is determined, and it is judged whether the third branch satisfies a preset downstream direction search condition. The preset downstream direction search condition includes that the first downstream node of the upstream node does not belong to the lower partition. If the condition is satisfied, the downstream direction search is performed from the upstream node as a starting point in the third branch, and the downstream direction search is stopped when the downstream node searched in the downstream direction is a node of the lower partition. The node consistent with the second data arrangement type identifier searched in the downstream direction search is determined to obtain a downstream second node having an association relationship with the upstream node. The upstream second node and the downstream second node are moved to a preset upper partition containing the upstream node as the second associated nodes, and other nodes in the second branch and the third branch that are not moved to the upper partition are moved to the lower partition. At least one second associated node in the upper partition and the upstream node are connected according to the preset computing graph topology dependency to obtain a second subgraph, so as to realize construction of the second subgraph having the same data arrangement and clear boundary, reduce the computing complexity, and improve the operator fusion efficiency and the GPU resource utilization rate.

[0115] Further, based on the above content, in some feasible embodiments, the subgraph merging process includes node fusion of the subgraph, and the step S60 of performing the subgraph merging process on all the second subgraphs belonging to the same graph attribute to obtain the third subgraph further includes the following implementation steps B601 to B602.

[0116] The step B601: for each second subgraph, the data arrangement type of the upstream node in the second subgraph is taken as the graph attribute of the second subgraph. In this embodiment, through the upstream and downstream bidirectional search and screening, it is ensured that the data arrangement types of all the nodes in the second subgraph are completely consistent with the upstream node. Therefore, the data arrangement type of the upstream node can be taken as the characteristic identifier of the entire subgraph, accurately reflects the core attribute of the subgraph, and ensures that only the subgraphs with consistent data arrangement are merged subsequently.

[0117] Optionally, by taking the arrangement type of the upstream node as the graph attribute, the structural characteristics of the subgraph are converted into identifiable attribute information, realizing the conversion from the "subgraph entity" to the "attribute label", and providing a standardized input for subsequent processing, wherein the graph attribute includes but is not limited to row priority and column priority.

[0118] The step B602: performing node fusion on the nodes in all the second subgraphs belonging to the same graph attribute to obtain the third subgraph. In this embodiment, specifically, the "same graph attribute" in the step B602 needs to be defined as the common constraint that can be fused between subgraphs, including but not limited to consistent data arrangement (such as memory storage format, tensor dimension matching), acyclic topology structure (no circular dependency after merging), and compatible operator operation type (such as convolution or full connection operator), so as to ensure that there is no conflict in data flow conversion between nodes and the structure is reasonable when different second subgraphs are merged. For example, if the data arrangements of two second subgraphs are NHWC and NCHW formats respectively, the two second subgraphs cannot be merged due to incompatible attributes, thereby avoiding memory access errors or calculation logic disorder caused by fusion.

[0119] Optionally, the "node fusion" is not a simple merging of nodes, and the topological dependency relationship of the original nodes in the second subgraph needs to be maintained, that is, the upstream and downstream connection directions are retained (such as the connection of node X→Y in the second subgraph A and the connection of node Y→Z in the second subgraph B, and the data flow of X→Y→Z is still maintained after merging), so as to ensure that the structure of the third subgraph is consistent with the local logic of the original calculation graph. If the connection order is changed at will, the data dependency may be broken, so that the subsequent operator fusion cannot be executed due to logical errors.

[0120] Optionally, there can be multiple second subgraphs belonging to the same graph attribute, and all second subgraphs satisfying the attribute constraints need to be merged first to maximize the size of the third subgraph. For example, when the upstream node generates three second subgraphs, all of which satisfy the attributes of "data arrangement NHWC and acyclic topology", all of them are merged into one third subgraph, instead of only part of them, so as to provide a larger optimization unit for operator fusion and improve the calculation efficiency after fusion (such as reducing the number of operator calls and reducing data transfer overhead).

[0121] Optionally, this step is connected to step S50 (cutting off the upstream and downstream connections and splitting the calculation graph into independent subgraphs), and the dispersed "intermediate subgraph" is reconstructed into a "fusible unit" by merging second subgraphs with the same attribute. If this step is skipped, multiple isolated second subgraphs will be insufficient in size, resulting in operator fusion only being able to be performed in a single small range, which cannot achieve batch optimization across subgraphs and weakens the performance improvement effect of the overall calculation graph. Therefore, node fusion connects "aggregated compatible subgraphs" to link the "subgraph splitting" and "operator fusion" links, forming a closed-loop logic of "splitting -> screening -> aggregation -> optimization", which guarantees the effectiveness of subsequent fusion operations.

[0122] In this embodiment, for each second subgraph, the data arrangement type of the current node to be split in the second subgraph is taken as the graph attribute of the second subgraph, and the nodes in all second subgraphs belonging to the same graph attribute are fused to obtain a third subgraph, thereby effectively supporting the efficiency and implementability of the operator fusion algorithm through the combination of technical logic and specification requirements.

[0123] Further, based on the content of any of the above embodiments, in some feasible embodiments, the step B602 of fusing the nodes in all second subgraphs belonging to the same graph attribute further includes the following implementation steps B6021 to B6023.

[0124] Step B6021: directional analysis is performed on any second subgraph belonging to the same graph attribute to determine whether there is a closed loop path in the second subgraph; Step B6022: if there is a closed loop path, stop fusing the nodes in all second subgraphs belonging to the same graph attribute; Step B6023: if all second subgraphs belonging to the same graph attribute do not have a closed loop path, fuse the nodes in all second subgraphs belonging to the same graph attribute.

[0125] Specifically, the "directional analysis" of step B6021 needs to traverse the directional connection relationship of the nodes in each second subgraph under the same graph attribute through algorithms such as depth-first search (DFS) or topological sorting to detect whether there is a closed loop path that "starts from a certain node and returns to itself after a number of edges".

[0126] Optionally, the "stop fusion" logic in step B6022 is based on the characteristic that "a single closed loop can pollute the entire fusion result": if any second subgraph under the same graph attribute has a closed loop, even if other subgraphs are acyclic, the merged third subgraph will still inherit that closed loop. For example, if second subgraph A contains a loop and B does not, after fusion, the loop of A will be embedded in the third subgraph, causing the computation graph to fail to terminate. Therefore, as soon as a single closed loop is detected, fusion is stopped immediately to avoid generating illegal subgraphs and reduce subsequent debugging costs.

[0127] Optionally, the "fusion prerequisite" in step B6023 requires that all second subgraphs with the same graph attribute be acyclic, ensuring the topology of the third subgraph is valid while maintaining the compatibility logic of "graph attribute filtering". For example, if all three second subgraphs satisfy "NHWC data arrangement, convolution operator, and acyclic topology", the merged third subgraph can safely perform operator fusion, avoiding fusion failure due to topology issues.

[0128] From a process flow perspective, B6021-6023 follows the "graph attribute grouping" step, supplementing it with a secondary topology validity check: even if the "graph attributes" implicitly contain the constraint of "topological acyclicity," the construction process of the second subgraph may unexpectedly introduce cycles due to the complex connections between associated nodes. Therefore, by using loop detection, the "potential topology risks in the subgraph construction stage" are addressed, ensuring the dual constraints of "graph attribute screening and loop verification," providing a set of structurally valid and attribute-compatible subgraphs for node fusion, ultimately guaranteeing the stability and performance improvement of operator fusion.

[0129] like Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the existence of multiple second sub-diagrams in the embodiments of this application, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of the merged third sub-diagram involved in the embodiment of this application. Figure 5 There are two second subgraphs: the upper second subgraph consists of the source node Q, the first upstream node P of the upstream node, and the upstream node U; the lower second subgraph consists of node K and node J (both have row-major-R data arrangement). Following step S601, the data arrangement type of the upstream nodes (U and K) in each second subgraph is taken as the graph attribute. Since they have the same graph attribute, they belong to the same second subgraph attribute. Step S6021 performs a directedness analysis on these two second subgraphs, finding that the link Q→P→U in the upper subgraph and the link K→J in the lower subgraph have no closed-loop paths. After satisfying the conditions in step S6023, the nodes of the second subgraphs with the same graph attribute are merged. During merging, based on the original computation graph topology dependencies, an implicit direct data flow relationship is found between upstream node U and node K. Since the branch search boundary was not connected during the previous splitting stage, the link U→K is connected, ultimately forming the following...Figure 7 The third subgraph of Q→P→U→K→J retains the original node data flow direction, realizes the node fusion of the second subgraph with the same attribute, and provides a more complete structural unit for subsequent operator fusion.

[0130] In this embodiment, whether there is a closed loop path in the second subgraph is determined by performing directionality analysis on any second subgraph belonging to the same graph attribute; if there is a closed loop path, the node fusion of the nodes in all second subgraphs belonging to the same graph attribute is stopped; if there is no closed loop path in all second subgraphs belonging to the same graph attribute, the node fusion of the nodes in all second subgraphs belonging to the same graph attribute is performed, realizing the closed loop detection to guarantee the legality of the subgraph structure, the same attribute screening to ensure the data compatibility, and the combination of forming a coherent logical link, which effectively supports the efficiency and implementability of the operator fusion algorithm.

[0131] Further, based on the content of any of the above embodiments, in some feasible embodiments, the operator fusion method further includes the following implementation steps B70 to B80.

[0132] Step B70: Real-time statistics of the total number of current subgraphs is performed to determine whether the total number of subgraphs is greater than a preset subgraph number threshold, wherein the total number of subgraphs includes the number of the first subgraph, the second subgraph, and the third subgraph after the subgraph merging. In this embodiment, one of the core goals of operator fusion is to reduce the complexity of the computation graph. If the number of subgraphs is too large, such as the first subgraph and the second subgraph are not effectively fused after splitting, or the third subgraph is still out of control after merging, it will lead to an increased burden of computation resource scheduling, and thus reduce the efficiency. Specifically, the total number of all effective subgraphs is real-time statistics, which is compared with the preset threshold to determine whether the subgraph size is within a reasonable range, thereby providing a decision basis for whether to continue splitting, fusion, or terminate the process.

[0133] Optionally, the total number of subgraphs is limited to the sum of the first subgraph, the second subgraph, and the third subgraph, without missing any effective subgraph generated in the previous steps. For example, if there are currently 3 first subgraphs, 2 second subgraphs, and 1 third subgraph, the total number is 6, which ensures that all subgraph units affecting the complexity of computation are covered, so as to avoid inaccurate threshold determination due to ambiguous statistical range, and ensure comprehensive and effective control of the subgraph size.

[0134] Optionally, the preset subgraph number threshold is a reasonable upper limit set based on the computation resource and model demand, which is used to trigger the regulation of the subgraph size. For example, if the threshold is set to 10, when the total number reaches 11, it is determined that it is out of range, and the subgraph merging optimization needs to be started. Therefore, the threshold mechanism is used to prevent unlimited growth of the number of subgraphs, and to avoid the efficiency decline caused by the surge of computation amount.

[0135] Step B80: If the total number of subgraphs is greater than the preset subgraph number threshold, a preset pruning operation is performed according to the calculation graph to stop generating new subgraphs.

[0136] In this embodiment, operator fusion needs to reduce the complexity of the calculation graph while improving efficiency. If the first subgraph and the second subgraph are excessively generated, the total number of subgraphs will increase sharply, increasing the scheduling burden of computing resources and reducing overall performance. The new subgraph includes the first subgraph and the second subgraph. Specifically, when step B70 determines that the total number of subgraphs exceeds the preset threshold, the generation of new first subgraphs and second subgraphs is terminated by a "preset pruning operation". These two types of subgraphs are the basic units for subsequent fusion, but their unrestricted generation will directly lead to out-of-control scale. The pruning operation is based on the topology of the calculation graph and ensures that the integrity of the existing subgraphs is not damaged while stopping the generation of new subgraphs, thereby controlling the size of the subgraph within a reasonable range.

[0137] Optionally, the pruning object of the pruning operation is only for "new subgraphs", and does not affect the first subgraph, the second subgraph, and the third subgraph that have been generated. For example, if 4 first subgraphs, 2 second subgraphs, and 1 third subgraph have been generated, and the total number exceeds the threshold, the pruning operation will no longer start the construction process of new first and second subgraphs, but the existing subgraphs can continue to participate in subsequent operations such as fusion to avoid damaging the processing link of the existing subgraphs due to pruning. The scale control only acts on the incremental part, controlling the total amount and preserving the results of the early processing.

[0138] Optionally, the pruning operation is performed "according to the calculation graph", that is, based on the node topology dependency relationship of the calculation graph to determine the boundary for stopping the generation of new subgraphs, such as not starting the subgraph construction for non-core link nodes that have not been split, rather than randomly terminating. For example, the node splitting authority of the core path in the calculation graph is preferentially retained, and the generation of new subgraphs for edge path nodes is stopped to ensure that the core calculation logic is not affected, thereby ensuring that the pruning operation does not damage the core function of the calculation graph, maintaining the effectiveness of the algorithm while controlling the scale.

[0139] It should be noted that the threshold judgment result of the pruning operation performed when the number of subgraphs exceeds the limit in this step B80 forms a response mechanism of "number exceeding the limit, starting pruning, and stopping new subgraph generation". Subsequent fusion optimization can be continued based on the existing subgraphs, forming a complete closed loop of "scale detection, excess control, and stock optimization".

[0140] In the embodiment, the total number of current subgraphs is counted in real time, and it is judged whether the total number of subgraphs is greater than a preset subgraph number threshold, wherein the total number of subgraphs includes the subgraph numbers of the constructed first subgraph and second subgraph and the third subgraph after subgraph merging; if the total number of subgraphs is greater than the preset subgraph number threshold, a preset pruning operation is performed according to the calculation graph to stop generating new subgraphs, wherein the new subgraphs include the first subgraph and the second subgraph, so as to realize dynamic control of the subgraph size to avoid calculation overload, and the combination of the step of forming a continuous regulation link controls the calculation complexity in a reasonable range while ensuring the fusion effect, thereby effectively supporting the efficiency and implementability of the operator fusion algorithm.

[0141] In summary, the application provides an operator fusion method, which determines whether there are two first nodes with different data arrangement types and direct connections in the calculation graph, wherein the two first nodes are a current node to be split and an upstream node of the current node, and the subgraph boundary to be split can be accurately located; if there are, a downstream one-way search is performed in the calculation graph starting from the current node to obtain a first associated node having an association relationship with the current node, and a first subgraph is constructed according to the current node and the first associated node, so as to ensure that all nodes in the subgraph maintain the same data arrangement as the current node; an upstream and downstream bidirectional search is performed in the calculation graph starting from the upstream node to obtain a second associated node having an association relationship with the upstream node, and a second subgraph is constructed according to the upstream node and the second node, so as to completely cover all associated nodes related to the upstream node and ensure the integrity of the upstream subgraph; the first subgraph and the second subgraph are both regarded as new calculation graphs, and the step of determining whether there are two first nodes with different data arrangement types and direct connections in the calculation graph is performed according to the new calculation graphs, until it is detected that there are no two first nodes with different data arrangement types and direct connections in the latest calculation graph, so as to realize full optimization of the calculation graph; if there are multiple second subgraphs, all second subgraphs belonging to the same graph attribute are subjected to subgraph merging processing to obtain a third subgraph, so as to reduce the total number of subgraphs under the premise of ensuring the effectiveness of fusion, so as to realize operator fusion in the calculation graph through the third subgraph, thereby ensuring independent fusion of subgraphs, reducing invalid calculation and dependency conflicts, and improving overall efficiency.

[0142] In addition, the application also provides an operator fusion device, please refer to Figure 8 , Figure 8 is a structural schematic diagram of the operator fusion device involved in the embodiment of the application. The operator fusion device provided by the application comprises: The node detection module H01 is configured to determine whether there are two first nodes with different data arrangement types and direct connections in the calculation graph, wherein the two first nodes are a current node to be split and an upstream node of the current node; The first subgraph construction module H02 is configured to, if existing, perform a downstream one-way search in the computation graph starting from the current node to obtain a first associated node having an association relationship with the current node, and construct a first subgraph according to the current node and the first associated node. The second subgraph construction module H03 is configured to perform an upstream and downstream bidirectional search in the computation graph starting from the upstream node to obtain a second associated node having an association relationship with the upstream node, and construct a second subgraph according to the upstream node and the second node. The iteration processing module H04 is configured to take both the first subgraph and the second subgraph as a new computation graph, and perform the step of determining whether there are two first nodes having different data arrangement types and being directly connected in the computation graph according to the new computation graph, until it is detected that there are no two first nodes having different data arrangement types and being directly connected in the latest computation graph. The subgraph merging module H05 is configured to, if there are multiple second subgraphs, perform subgraph merging processing on all second subgraphs belonging to the same graph attribute to obtain a third subgraph, so as to realize operator fusion in the computation graph through the third subgraph.

[0143] The operator fusion device provided in the application adopts the operator fusion method in the above embodiment, and can solve the technical problem of low efficiency of the operator fusion device. Compared with the prior art, the operator fusion device provided in the application has the same beneficial effects as the operator fusion device method provided in the above embodiment, and other technical features in the operator fusion device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0144] In addition, the application also provides an operator fusion device. Please refer to Figure 9 , Figure 9 is a structural schematic diagram of an operator fusion device related to the embodiment scheme of the application. The device of the embodiment of the application can be a device for locally running the operator fusion method.

[0145] The application provides an operator fusion device, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the operator fusion device method in the above embodiment one.

[0146] The following refers to Figure 9The diagram illustrates a structural schematic suitable for implementing the operator fusion device of the embodiments of this application. The operator fusion device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The operator fusion device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0147] like Figure 9 As shown, the operator fusion device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the operator fusion device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following devices may be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the operator fusion device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows an operator fusion device with various devices, it should be understood that it is not required to implement or possess all of the shown devices. More or fewer devices may be implemented alternatively.

[0148] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0149] The operator fusion device provided in the present application adopts the operator fusion method in the above embodiments, and can solve the technical problem of low efficiency of the operator fusion device. Compared with the prior art, the operator fusion device provided in the present application has the same beneficial effects as the operator fusion method provided in the above embodiments, and other technical features in the operator fusion device are the same as the features disclosed in the previous embodiment method, which will not be described here.

[0150] In addition, the present application provides a computer readable storage medium. The computer readable storage medium stores an operator fusion program. When the operator fusion program is executed by a processor, the steps of the above-mentioned operator fusion method are implemented.

[0151] It should be noted that in this paper, the term "including" "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0152] The above-mentioned embodiment number of the present application is only for description, not representing the advantages and disadvantages of the embodiments.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a computer readable storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) as described above, and includes a plurality of instructions for causing an apparatus (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the embodiments of the present application.

[0154] The preferred embodiments of the present application have been described above with the illustrated embodiments, and are not intended to limit the scope of patent protection for the present application. Any equivalent structure or equivalent process variations, which directly or indirectly incorporate the contents of the specification and drawings of the present application, are also intended to be included within the scope of patent protection for the present application.

Claims

1. An operator fusion method, characterized by, The operator fusion method comprises: determining whether two first nodes with different data arrangement types and directly connected exist in the calculation graph, wherein the two first nodes are a current node to be split and an upstream node of the current node; if so, performing downstream unidirectional search in the calculation graph starting from the current node to obtain first associated nodes having an association relationship with the current node, and constructing a first subgraph according to the current node and the first associated nodes; performing upstream and downstream bidirectional search in the calculation graph starting from the upstream node to obtain second associated nodes having an association relationship with the upstream node, and constructing a second subgraph according to the upstream node and the second nodes; taking the first subgraph and the second subgraph as new calculation graphs, and performing the step of determining whether two first nodes with different data arrangement types and directly connected exist in the calculation graph according to the new calculation graphs respectively until it is detected that no two first nodes with different data arrangement types and directly connected exist in the latest calculation graph; if multiple second subgraphs exist, performing subgraph merging processing on all second subgraphs belonging to the same graph attribute to obtain a third subgraph, so as to realize operator fusion in the calculation graph through the third subgraph.

2. The operator fusion method of claim 1, wherein, The step of determining whether two first nodes with different data arrangement types and directly connected exist in the calculation graph comprises: traversing each node in the calculation graph based on a preset calculation graph topology dependency relationship; if the traversed node has an upstream node, determining a first data arrangement type identifier of the traversed node, and determining a second data arrangement type identifier of the upstream node corresponding to the traversed node; detecting whether the first data arrangement type identifier and the second data arrangement type identifier are consistent; if not, determining that the data arrangement types between the traversed node and the upstream node corresponding to the traversed node are different, and taking the traversed node as a current node to be split and taking the upstream node corresponding to the traversed node as an upstream node of the current node.

3. The operator fusion method of claim 2, wherein, The step of performing downstream unidirectional search in the calculation graph starting from the current node to obtain first associated nodes having an association relationship with the current node, and constructing a first subgraph according to the current node and the first associated nodes comprises: determining a first branch containing the current node and a first downstream node of the current node in the calculation graph; performing downstream unidirectional search in the first branch starting from the current node until the last downstream node in the first branch is searched; moving all downstream nodes searched by the downstream unidirectional search to a preset lower partition containing the current node as the first associated nodes; connecting at least one first associated node in the lower partition and the current node according to a preset calculation graph topology dependency relationship to obtain a first subgraph.

4. The operator fusion method of claim 3, wherein, The upstream and downstream bidirectional search includes an upstream direction search and a downstream direction search, the upstream and downstream bidirectional search is performed in the calculation graph starting from the upstream node, second associated nodes having an association relationship with the upstream node are obtained, and the step of constructing a second subgraph according to the upstream node and the second node includes: determining a second branch containing the upstream node and a first upstream node of the upstream node in the calculation graph, and judging whether the second branch meets a preset upstream direction search condition; if yes, performing an upstream direction search in the second branch starting from the upstream node, and stopping the upstream direction search when an upstream node searched in the upstream direction search is a source node of the second branch; determining a node searched in the upstream direction search and having the same second data arrangement type identifier, to obtain an upstream second node having an association relationship with the upstream node; determining a third branch containing the upstream node and a first downstream node of the upstream node in the calculation graph, and judging whether the third branch meets a preset downstream direction search condition; if yes, performing a downstream direction search in the third branch starting from the upstream node, and stopping the downstream direction search when a downstream node searched in the downstream direction search is a node of the lower partition; determining a node searched in the downstream direction search and having the same second data arrangement type identifier, to obtain a downstream second node having an association relationship with the upstream node; moving the upstream second node and the downstream second node to a preset upper partition containing the upstream node as the second associated nodes, and moving other nodes in the second branch and the third branch that are not moved to the upper partition to the lower partition; connecting at least one second associated node in the upper partition and the upstream node according to a preset calculation graph topology dependency relationship, to obtain a second subgraph; the connecting at least one first associated node in the lower partition and the current node according to the preset calculation graph topology dependency relationship to obtain a first subgraph includes: connecting at least one first associated node in the lower partition, the current node and other nodes according to the preset calculation graph topology dependency relationship to obtain a first subgraph.

5. The operator fusion method of claim 4, wherein, The subgraph merging processing includes node fusion of the subgraph, the subgraph merging processing is performed on all second subgraphs belonging to the same graph attribute to obtain a third subgraph, including: for each second subgraph, taking a data arrangement type of the upstream node in the second subgraph as a graph attribute of the second subgraph; performing node fusion on nodes in all second subgraphs belonging to the same graph attribute to obtain a third subgraph.

6. The operator fusion method of claim 5, wherein, The node fusion on nodes in all second subgraphs belonging to the same graph attribute includes: performing directedness analysis on any second subgraph belonging to the same graph attribute, to determine whether there is a closed loop path in the second subgraph; if there is a closed loop path, stop the node fusion on nodes in all second subgraphs belonging to the same graph attribute; If all the second subgraphs belonging to the same graph attribute do not exist a closed loop path, then the nodes in all the second subgraphs belonging to the same graph attribute are fused.

7. The operator fusion method of claim 1, wherein, The operator fusion method further includes: The total number of subgraphs is counted in real time, and it is determined whether the total number of subgraphs is greater than a preset subgraph number threshold, wherein the total number of subgraphs includes the number of subgraphs of the first subgraph and the second subgraph constructed and the third subgraph after subgraph merging. If the total number of subgraphs is greater than the preset subgraph number threshold, a preset pruning operation is performed according to the computation graph to stop generating new subgraphs.

8. An operator fusion device, characterized by The operator fusion device includes: A node detection module is configured to determine whether two first nodes with different data arrangement types and directly connected exist in a computation graph, wherein the two first nodes are a current node to be split and an upstream node of the current node; A first subgraph construction module is configured to, if the two first nodes exist, perform a downstream one-way search in the computation graph starting from the current node to obtain a first associated node having an association relationship with the current node, and construct a first subgraph according to the current node and the first associated node; A second subgraph construction module is configured to perform an upstream and downstream bidirectional search in the computation graph starting from the upstream node to obtain a second associated node having an association relationship with the upstream node, and construct a second subgraph according to the upstream node and the second node; An iteration processing module is configured to take the first subgraph and the second subgraph as new computation graphs, and perform the step of determining whether two first nodes with different data arrangement types and directly connected exist in a computation graph according to the new computation graphs, respectively, until it is detected that no two first nodes with different data arrangement types and directly connected exist in a latest computation graph; A subgraph merging module is configured to, if multiple second subgraphs exist, perform subgraph merging processing on all the second subgraphs belonging to the same graph attribute to obtain a third subgraph, so that operator fusion in the computation graph is realized through the third subgraph.

9. An operator fusion device, characterized by The operator fusion device includes a memory, a processor, and an operator fusion program stored on the memory and executable on the processor, and the processor implements the steps of the operator fusion method according to any one of claims 1 to 7 when executing the operator fusion program.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores an operator fusion program, and the operator fusion program implements the steps of the operator fusion method according to any one of claims 1 to 7 when executed by a processor.

Images