Data flow partitioning and scheduling method and device based on topology structure

Abstract

The application relates to a data flow division and scheduling method and device based on a topological structure. The method comprises the following steps: obtaining an initial DAG graph corresponding to a target neural network model; performing pipeline splitting on the initial DAG graph to obtain a plurality of target pipelines; the nodes in the target pipelines satisfy a serial data dependency condition; performing hierarchical division on the plurality of target pipelines to obtain a plurality of scheduling hierarchical levels with serial dependency relationships; the target pipelines in the same scheduling hierarchical level satisfy a parallel data dependency condition, and the target pipelines in the same scheduling hierarchical level respectively occupy independent hardware resources. The method can obtain scheduling hierarchical levels with serial dependency relationships, and the scheduling hierarchical levels include at least one pipeline with a parallel data dependency condition; the pipelines are decoupled in terms of hardware resources, the parallelism of network operation is improved, the communication overhead of cross-layer pipelines is reduced, and the hardware resource utilization rate is improved.

Description

Technical Field

[0001] This application relates to the field of neural network model accelerator technology, and in particular to a data flow partitioning and scheduling method and apparatus based on topology structure. Background Technology

[0002] With the increasing structural complexity, number of layers, and scale of parameters in neural network models, mapping and scheduling technologies for large-scale AI chips, such as multi-core and multi-chip systems, have become a key element in improving system performance.

[0003] In related technologies, the Layer-Pipeline (LP) approach is typically used to divide the Directed Acyclic Graph (DAG) of the neural network model into a single long pipeline for hardware resource scheduling. In addition, micro-batches are used to enable inter-layer computational overlap between hardware resources, thereby improving hardware resource utilization.

[0004] However, because the LP method uses a single long pipeline scheduling, the coupling between hardware resources is relatively high. Therefore, there are problems such as insufficient utilization of parallelism and high communication overhead for cross-layer pipelines. Summary of the Invention

[0005] Based on this, it is necessary to provide a topology-based data flow partitioning and scheduling method, apparatus, computer equipment, computer-readable storage medium, and computer program product that, without changing data dependencies, retains parallel pipeline paths, thereby reducing the coupling of multi-core hardware resources, improving the parallelism of neural network models during operation, and reducing the communication overhead of cross-layer pipelines.

[0006] Firstly, this application provides a data flow partitioning and scheduling method based on topology, including:

[0007] Obtain the initial DAG graph corresponding to the target neural network model;

[0008] The initial DAG graph is decomposed into multiple target pipelines; the nodes in the target pipelines satisfy the serial data dependency condition.

[0009] Multiple target pipelines are hierarchically divided to obtain multiple scheduling levels with serial dependencies; the target pipelines within the same scheduling level satisfy the condition of parallelism without data dependencies, and each target pipeline within the same scheduling level occupies independent hardware resources.

[0010] In one embodiment, the initial DAG graph is pipelined to obtain multiple target pipelines, including:

[0011] Identify at least one source node with an in-degree of zero in the initial DAG graph; the source node serves as the first node of the corresponding target pipeline.

[0012] For each source node, each node is traversed layer by layer along the data flow direction of the initial DAG graph to determine at least one target node corresponding to the source node that satisfies the first preset condition; the first preset condition includes that the successor node of the previous node of the target node is unique, and the predecessor node of the target node is also unique.

[0013] Based on the source node and at least one corresponding target node, generate the corresponding target pipeline.

[0014] In one embodiment, the method further includes:

[0015] If the target node is a diverging point, or if the successor node of the target node is a convergence point, then all successor nodes of the target node will be used as new source nodes.

[0016] Return to the execution step of traversing each node layer by layer along the data flow direction of the initial DAG graph for each source node, and determining at least one target node corresponding to the source node that meets the preset conditions.

[0017] In one embodiment, multiple target pipelines are hierarchically divided to obtain multiple scheduling levels with serial dependencies, including:

[0018] From multiple target pipelines, at least one target pipeline that meets the second preset condition is determined; the second preset condition includes that all predecessor pipelines of the target pipeline are included in the prior scheduling hierarchy.

[0019] Based on at least one target pipeline that satisfies the second preset condition, construct at least one scheduling level corresponding to the target pipeline.

[0020] In one embodiment, determining at least one target pipeline that satisfies a second preset condition from a plurality of target pipelines includes:

[0021] Construct a set of pipelines to be scheduled based on multiple target pipelines;

[0022] Determine the target pipeline corresponding to each source node with an in-degree of zero from the set to be scheduled;

[0023] The first scheduling level is constructed based on the target pipelines corresponding to each source node, and the target pipelines corresponding to each source node in the set to be scheduled are deleted.

[0024] From the deleted set of pipelines to be scheduled, determine at least one target pipeline that meets the second preset condition; the target pipelines whose first node contains all the predecessor nodes of the target pipeline that meets the second preset condition are not in the deleted set of pipelines to be scheduled.

[0025] In one embodiment, the method further includes:

[0026] For multiple scheduling levels with serial dependencies, a resource scheduling scheme is determined for each scheduling level; the resource scheduling scheme for each scheduling level includes the hardware resources corresponding to each target pipeline in the scheduling level.

[0027] Based on the serial dependency relationship, hardware resources are mapped according to the resource scheduling scheme corresponding to the current scheduling level, so as to instruct the corresponding hardware resources to execute each target pipeline in the current scheduling level.

[0028] Secondly, this application also provides a data flow partitioning and scheduling device based on topology, comprising:

[0029] The acquisition module is used to acquire the initial DAG graph corresponding to the target neural network model;

[0030] The pipeline splitting module is used to split the initial DAG graph into multiple target pipelines; the nodes in the target pipelines satisfy the serial data dependency condition.

[0031] The hierarchical partitioning module is used to partition multiple target pipelines into hierarchical levels, resulting in multiple scheduling levels with serial dependencies. The target pipelines within the same scheduling level meet the condition of parallelism without data dependencies, and each target pipeline within the same scheduling level occupies independent hardware resources.

[0032] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the topology-based data flow partitioning and scheduling method described in the first aspect above.

[0033] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the topology-based data flow partitioning and scheduling method described in the first aspect above.

[0034] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the topology-based data flow partitioning and scheduling method described in the first aspect above.

[0035] The aforementioned data flow partitioning and scheduling method, apparatus, computer equipment, storage medium, and computer program product based on topology structure obtain an initial DAG graph corresponding to the target neural network model and perform pipeline decomposition on the initial DAG graph to obtain multiple target pipelines; the nodes in each target pipeline satisfy the condition of serial data dependency; then, the multiple target pipelines are hierarchically partitioned to obtain multiple scheduling levels with serial dependencies; wherein, the target pipelines within the same scheduling level satisfy the condition of parallel non-data dependency, and each target pipeline within the same scheduling level occupies independent hardware resources; that is, in the method proposed in this application, by obtaining an initial DAG graph corresponding to the target neural network model and performing pipeline decomposition on the initial DAG graph to obtain multiple target pipelines; the nodes in each target pipeline satisfy the condition of serial data dependency, and the nodes in each target pipeline occupy independent hardware resources; that is, in the method proposed in this application, by obtaining an initial DAG graph corresponding to the target neural network model and performing pipeline decomposition on the initial DAG graph to obtain multiple target pipelines; the nodes in each target pipeline satisfy the condition of parallel non-data dependency, and the nodes in each target pipeline occupy independent hardware resources; that is, in the method proposed in this application, the initial DAG graph is obtained by obtaining an initial DAG graph corresponding to the target neural network model and performing pipeline decomposition on the target ... satisfy the condition of parallel non-data dependency, and the nodes in each target pipeline satisfy the condition of parallel non-data dependency, and the nodes in each target pipeline satisfy the condition of parallel non-data dependency, and the nodes in each target pipeline satisfy the condition of parallel non-data dependency, and the nodes in each target pipeline satisfy the condition of parallel non-data dependency, and the nodes in The DAG graph is decomposed and hierarchically partitioned to obtain multiple scheduling levels with serial dependencies. Each scheduling level includes multiple pipelines that can be executed in parallel and have no data dependencies on each other. Based on this, when allocating hardware resources based on the scheduling level, the hardware resources occupied by each pipeline can be decoupled, and different hardware resources can execute their respective pipelines in parallel and release hardware resources in advance. Furthermore, data transmission across layers can be avoided when each hardware resource executes its corresponding pipeline. Therefore, this method not only improves the parallelism of network operation but also reduces the communication overhead of cross-layer pipelines, further improving the utilization rate of hardware resources. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is an application environment diagram of a data flow partitioning and scheduling method based on topology in one embodiment;

[0038] Figure 2 This is a flowchart illustrating the data flow partitioning and scheduling method based on topology in the first embodiment;

[0039] Figure 3 This is a flowchart illustrating the data flow partitioning and scheduling method based on topology in the second embodiment;

[0040] Figure 4 This is a flowchart illustrating the data flow partitioning and scheduling method based on topology in the third embodiment;

[0041] Figure 5 This is a schematic diagram illustrating the construction of a parallel pipeline and scheduling hierarchy in one embodiment;

[0042] Figure 6 This is a flowchart illustrating the data flow partitioning and scheduling method based on topology in the fourth embodiment;

[0043] Figure 7 This is a block diagram of a data flow partitioning and scheduling device based on topology in one embodiment;

[0044] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] With the increasing complexity, number of layers, and parameter size of DNN models, mapping and scheduling techniques for large-scale AI chips, such as multi-core and multi-chip systems, have become crucial for improving system performance. Among related technologies, layer-sequential (LS) mapping and layer-pipeline (LP) mapping are commonly used. LP, by mapping multiple DNN layers to hardware for cross-layer pipelined execution, can reduce off-chip memory access to some extent and is therefore widely used in large-scale DNN acceleration scenarios. However, existing LP schemes typically partition the graph and then linearly expand the subgraphs into a single long pipeline for scheduling. This approach has at least the following drawbacks in practical applications:

[0047] (1) The pipeline stage imbalance problem is prominent. Due to the significant differences in computing scale, memory access requirements and communication costs among different layers, the execution time of each stage of a single long pipeline is inconsistent, and the overall completion time is limited by the slowest stage, resulting in a decrease in pipeline efficiency.

[0048] (2) Insufficient utilization of the parallelism of DAG structure. Existing LP adopts a scheduling method that executes DNN layers in a sequential manner, which transforms the branch sub-paths that could originally be executed concurrently into sequential or weakly parallel execution in time, making it difficult to fully utilize the structural parallelism features of DNN directed acyclic graph.

[0049] (3) High cross-layer communication overhead. The cross-layer data transport path is relatively long under linear mapping (e.g., the output data of chip 1 needs to be transmitted to chip 3 together with the output data of chip 2 after chip 2 has completed the calculation, and chip 3 needs to use the output data of chip 1 for analysis and calculation), which is prone to forming hot links and congestion in on-chip interconnects; in multi-chip scenarios, it will also increase the D2D communication burden, further increasing latency and energy consumption.

[0050] Based on this, this application addresses the problems of insufficient parallelism utilization, pipeline stage imbalance, and high cross-layer pipeline communication overhead caused by the linearization of multi-branch topology and subsequent single long pipeline scheduling in existing LP methods. It proposes a novel DNN DAG graph partitioning method that efficiently partitions the DAG graph and constructs an efficient parallel pipeline while ensuring correct dependencies. The parallel pipeline has at least the following advantages: First, it maintains linear dependencies within the pipeline, allowing for spatially adjacent data mapping between layers, resulting in shorter data transport paths and fewer communication hops, thus reducing link load. Second, there are no data dependencies between different pipelines, enabling synchronous execution on different hardware partitions and reducing hardware idleness caused by single pipeline bubbles through temporal staggered coverage. Third, multiple pipelines can be relatively isolated in terms of resources, reducing link contention and congestion, and improving overall load balancing, thereby enhancing system throughput stability.

[0051] The method proposed in this application relates to the fields of computer architecture and DNN accelerators, and more particularly to a computation graph partitioning and scheduling hierarchy construction method for multi-core deep neural network (DNN) accelerators. Specifically, it involves a method for topology-aware partitioning, parallel pipeline construction, and scheduling hierarchy organization of DNN directed acyclic graphs (DAGs), which is suitable for efficient mapping and execution of complex topological networks on resource-constrained hardware.

[0052] The data flow partitioning and scheduling method based on topology provided in this application can be applied to, for example... Figure 1 In the application environment shown, computer device 101 deploys and runs a neural network model, and computer device 101 includes a computing system, such as a multi-core or multi-chip system, that runs the neural network model. When running the neural network model, computer device 101 can use the method proposed in this application to partition the directed acyclic graph (DAG) of the neural network model and construct an efficient parallel pipeline. Then, based on the parallel pipeline, hardware resource mapping is performed to instruct the corresponding hardware resources to execute the relevant computation steps of the neural network model.

[0053] For example, computer device 101 can be a terminal device or a server device. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster composed of multiple servers.

[0054] In one exemplary embodiment, such as Figure 2As shown, a data flow partitioning and scheduling method based on topology structure is provided, which is then applied to... Figure 1 The following steps, 201 to 203, are used as an example of computer equipment.

[0055] Step 201: Obtain the initial DAG graph corresponding to the target neural network model.

[0056] For example, a computer device can first obtain the model structure of the target neural network model, and then generate an initial DAG graph corresponding to the target neural network model based on the model structure; wherein, the initial DAG graph includes multiple nodes and edges between each node, the nodes refer to the operators / layers of the target neural network model (such as convolutional layer Conv, linear layer Linear, activation layer ReLU, etc.); the edges refer to the flow of tensors, that is, the direction of data propagation.

[0057] Step 202: Decompose the initial DAG graph into multiple target pipelines.

[0058] In this context, the nodes in the target pipeline satisfy the condition of serial data dependency, that is, the nodes in the target pipeline propagate in a unidirectional sequence, and each intermediate node has only one predecessor node and each node has only one successor node. It should be noted that the first node of the target pipeline has zero predecessor nodes, or multiple predecessor nodes; the last node of the target pipeline has zero successor nodes, or multiple successor nodes.

[0059] For example, when decomposing the initial DAG graph into pipelines, each node in the initial DAG graph can be traversed based on a preset decomposition rule to obtain multiple target pipelines that satisfy the preset decomposition rule. For example, the preset decomposition rule may include: for the current node u, if the current node u has only one successor node w, and the predecessor node of the successor node w includes only one (i.e., the current node u), then the successor node w of the current node u can be added to the pipeline where the current node u is located; then, node w is taken as the new current node, and the process continues forward until the preset decomposition rule is no longer satisfied, thereby forming a target pipeline.

[0060] Step 203: Divide the multiple target pipelines into hierarchical levels to obtain multiple scheduling levels with serial dependencies; the target pipelines within the same scheduling level satisfy the condition of parallelism without data dependencies, and each target pipeline within the same scheduling level occupies independent hardware resources.

[0061] For example, after performing the first stage of node partitioning on the initial DAG graph based on preset partitioning rules, multiple target pipelines can be obtained, each of which includes at least one node. Then, for the multiple target pipelines, a second stage of hierarchical partitioning can be performed to obtain multiple scheduling levels. There is a serial linear dependency relationship between the scheduling levels, that is, the execution of the next scheduling level can only be triggered after the execution of the previous scheduling level is completed. Here, the completion of the execution of the previous scheduling level means that all target pipelines in the previous scheduling level have completed execution.

[0062] For a single scheduling level, the scheduling level may include at least one target pipeline. When there are multiple target pipelines, the multiple target pipelines can be executed in parallel. That is, there is no data dependency between multiple target pipelines in the same scheduling level, and they can form parallel pipelines to improve parallelism and execution efficiency.

[0063] Furthermore, each target pipeline within the same scheduling level occupies independent hardware resources. In traditional LP mapping, hardware resources are uniformly divided into serial resource mappings, resulting in strong coupling between these resources. However, the method proposed in this application allows hardware resources to be divided into multiple independent hardware resource modules, each capable of independently executing different target pipelines. It should be noted that hardware resources can be dynamically allocated. For example, when executing the first scheduling level, hardware resources can be divided into multiple hardware resource modules corresponding to the number of target pipelines in the first scheduling level, with the processing capacity of each module corresponding to the load of the corresponding target pipeline. When executing the second scheduling level, these hardware resources can be reallocated based on the number of target pipelines in the second scheduling level. For example: hardware resources include cores 1 to 8, the first scheduling level A includes one target pipeline, which corresponds to cores 1 to 3, the second scheduling level B includes three target pipelines, which correspond to cores 1 to 2, target pipeline B2 corresponds to cores 3 to 5, and target pipeline B3 corresponds to cores 6 to 8.

[0064] For example, when allocating hardware resources, hardware resources can also be allocated based on the number and / or load of each target pipeline, so that the execution time of each target pipeline is similar, avoiding the situation where the execution time of individual target pipelines in the same scheduling level is too long, which would slow down the execution rate of that scheduling level and thus delay the execution of the next scheduling level.

[0065] For example, for a single scheduling level, after the target pipeline completes its execution, the corresponding hardware resources can be released. The released hardware resources can be used by other target pipelines in the scheduling level, or by other tasks outside the scheduling level, in order to reduce hardware resource idling and improve the utilization rate of hardware resources.

[0066] In the aforementioned data flow partitioning and scheduling method based on topology structure, an initial DAG graph corresponding to the target neural network model is obtained, and the initial DAG graph is pipelined to obtain multiple target pipelines. The nodes in each target pipeline satisfy the condition of serial data dependency. Then, the multiple target pipelines are hierarchically partitioned to obtain multiple scheduling levels with serial dependencies. Within the same scheduling level, the target pipelines satisfy the condition of parallelism without data dependency, and each target pipeline within the same scheduling level occupies independent hardware resources. In other words, the method proposed in this application, by pipelined from the initial DAG graph... By dividing the system into hierarchical levels, multiple scheduling levels with sequential dependencies are obtained. Each scheduling level includes multiple pipelines that can be executed in parallel and have no data dependencies on each other. Based on this, when allocating hardware resources based on the scheduling levels, the hardware resources occupied by each pipeline can be decoupled. Different hardware resources can also execute their respective pipelines in parallel and release hardware resources in advance. Furthermore, data transmission across layers can be avoided when each hardware resource executes its corresponding pipeline. Therefore, this method not only improves the parallelism of network operation but also reduces the communication overhead of cross-layer pipelines, further improving the utilization rate of hardware resources.

[0067] In one exemplary embodiment, such as Figure 3 As shown, step 202 above may include steps 301 to 303. Wherein:

[0068] Step 301: Determine at least one source node with an in-degree of zero in the initial DAG graph.

[0069] In this context, the source node is a node with an in-degree of zero (i.e., the number of edges pointing to this node is zero), which is a node without a predecessor node. In other words, it is a node that directly uses the model's input data (rather than intermediate result data) as its input. The source node can be the first node of the corresponding target pipeline.

[0070] For example, the initial DAG graph may include at least one source node. The computer device can traverse all edges in the initial DAG graph and count the in-degree of each node to determine all source nodes with in-degree and out-degree of zero.

[0071] Step 302: For each source node, traverse each node layer by layer along the data flow direction of the initial DAG graph to determine at least one target node corresponding to the source node that satisfies the first preset condition.

[0072] The first preset condition includes that the successor node of the previous node of the target node is unique, and the predecessor node of the target node is also unique.

[0073] For example, in the pipeline construction (Build_Pipelines) stage, for each source node, the source node is taken as the current node u, and it is determined whether the next node of the source node u satisfies the first preset condition, that is, whether the successor node of the source node u is unique, and whether the predecessor node of the successor node is unique; if the successor node of the source node u is unique (|succ(u)|=1), and the predecessor node of the successor node w is also unique (|pred(w)|=1), then it can be determined that the successor node w satisfies the first preset condition, and thus the successor node w is taken as the target node corresponding to the source node u.

[0074] Next, continue traversing layer by layer along the data flow direction, taking the successor node of the source node as the new current node u, and determining whether its successor node meets the first preset condition. If it does, then take its successor node as the target node; if it does not, then stop the judgment, and the current pipeline terminates here.

[0075] It should be noted that the source node may not have a target node that meets the first preset condition. In this case, the source node can be used as a target pipeline independently, that is, the target pipeline contains only one node.

[0076] Step 303: Generate the corresponding target pipeline based on the source node and at least one corresponding target node.

[0077] For example, if a source node has at least one corresponding target node, then the corresponding target pipeline can be generated by sequentially connecting the source node and the corresponding target node according to the data flow direction. If the source node does not have a corresponding target node, then the corresponding target pipeline is generated independently based on the source node.

[0078] Furthermore, when judging the successor nodes of the target node, if the target node includes multiple successor nodes (|succ(u)|≠1, then the successor nodes do not meet the first preset condition), that is, the target node is a divergence point, indicating that the data flow branches here. At this time, the judgment of the current pipeline can be terminated, and all successor nodes of the target node can be used as new source nodes. For example, all successor nodes of the target node can be added to the processing queue Q, and then new pipeline construction can be started from each node in the processing queue Q, that is, new pipeline construction can be started from each branch of the divergence point.

[0079] In addition, when judging the successor node of the target node, if the target node includes a successor node, but the successor node includes multiple predecessor nodes (|pred(w)|>1, then the successor node does not meet the first preset condition), that is, the successor node of the target node is the convergence point, indicating that the data flow converges at the successor node, the judgment of the current pipeline is also terminated, and the successor node is added to the waiting queue Q as the starting node of the subsequent new pipeline, that is, the source node.

[0080] For each node in the queue to be processed Q, return to step 302 above to determine at least one target node corresponding to each node and form the corresponding target pipeline.

[0081] The above process is repeated until the queue Q is empty, at which point all nodes have been assigned to a pipeline. The final output pipeline set P consists of all the longest linear subsequences in the entire DAG that satisfy the "internal linear dependency" condition, i.e., the multiple target pipelines mentioned above.

[0082] In this embodiment, starting from the source node with an in-degree of zero in the initial DAG graph, each node is traversed layer by layer along the data flow direction. When a node meets the first preset condition, it is added to the corresponding pipeline. For nodes that do not meet the first preset condition, a new pipeline is restarted starting from the node that does not meet the first preset condition, until all nodes have been traversed, thereby forming multiple target pipelines that meet the serial data dependency condition. Using this method, the DAG graph can be partitioned in one traversal, reducing the complexity of DAG graph partitioning and improving graph partitioning efficiency.

[0083] In one exemplary embodiment, such as Figure 4 As shown, step 203 above may include steps 401 to 402. Wherein:

[0084] Step 401: Determine at least one target pipeline from multiple target pipelines that meets the second preset condition; the second preset condition includes that all predecessor pipelines of the target pipeline are included in the prior scheduling hierarchy.

[0085] Step 402: Based on at least one target pipeline that satisfies the second preset condition, construct a scheduling level corresponding to at least one target pipeline.

[0086] For example, a computer device can construct a set to be scheduled based on multiple target pipelines, determine the target pipelines corresponding to each source node with an in-degree of zero from the set to be scheduled, construct a first scheduling level based on the target pipelines corresponding to each source node, and delete the target pipelines corresponding to each source node in the set to be scheduled; further, determine at least one target pipeline that satisfies a second preset condition from the deleted set to be scheduled; wherein, the target pipelines containing all predecessor nodes of the first node of the target pipeline that satisfies the second preset condition are not in the deleted set to be scheduled; then, construct a corresponding scheduling level based on at least one target pipeline that satisfies the second preset condition.

[0087] For example, after designating the target pipelines corresponding to each source node with an in-degree of zero as the first scheduling level, it is possible to determine from the set to be scheduled (the target pipelines corresponding to each source node have been deleted) at least one target pipeline that has been deleted or included in the first scheduling level. At this point, the selected at least one target pipeline can constitute the second scheduling level. Then, it is possible to determine again from the set to be scheduled (the target pipelines corresponding to each source node have been deleted, and the target pipelines deleted and included in the second scheduling level) at least one target pipeline that has been deleted or included in the first or second scheduling level, and to include the selected at least one target pipeline in the third scheduling level. This process continues until all target pipelines in the set to be scheduled are deployed to their corresponding scheduling levels.

[0088] Specifically, after obtaining the pipeline set P, it is necessary to further determine the parallel relationships between the target pipelines to facilitate subsequent hardware resource allocation and execution scheduling. In the scheduling level construction (Build_Levels) phase, a topology hierarchical strategy can be adopted, maintaining a set R to be scheduled (initially P). In each iteration, all target pipelines whose predecessor pipelines have been scheduled are selected from R and assigned to the current scheduling level. Then remove the scheduled target pipeline from R, level number Increment. This process is repeated until R is empty. The condition for determining whether a target pipeline p can be scheduled is: the target pipelines containing all predecessor nodes of the first-level node p[0] of p are not in the current set to be scheduled R (i.e., {pipe(v): v∈pred(p[0])}∩R= This ensures that the dependency constraints between levels are strictly adhered to.

[0089] The final output scheduling hierarchy L={L0,L1,...,Lk-1} has the following properties: pipelines within the same hierarchy have no data dependencies and can be executed in complete parallelism; however, there are strict order constraints between different hierarchies, and they must be scheduled sequentially according to their hierarchy numbers. This step provides explicit parallelism information and dependency topology for subsequent pipeline parallel mapping and resource allocation.

[0090] For example, refer to Figure 5 As shown, the dashed lines represent scheduling hierarchy boundaries, the solid lines represent data dependencies, and the boxes labeled P1, P2, P3, and P4 represent the constructed pipeline units.

[0091] exist Figure 5 In the left-hand structure, the upstream node first enters the fork point, then expands into multiple parallel branches in the middle, and finally converges at the downstream node. The pipeline construction starts from the node with an in-degree of 0: when the condition of "unique successor and unique predecessor" is met, the successor is merged into the current pipeline; when a divergence point (|succ(u)|≠1) is encountered, the current pipeline is terminated and each successor is added to the waiting queue Q; when a convergence point (|pred(w)|>1) is encountered, the current pipeline is terminated before convergence and the convergence node is added to the waiting queue Q. This results in a pipeline set of "one upstream – multiple parallel middle branches – one downstream pipeline". Subsequently, when constructing the scheduling hierarchy, topology layering can be performed according to "whether the predecessor pipeline has been scheduled," resulting in Level i-1, Level i, and Level i+1. Level i-1 includes one target pipeline, Level i includes four target pipelines, and Level i+1 includes one target pipeline. The result satisfies the following conditions: pipelines within the same level have no data dependencies and can be executed in parallel; pipelines at different levels are executed strictly in the order of their dependencies.

[0092] Figure 5 The right-hand structure contains residual / shortcut connections, resulting in cross-layer dependency edges. The pipeline construction still divides linear subsequences according to divergence and convergence points, but because some nodes depend on both main branch paths and shortcut paths, pipeline boundaries are triggered earlier than convergence. During the scheduling hierarchy construction phase, computer devices are not layered according to graphical location, but rather according to whether the predecessor readiness condition is met. Since residual convergence nodes need to wait for multiple predecessors to complete, some nodes are assigned to Level i+1 or Level i+2, ensuring that the execution timing is completely consistent with the original DAG dependencies. Specifically, Level i-1 includes one target pipeline, Level i includes three target pipelines, Level i+1 includes one target pipeline, and Level i+2 includes one target pipeline.

[0093] The aforementioned data flow partitioning and scheduling method based on topology structure involves two stages: the pipeline construction (Build_Pipelines) stage and the scheduling level construction (Build_Levels) stage. These two stages together transform the directed acyclic graph (DAG) G=(V,E) of the DNN into a set of pipelines P={p1,p2,...,pm} satisfying linear dependency constraints, and a scheduling level sequence L={L0,L1,...,Lk-1} hierarchically based on dependencies. In other words, the multi-branch DAG is converted into a standardized scheduling representation of "pipeline set P + level sequence L," enabling the identification and partitioning of the longest parallelizable subgraph in the DAG. Firstly, the pipeline construction sub-process (Build_Pipelines) automatically identifies boundaries at the divergence and convergence points of the DAG, decomposing the computation graph into a set of the longest linear subsequences satisfying internal linear dependency constraints. Secondly, the scheduling level construction sub-process (Build_Levels) hierarchically generates a scheduling level sequence that satisfies "parallelism within the same level and ordered behavior across levels" based on topological dependencies.

[0094] This method, while maintaining the correctness of the original dependencies, fully preserves the parallelizable structure of the DNN DAG, avoiding the forced linear concatenation of branches. It provides explicit and executable parallel scheduling information for subsequent mapping and resource allocation. Compared to LP algorithms that force DAG concatenation, this method can more fully leverage the advantages of parallel pipelines, demonstrating good engineering practicality and scalability.

[0095] In one exemplary embodiment, such as Figure 6 As shown, the above method may further include steps 601 to 602. Wherein:

[0096] Step 601: For multiple scheduling levels with serial dependencies, determine the resource scheduling scheme corresponding to each scheduling level; the resource scheduling scheme corresponding to each scheduling level includes the hardware resources corresponding to each target pipeline in the scheduling level.

[0097] Step 602: Based on the serial dependency relationship, perform hardware resource mapping according to the resource scheduling scheme corresponding to the current scheduling level, so as to instruct the corresponding hardware resources to execute each target pipeline in the current scheduling level.

[0098] For example, when multiple scheduling levels are determined, and at least one target pipeline corresponding to each scheduling level, corresponding hardware resources can be allocated to each scheduling level. After the execution of the previous scheduling level is completed, the computer device can determine the resource scheduling scheme corresponding to the next scheduling level based on the current hardware resources. Alternatively, hardware resources can be allocated to each target pipeline in each scheduling level synchronously based on the resource load of each target pipeline in each scheduling level. The hardware resources corresponding to a target pipeline in the next scheduling level can be wholly or partially the same as the scheduling resources of another target pipeline in the previous scheduling level. The same hardware resource (such as chip 1) can execute a target pipeline in the next scheduling level after the execution of another target pipeline in the previous scheduling level is completed.

[0099] For example, different target pipelines in the same scheduling level can also correspond to the same hardware resource. The execution timing of the different target pipelines in the same scheduling level is different for the same hardware resource. For example, the same scheduling level includes a first target pipeline and a second target pipeline, wherein the node length of the first target pipeline is greater than the node length of the second target pipeline. When allocating hardware, the first target pipeline can be executed by core 1 and core 2, and the second target pipeline can be executed by core 2. However, the execution order can be: while core 1 executes the first target pipeline, core 2 executes the second target pipeline simultaneously. After core 2 completes the second target pipeline, it combines with core 1 to execute the first target pipeline.

[0100] Furthermore, once the resource scheduling scheme corresponding to each scheduling level or the current scheduling level is determined, hardware resource mapping can be performed based on the resource scheduling scheme corresponding to the current scheduling level to instruct the corresponding hardware resources to execute each target pipeline in the current scheduling level.

[0101] In this embodiment, pipeline construction and scheduling hierarchy construction are used to construct the directed acyclic graph of the neural network model in parallel pipeline and in serial hierarchy, thereby maximizing the use of the parallel structure in the DAG graph and allocating hardware resources efficiently and reasonably according to the parallel structure. This not only improves the utilization rate of hardware resources and reduces cavitation, but also improves the running efficiency of the model.

[0102] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0103] Based on the same inventive concept, this application also provides a topology-based data flow partitioning and scheduling apparatus for implementing the aforementioned topology-based data flow partitioning and scheduling method. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more topology-based data flow partitioning and scheduling apparatus embodiments provided below can be found in the limitations of the topology-based data flow partitioning and scheduling method described above, and will not be repeated here.

[0104] In one exemplary embodiment, such as Figure 7 As shown, a data flow partitioning and scheduling device 700 based on topology structure is provided, including: an acquisition module 701, a pipeline splitting module 702, and a hierarchy partitioning module 703, wherein:

[0105] The acquisition module 701 is used to acquire the initial DAG graph corresponding to the target neural network model.

[0106] The pipeline splitting module 702 is used to split the initial DAG graph into multiple target pipelines; the nodes in the target pipelines satisfy the serial data dependency condition.

[0107] The hierarchical partitioning module 703 is used to partition multiple target pipelines into hierarchical levels to obtain multiple scheduling levels with serial dependencies. The target pipelines within the same scheduling level meet the condition of parallelism without data dependencies, and each target pipeline within the same scheduling level occupies independent hardware resources.

[0108] In one embodiment, the pipeline splitting module 702 includes:

[0109] The first determining unit is used to determine at least one source node with an in-degree of zero in the initial DAG graph; the source node serves as the first node of the corresponding target pipeline.

[0110] The second determining unit is used to traverse each node layer by layer along the data flow direction of the initial DAG graph for each source node, and determine at least one target node corresponding to the source node that satisfies the first preset condition; the first preset condition includes that the successor node of the previous node of the target node is unique, and the predecessor node of the target node is also unique.

[0111] The generation unit is used to generate the corresponding target pipeline based on the source node and at least one corresponding target node.

[0112] In one embodiment, the device further includes:

[0113] The loop module is used to treat all successor nodes of the target node as new source nodes when the target node is a divergence point or the successor node of the target node is a convergence point; and return to the step of traversing each node layer by layer along the data flow direction of the initial DAG graph for each source node to determine at least one target node corresponding to the source node that meets the preset conditions.

[0114] In one embodiment, the hierarchy partitioning module 703 includes:

[0115] The third determining unit is used to determine at least one target pipeline from multiple target pipelines that meets the second preset condition; the second preset condition includes that all predecessor pipelines of the target pipeline are included in the prior scheduling hierarchy;

[0116] A hierarchical construction unit is used to construct a scheduling hierarchy corresponding to at least one target pipeline based on at least one target pipeline that satisfies a second preset condition.

[0117] In one embodiment, the third determining unit is specifically configured to: construct a set to be scheduled based on multiple target pipelines; determine the target pipelines corresponding to each source node with an in-degree of zero from the set to be scheduled; construct a first scheduling level based on the target pipelines corresponding to each source node, and delete the target pipelines corresponding to each source node in the set to be scheduled; determine at least one target pipeline that satisfies a second preset condition from the deleted set to be scheduled; and ensure that the target pipelines containing all predecessor nodes of the first node of the target pipeline that satisfies the second preset condition are not in the deleted set to be scheduled.

[0118] In one embodiment, the device further includes:

[0119] The scheduling scheme generation module is used to determine the resource scheduling scheme corresponding to each scheduling level for multiple scheduling levels with serial dependencies; the resource scheduling scheme corresponding to the scheduling level includes the hardware resources corresponding to each target pipeline in the scheduling level.

[0120] The resource mapping module is used to map hardware resources according to serial dependencies and based on the resource scheduling scheme corresponding to the current scheduling level, so as to instruct the corresponding hardware resources to execute each target pipeline in the current scheduling level.

[0121] Each module in the aforementioned topology-based data flow partitioning and scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0122] In one exemplary embodiment, a computer device is provided, taking a server as an example, and its internal structure diagram can be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores initial DAG graph data, pipeline sets P, scheduling sets R, and resource scheduling schemes. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a topology-based data flow partitioning and scheduling method.

[0123] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0124] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the topology-based data flow partitioning and scheduling method in any of the above embodiments.

[0125] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the topology-based data flow partitioning and scheduling method in any of the above embodiments.

[0126] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the topology-based data flow partitioning and scheduling method in any of the above embodiments.

[0127] It should be noted that the data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all information and data that have been fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0128] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data flow partitioning and scheduling method based on topology, characterized in that, The method includes: Obtain the initial DAG graph corresponding to the target neural network model; The initial DAG graph is decomposed into multiple target pipelines; the nodes in the target pipelines satisfy the condition of serial data dependency. The multiple target pipelines are hierarchically divided to obtain multiple scheduling levels with serial dependencies; the target pipelines within the same scheduling level satisfy the condition of parallelism without data dependency, and each target pipeline within the same scheduling level occupies independent hardware resources.

2. The method according to claim 1, characterized in that, The initial DAG graph is decomposed into multiple target pipelines, including: Identify at least one source node with an in-degree of zero in the initial DAG graph; the source node serves as the first node of the corresponding target pipeline. For each source node, each node is traversed layer by layer along the data flow direction of the initial DAG graph to determine at least one target node corresponding to the source node that satisfies the first preset condition; the first preset condition includes that the successor node of the previous node of the target node is unique, and the predecessor node of the target node is also unique. Based on the source node and at least one corresponding target node, a corresponding target pipeline is generated.

3. The method according to claim 2, characterized in that, The method further includes: If the target node is a diverging point, or if the successor node of the target node is a converging point, then all successor nodes of the target node are taken as new source nodes. Return to the step of traversing each node layer by layer along the data flow direction of the initial DAG graph for each of the source nodes, and determining at least one target node corresponding to the source node that satisfies the preset conditions.

4. The method according to claim 1, characterized in that, The hierarchical division of the multiple target pipelines to obtain multiple scheduling levels with serial dependencies includes: At least one target pipeline that meets a second preset condition is determined from the plurality of target pipelines; the second preset condition includes that all predecessor pipelines of the target pipeline are included in the prior scheduling hierarchy; Based on at least one target pipeline that satisfies the second preset condition, a scheduling level corresponding to the at least one target pipeline is constructed.

5. The method according to claim 4, characterized in that, The step of determining at least one target pipeline that satisfies the second preset condition from the plurality of target pipelines includes: A set to be scheduled is constructed based on the multiple target pipelines; Determine the target pipeline corresponding to each source node with an in-degree of zero from the set to be scheduled; A first scheduling level is constructed based on the target pipelines corresponding to each source node, and the target pipelines corresponding to each source node in the set to be scheduled are deleted. From the deleted set of pipelines to be scheduled, at least one target pipeline that meets the second preset condition is determined; the target pipelines whose first node contains all the predecessor nodes of the target pipeline that meets the second preset condition are not in the deleted set of pipelines to be scheduled.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: For the multiple scheduling levels with serial dependencies, a resource scheduling scheme is determined for each scheduling level; the resource scheduling scheme for each scheduling level includes the hardware resources corresponding to each target pipeline in the scheduling level. According to the serial dependency relationship, hardware resources are mapped based on the resource scheduling scheme corresponding to the current scheduling level, so as to instruct the corresponding hardware resources to execute each of the target pipelines in the current scheduling level.

7. A data flow partitioning and scheduling device based on topology, characterized in that, The device includes: The acquisition module is used to acquire the initial DAG graph corresponding to the target neural network model; The pipeline splitting module is used to split the initial DAG graph into multiple target pipelines; the nodes in the target pipelines satisfy the serial data dependency condition. The hierarchical partitioning module is used to partition the multiple target pipelines into multiple scheduling levels with serial dependencies; the target pipelines within the same scheduling level satisfy the condition of parallelism without data dependency, and each target pipeline within the same scheduling level occupies independent hardware resources.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images