Directed acyclic graph splitting method and device, electronic equipment and readable storage medium

Abstract

The invention provides a directed acyclic graph splitting method and device, electronic equipment and a readable storage medium, and relates to the field of computers.The method comprises the steps that a target node is selected from a directed acyclic graph every time on the basis of a depth-first traversal principle, and first alternative nodes needing to be passed by for constructing the target node are determined, and then determining a second alternative node through which a root node for constructing the sub-tree needs to pass from the first alternative nodes, and further dividing the second alternative node into the sub-tree. According to the method, the complete directed acyclic graph is split into the plurality of sub-trees without repeated nodes, and when the project implemented according to the directed acyclic graph is constructed, the project construction speed can be increased, and the project construction time can be shortened.

Description

Technical Field

[0001] The present application relates to the computer field, and more specifically, to a directed acyclic graph splitting method, device, electronic device and computer-readable storage medium in the computer field. Background Art

[0002] The Ninja (A build system developed by Google) build system has been used in the build process of many large software projects because of its fast, efficient and convenient features. When executing a build task, the Ninja build system will first load the build manifest file, typically a build.ninja file. This file uses a specific format to store all the build steps and dependencies. When the Ninja build system reads and parses the manifest file and stores it in memory, it represents the entire build process as a Plan, each build step as an Edge, and the file generated by each step is a Target. All the steps are connected together to form a DAG (Directed acyclic diagram), thereby realizing the project construction.

[0003] Currently, for large projects such as operating systems and functional platforms, due to their extremely large source code scale, implementing a single-machine complete build is already a very time-consuming and laborious task. The Ninja build system was designed to run as a single machine, single process, and single instance. It cannot support multi-process and distributed build methods. For example, for the AOSP project, which has more than 100,000 Edge edges, it takes nearly 50 minutes to fully build it even on a single machine with up to 32G memory and 48-core CPU. It can be seen that it is time-consuming to build software projects with extremely large source code scales through the Ninja build system. Summary of the invention

[0004] The present application provides a directed acyclic graph splitting method, device, electronic device and computer-readable storage medium, which can reduce the time it takes for a Ninja build system to build a software project with a very large source code scale.

[0005] In a first aspect, a method for splitting a directed acyclic graph is provided, and the directed acyclic graph splitting method comprises:

[0006] Based on the depth-first traversal principle, select one target node at a time from the directed acyclic graph;

[0007] Determine the first candidate node that needs to be passed through to build the above target node;

[0008] Determine a second candidate node from the first candidate node, where the second candidate node is a node that needs to be passed through to construct the root node of the subtree;

[0009] The second candidate node is divided into the subtrees, and nodes between different subtrees are not repeated.

[0010] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the step of determining the second candidate node from the above-mentioned first candidate node includes: determining the node number of the above-mentioned first candidate node according to the number of annotations of the above-mentioned target node; determining the comparison relationship between the above-mentioned node number and the expected interval; and determining the above-mentioned second candidate node from the above-mentioned first candidate node according to the above-mentioned comparison relationship.

[0011] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the above-mentioned first candidate node includes the above-mentioned target node and each first child node corresponding to the above-mentioned target node, and the above-mentioned second candidate node includes the above-mentioned root node and the second child node corresponding to the above-mentioned root node; the above-mentioned step of determining the above-mentioned second candidate node from the above-mentioned first candidate node according to the above-mentioned comparison relationship includes: when the above-mentioned comparison relationship is that the number of the above-mentioned nodes is within the expected range, the above-mentioned target node is determined as the above-mentioned root node, and the above-mentioned each first child node is determined as the above-mentioned second child node.

[0012] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the above-mentioned first candidate node includes the above-mentioned target node and each first child node corresponding to the above-mentioned target node, and the above-mentioned second candidate node includes the above-mentioned root node and the second child node corresponding to the above-mentioned root node; the above-mentioned step of determining the above-mentioned second candidate node from the above-mentioned first candidate node according to the above-mentioned comparison relationship includes: when the above-mentioned comparison relationship is that the above-mentioned number of nodes is greater than the maximum value of the above-mentioned expected interval, obtaining the number of annotations carried by each of the above-mentioned first child nodes; when the number of annotations carried by the first child node is within the above-mentioned expected interval, based on the above-mentioned depth-first traversal principle, the first child node with the number of annotations carried within the above-mentioned expected interval is used as the above-mentioned target node, and the step of determining the node number of the above-mentioned first candidate node according to the number of annotations of the above-mentioned target node is executed, so as to select the above-mentioned root node and the above-mentioned second child node from the above-mentioned target node and each first child node corresponding to the above-mentioned target node.

[0013] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, after the step of obtaining the number of annotations carried by each of the above-mentioned first child nodes when the above-mentioned comparison relationship is that the number of nodes is greater than the maximum value of the above-mentioned expected interval, the above-mentioned directed acyclic graph splitting method also includes: when the number of annotations carried by each of the above-mentioned first child nodes is less than the minimum value of the above-mentioned expected interval, determining the above-mentioned target node as the above-mentioned root node, and determining each of the above-mentioned first child nodes as the above-mentioned second child nodes.

[0014] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, before the above-mentioned step of selecting a target node from the directed acyclic graph each time based on the depth-first traversal principle, the above-mentioned directed acyclic graph splitting method also includes: obtaining the total number of first child nodes required to pass through to construct the above-mentioned target node; and using the above-mentioned total number as the label of the above-mentioned target node.

[0015] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, after the step of dividing the above-mentioned second alternative node into the above-mentioned subtrees, the above-mentioned directed acyclic graph splitting method also includes: obtaining the task corresponding to each of the above-mentioned subtrees; determining the execution order of the tasks corresponding to each of the above-mentioned subtrees based on the dependency relationship between different above-mentioned subtrees; scheduling the target execution object to execute the tasks corresponding to each of the above-mentioned subtrees in sequence according to the above-mentioned execution order; wherein the above-mentioned target execution object is a virtual machine in an idle state among multiple virtual machines; when the tasks corresponding to the above-mentioned subtrees are executed, a project constructed according to the above-mentioned directed acyclic graph is obtained.

[0016] In a second aspect, a directed acyclic graph splitting device is provided, and the directed acyclic graph splitting device comprises:

[0017] A selection module is used to select a target node from the directed acyclic graph one at a time based on the depth-first traversal principle;

[0018] A first determination module is used to determine a first candidate node that needs to be passed through to build the above target node;

[0019] A second determination module is used to determine a second candidate node from the first candidate node, where the second candidate node is a node that needs to be passed through to construct the root node of the subtree;

[0020] The splitting module is used to divide the second candidate node into the subtrees, and the nodes between different subtrees are not repeated.

[0021] In conjunction with the second aspect, in some possible implementations, the second determining module includes:

[0022] A node quantity acquisition unit, used to determine the node quantity of the first candidate node according to the labeled quantity of the target node;

[0023] A data comparison unit, used to determine the comparison relationship between the above node quantity and the expected interval;

[0024] The candidate node selection unit is used to determine the second candidate node from the first candidate node according to the comparison relationship.

[0025] In combination with the second aspect and the above implementation, in some possible implementations, the first candidate node includes the target node and each first child node corresponding to the target node, the second candidate node includes the root node and the second child node corresponding to the root node, and the candidate node selection unit includes:

[0026] The first node selection subunit is used to determine the target node as the root node and determine each of the first child nodes as the second child node when the comparison relationship is that the number of nodes is within the expected range.

[0027] In combination with the second aspect and the above implementation, in some possible implementations, the first candidate node includes the target node and each first child node corresponding to the target node, the second candidate node includes the root node and the second child node corresponding to the root node, and the candidate node selection unit further includes:

[0028] The annotation quantity acquisition subunit is used to acquire the annotation quantity carried by each of the first subnodes when the comparison relationship is that the number of nodes is greater than the maximum value of the expected interval;

[0029] The second node selection sub-unit is used to, when the number of annotations carried by the first child node is within the above-mentioned expected interval, based on the above-mentioned depth-first traversal principle, take the first child node whose number of annotations carried is within the above-mentioned expected interval as the above-mentioned target node, and execute the step of determining the node number of the above-mentioned first candidate node according to the number of annotations of the above-mentioned target node, so as to select the above-mentioned root node and the above-mentioned second child node from the above-mentioned target node and each first child node corresponding to the above-mentioned target node.

[0030] In combination with the second aspect and the above implementation, in some possible implementations, the candidate node selection unit further includes:

[0031] The third node selection subunit is used to determine the target node as the root node and determine the first subnodes as the second subnodes when the number of annotations carried by each of the first subnodes is less than the minimum value of the expected interval.

[0032] In conjunction with the second aspect, in some possible implementations, the directed acyclic graph splitting device further includes:

[0033] A quantity counting unit, used to obtain the total number of first child nodes required to construct the above target node;

[0034] The labeling unit is used to use the total number as a label of the target node.

[0035] In conjunction with the second aspect, in some possible implementations, the directed acyclic graph splitting device further includes:

[0036] A task acquisition unit, used to acquire the task corresponding to each of the above subtrees;

[0037] An order determination unit, used to determine the execution order of the tasks corresponding to each of the above subtrees based on the dependency relationship between the different above subtrees;

[0038] A target scheduling unit, used for scheduling the target execution object to execute the tasks corresponding to each of the above subtrees in sequence according to the above execution order; wherein the above target execution object is a virtual machine in an idle state among the multiple virtual machines;

[0039] The project construction unit is used to obtain a project constructed according to the directed acyclic graph when the tasks corresponding to the subtrees are completed.

[0040] In a third aspect, an electronic device is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the method in the first aspect or any possible implementation of the first aspect.

[0041] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0042] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0043] The directed acyclic graph splitting method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application have the following technical effects:

[0044] The present application adopts a technical solution of selecting one target node each time from a directed acyclic graph based on the depth-first traversal principle, determining the first candidate node that needs to be passed through to construct the target node, then determining the second candidate node that needs to be passed through to construct the root node of the subtree from the first candidate node, and then dividing the second candidate node into subtrees. This realizes splitting the complete DAG into multiple subtrees without duplicate nodes, and can obtain subtrees without dependencies and subtrees with dependencies. When constructing a project implemented based on DAG, tasks corresponding to subtrees without dependencies can be executed in parallel. After the tasks corresponding to the subtrees without dependencies are completed, the tasks corresponding to the subtrees with dependencies are executed, which is conducive to improving the project construction speed and reducing the project construction time. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 An exemplary schematic diagram of a DAG is shown;

[0046] Figure 2 An exemplary subtree partitioning diagram of DAG splitting is shown;

[0047] Figure 3 A schematic flow chart of a directed acyclic graph splitting method provided in an embodiment of the present application is shown;

[0048] Figure 4 A partial schematic diagram of a subtree split from the DAG of the AOSP project is shown;

[0049] Figure 5 A schematic diagram of labeling each node in a DAG is shown;

[0050] Figure 6 An exemplary splitting diagram of a subtree is shown;

[0051] Figure 7-1 Another exemplary splitting diagram of a subtree is shown;

[0052] Figure 7-2 Another exemplary splitting diagram of a subtree is shown;

[0053] Figure 8 Another exemplary splitting diagram of a subtree is shown;

[0054] Fig. 9 It is a structural schematic diagram of a directed acyclic graph splitting device provided in an embodiment of the present application;

[0055] Fig.10 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0057] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0058] When the Ninja build system reads and parses the manifest file and stores it in memory, it represents the entire build process as a Plan, each build step as an Edge, and the file generated by each step as a Target. All the steps are connected together to form a DAG, thereby realizing the project build. Each source file is a Leaf starting point or leaf node of this DAG, and the final generated Final target is the end point of the entire DAG, which can also be understood as the root node of the entire DAG. The entire DAG construction process is a sequence that starts from each Leaf, moves forward step by step along the directed edges of the DAG, and finally reaches the Final target. In the following, Leaf is called the leaf node of DAG, and Final target is the root node of DAG.

[0059] like Figure 1 As shown, Figure 1 An exemplary schematic diagram of a DAG is shown. Figure 1 A, B, C, and D are each a Target. A specifically refers to the root node of DAG, B, C, D, E, F, G, H, and I can be understood as child nodes of A, E, F, G, H, and I can be understood as leaf nodes of DAG, and B, C, and D can be understood as intermediate nodes of DAG. Each edge with an arrow is an Edge (or several edges with arrows together form an Edge). When building DAG, it is executed from bottom to top according to the dependency relationship until the final result A is generated. The construction of A needs to go through A, B, C, D, E, F, G, H, and I, the construction of B needs to go through B, E, and F, the construction of C needs to go through C, G, and H, and the construction of D needs to go through D and I.

[0060] For large projects such as operating systems and functional platforms, due to their large source code scale, implementing a single-machine complete build is already a very time-consuming and laborious task. The Ninja build system was designed to run as a single machine, single process, and single instance. It cannot support multi-process and distributed build methods. For example, the AOSP project has more than 100,000 Edges. Even on a single machine with up to 32G memory and 48-core CPU, it takes nearly 50 minutes to fully build it. It can be seen that it is time-consuming to build software projects with large source code scales through the Ninja build system. However, due to the existing high efficiency of the Ninja build system, the Ninja build system will be used for projects like AOSP for a long time as the underlying build system. In this case, in order to further improve the project build speed, while retaining the original fast and efficient features of the Ninja build system, a multi-machine (multiple Workers) collaborative distributed build mechanism is introduced to the Ninja build system to improve the build speed of software projects with large source code scales. Among them, Worker can be understood as a virtual machine.

[0061] Introducing a multi-worker collaborative distributed construction mechanism into the Ninja build system and realizing multi-machine collaborative distributed construction requires a task allocation method that can reasonably, optimally, efficiently and relatively evenly distribute all the edges in the DAG to different workers, so that each worker can cooperate with each other and execute different parts of the DAG respectively. Each part is called a Child tree (which can be understood as a subtree), that is, the entire DAG is split into several subtrees, and different subtrees are assigned a Worker. Each Worker cooperates with each other to execute the right subtree, thereby completing the construction of the project built according to the DAG. This method can effectively improve the construction speed and efficiency of large projects. Therefore, the present application provides a directed acyclic graph splitting method, device, electronic device and computer-readable storage medium to solve the problem that it is time-consuming to build a software project with a very large source code scale through the Ninja build system.

[0062] The following is an embodiment of a directed acyclic graph splitting method provided in an embodiment of the present application.

[0063] Split the entire DAG into several subtrees. The splitting algorithm adopted must meet the following conditions:

[0064] Condition 1: The split result of the entire DAG is non-repetitive, non-omitted, and restorable. That is, there are no repeated edges between the split subtrees (note: some targets will have intersections because they serve as both input and output), and all subtrees can be re-merged and spliced ​​into a complete DAG (that is, each edge must be in a unique subtree).

[0065] Condition 2: Each subtree is an independent construction unit, which can be divided into two cases: Case 1, if the subtree is at the bottom layer of the DAG (that is, the layer closest to the leaf node), then it can be independently constructed at any time; Case 2, if the subtree is not at the bottom layer of the DAG, that is, it is built on a lower subtree, then the subtree can be independently constructed if and only if the lower-level subtrees it depends on have been constructed.

[0066] Condition 3: The dependency between subtrees is clear. If tree A depends on tree B, tree B must be generated first, and its generation result or part of its generation result will be used as the input of tree A. This requires that each subtree must be able to accurately list all its inputs, its dependencies on all lower-level subtrees, and the dependencies that can be provided by all higher-level subtrees. The dependency between subtrees is unidirectional and transitive, that is, if tree A depends on tree B, then tree B must not depend on tree A; if tree A depends on tree B, and tree B depends on tree C, then tree A must depend on tree C.

[0067] Condition 4: Splitting does not change the entire DAG construction process, but only adjusts the construction order of some edges. In the case of no splitting, DAG construction is carried out completely according to the depth-first principle, layer by layer and branch by branch. After splitting into subtrees, since the division of subtrees is affected by different granularity parameters, it will not be exactly the same as the original hierarchical division, so the local construction order will be slightly adjusted. However, since all subtrees are merged into the original DAG, the entire construction process does not actually change, no new construction actions are added, and no existing construction actions are skipped.

[0068] like Figure 2 As shown, Figure 2 An exemplary subtree partitioning diagram of DAG splitting is shown. First, B and its subordinate E and F can be taken as the first subtree, called subtree 1; then C and its subordinate F, G, H can be taken as the second subtree, called subtree 2; finally, A and its subordinate B, C, D, I can be taken as the third subtree, called subtree 3. After splitting the DAG, three subtrees are obtained, namely subtree 1, subtree 2 and subtree 3. It can be seen that subtree 1 where B is located and subtree 2 where C is located are both bottom-level subtrees. Subtree 1 and subtree 2 can be directly constructed at any time, while subtree 3 where A is located needs to rely on subtree 1 and subtree 2. Subtree 3 can only be completed after subtree 1 and subtree 2 are completed. The three subtrees of tree 1, subtree 2 and subtree 3 are merged and connected to form the original complete DAG.

[0069] Figure 3 FIG. 1 is a schematic flow chart of a directed acyclic graph splitting method provided by an embodiment of the present application. Figure 3As shown, the directed acyclic graph splitting method provided in the embodiment of the present application is applied to the Ninja construction system, and the directed acyclic graph splitting method includes the following schemes:

[0070] S110: Based on the depth-first traversal principle, select one target node each time from the directed acyclic graph.

[0071] In an exemplary embodiment, the depth-first traversal principle means starting from the root node of the tree, traversing the left subtree first, and then traversing the right subtree. Figure 1 As shown, DAG can be understood as a whole tree. Starting from A, the traversal order is: ABEFCGHDI.

[0072] Based on the depth-first traversal principle, a target node is selected from the DAG each time, that is, the node currently traversed in the DAG is the target node. For example, if the currently traversed node is A, A is the target node; if the currently traversed node is B, B is the target node; if the currently traversed node is C, C is the target node.

[0073] S120: Determine the first candidate node that needs to be passed through to construct the above target node.

[0074] After each target node is selected from the DAG, the first candidate node that needs to be passed through to build the target node is determined from the DAG. The first candidate node includes the target node itself and all child nodes under the target node. Each child node under the target node is called the first child node. For example, A is the target node, and the child nodes under A are B, C, D, E, F, G, H, and I. Then the first candidate nodes include A, B, C, D, E, F, G, H, and I. For another example, B is the target node, and the child nodes under B are E and F. Then the first candidate nodes include A, E, and F.

[0075] S130: Determine a second candidate node from the first candidate nodes, where the second candidate node is a node that needs to be passed through to construct the root node of the subtree.

[0076] After determining the first candidate node that needs to be passed through to construct the selected target node from the DAG, determine the second candidate node from the first candidate node. The second candidate node is the node that needs to be passed through to construct the root node of the subtree. The second candidate node includes the root node and all child nodes under the root node. Each child node under the root node is called a second child node. The root node and all child nodes under the root node constitute the subtree, that is, the second candidate node constitutes the subtree.

[0077] How to determine the second candidate node from the first candidate node can be determined by judging whether the first candidate node is suitable for constituting a subtree with a suitable size.

[0078] If the size of the subtree formed by the first candidate node is appropriate, the first candidate node is used as the second candidate node, that is, it is determined that a subtree can be formed by the first candidate node, that is, the target node in the first candidate node is used as the root node of the subtree, and all child nodes under the target node are used as all child nodes under the root node.

[0079] If the subtree formed by the first candidate node is larger, a second candidate node is selected from all the child nodes under the target node included in the first candidate node, and it is determined whether the size of the subtree formed by the selected second candidate node is appropriate. If the size of the subtree formed by the selected second candidate node is appropriate, it is determined that a subtree can be formed by the selected second candidate node; if the subtree formed by the selected second candidate node is larger, the selected second candidate node is used as the first candidate node, and the above process is repeated if the subtree formed by the first candidate node is larger, then the second candidate node is selected from all the child nodes under the target node included in the first candidate node, and it is determined whether the size of the subtree formed by the selected second candidate node is appropriate, until the second candidate node used to form a subtree of appropriate size is determined.

[0080] If the subtree formed by the first candidate node is smaller, the first candidate node is used as the second candidate node, the target node in the first candidate node is used as the root node of the subtree, and all child nodes under the target node are used as all child nodes under the root node.

[0081] In a possible implementation, before the step of selecting a target node from the directed acyclic graph each time based on the depth-first traversal principle, the directed acyclic graph splitting method further includes the following scheme:

[0082] Get the total number of first child nodes that need to be passed to build the above target node;

[0083] The above total number is used as the label of the above target node.

[0084] For large projects such as AOSP, the number of Targets and Edges is in the hundreds of thousands, and the dependency relationships within the DAG are extremely complex. Conventional DAG splitting algorithms are either difficult to complete or the complexity and execution efficiency are unbearable. Figure 4 As shown, Figure 4 A partial schematic diagram of a subtree split from the DAG of the AOSP project is shown. Figure 4 It can be seen that the Edge connections between the Target targets in this super large DAG are like a tangled mess.

[0085] In order to solve this problem of ultra-large-scale and ultra-complex DAG splitting, this application is closely integrated with the memory construction process of the DAG inside the Ninja construction system. Considering that, except for leaf nodes, each Target target must complete the construction of its Child target sub-target in advance before construction, that is, before each node is built in the DAG, the construction of the sub-nodes under the node needs to be completed. Therefore, the cumulative number of Target targets can be used as the basis for splitting. The cumulative number is the total number of all sub-nodes under each node and the node, and the total number = the number of nodes + the number of all sub-nodes under the node. The first step of splitting is to first record the total number of all sub-targets that each Target target needs to pass through when building, which can also be understood as recording the total number of all sub-nodes that the node needs to pass through when building. It should be noted that the Target target cannot be counted repeatedly, because some Target targets will participate in many Edges. After the DAG is split into multiple sub-trees, there are no duplicate nodes between different sub-trees.

[0086] From the bottom to the top of the DAG, that is, from the bottom to the top, each time a node is selected and regarded as the target node, the total number of child nodes that need to be passed to build the target node is counted, and the child node that needs to be passed to build the target node is represented as the first child node. Then, the total number of first child nodes that need to be passed to build the target node is used as the label of the target node. The label is specifically the value of the total number of first child nodes that need to be passed to build the target node.

[0087] like Figure 5 As shown, Figure 5 A schematic diagram of labeling each node in the DAG is shown.

[0088] Assume that the nodes E, F, G, H, and I in the bottom layer of DAG each have leaf nodes under them. Therefore, when E, F, G, H, and I are respectively used as target nodes, the total number of first child nodes that need to be passed through to build E is 1, the total number of first child nodes that need to be passed through to build F is 1, the total number of first child nodes that need to be passed through to build G is 1, the total number of first child nodes that need to be passed through to build H is 1, and the total number of first child nodes that need to be passed through to build I is 1, that is, 1 is the label of E, F, G, H, and I. Among them, the leaf nodes under E, F, G, H, and I are not included in the statistics.

[0089] When B is the target node, the child nodes under B are E and F. The total number of first child nodes that need to be passed through to build B is 3, and 3 is the label of B, that is, the first child node that needs to be passed through to build B includes B itself. When C is the target node, the child nodes under C are F, G, and H. Among them, F has been included in the statistics of the first child node that needs to be passed through to build B, so no more statistics are performed under C. The total number of first child nodes that need to be passed through to build C is 3, and 3 is the label of C, that is, the first child node that needs to be passed through to build C includes C itself. When D is the target node, the child node under D is I, and the total number of first child nodes that need to be passed through to build D is 2, and 2 is the label of D, that is, the first child node that needs to be passed through to build D includes D itself. When A is the target node, the child nodes under A are B, C, D, E, F, G, H and I. The total number of first child nodes that need to be passed to construct A is 9, 3+3+2+1=9, 9 is the label of A, that is, the first child node that needs to be passed to construct A includes A itself.

[0090] After marking the total number of first child nodes that need to be passed through to build each target node in the DAG, the DAG is split into subtrees according to the number of annotations carried by each target node. By marking the total number of first child nodes that need to be passed through to build each target node in the DAG, it is helpful to speed up the subtree splitting of the DAG.

[0091] The construction and traversal of the internal DAG of the Ninja build system all use the depth-first traversal principle. The subtree splitting also adopts this principle, starting from the root node of the DAG, recursively traversing each child node in turn, and determining when to split the subtree based on the number of annotations carried by each node.

[0092] The subtree splitting is realized based on the splitting parameters. The main splitting parameter is the granularity, that is, the average size of the split subtree. Choosing the right granularity helps to achieve the best distributed construction effect. The smaller the granularity, the smaller the size of each subtree, which will help to improve the parallelism, but the dependency between subtrees will also increase, the amount of data exchange will increase accordingly, and the complexity of scheduling execution will also increase; the larger the granularity, the less dependency between subtrees, the less data exchange, and the complexity of execution scheduling, but the parallelism will decrease, and it is easy to cause uneven load on each execution node, reducing the efficiency of distributed construction.

[0093] In the specific setting of granularity, the expected interval is adopted, and the maximum and minimum values ​​of a subtree size are set. Whenever the number of annotations carried by the root node of a branch falls within the expected interval, it can be split into an independent subtree. It should be noted that not all subtree sizes will necessarily fall within this interval. In the AOSP project, there will be some very large Edges, whose directly input Target target number may reach thousands. No matter how the subtree below the root node is split, the subtree where the root node is located will easily exceed the expected interval (unless the upper limit of the expected interval is set very large, exceeding the directly input Target target number). Among them, the expected interval can be flexibly set according to the actual project needs, and different expected areas correspond to different subtree splitting results.

[0094] In a possible implementation, the determining of the second candidate node from the first candidate node includes the following scheme:

[0095] Determine the number of nodes of the first candidate node according to the number of annotations of the target node;

[0096] Determine the comparison relationship between the above number of nodes and the expected interval;

[0097] The second candidate node is determined from the first candidate nodes according to the comparison relationship.

[0098] Based on the depth-first traversal principle, each time a target node is selected from the directed acyclic graph, the number of annotations carried by the target node can be obtained. The number of annotations carried by the target node is the number of nodes of the first candidate node. Since the first candidate node includes the target node and the first child nodes corresponding to the target node, the sum of the number of target nodes and the number of first child nodes corresponding to the target node is the number of nodes of the first candidate node.

[0099] After obtaining the number of nodes of the first candidate node, the number of nodes of the first candidate node is compared with the expected interval to obtain a comparison relationship between the two. Different comparison relationships can reflect whether the subtree formed by the first candidate node is appropriate, large, or small. The comparison relationship includes: the number of nodes is within the expected interval, the number of nodes is greater than the maximum value of the expected interval, and the number of nodes is less than the minimum value of the expected interval. The number of nodes within the expected interval can be used to indicate that the size of the subtree formed by the first candidate node is appropriate, the number of nodes is greater than the maximum value of the expected interval, which can be used to indicate that the subtree formed by the first candidate node is large, and the number of nodes is less than the minimum value of the expected interval, which can be used to indicate that the subtree formed by the first candidate node is small.

[0100] If the comparison relationship determines that the subtree formed by the first candidate node is appropriate or the subtree formed by the first candidate node is small, it can be seen that the first candidate node is used as the second candidate node, and the subtree is formed by the second candidate node. If the comparison relationship determines that the subtree formed by the first candidate node is large, it is necessary to select the second candidate node that can form a subtree of appropriate size from each first child node under the target node included in the first candidate node. By determining the second candidate node used to form a subtree of appropriate size from the first candidate node based on the comparison relationship between the number of nodes of the first candidate node and the expected interval, it is beneficial to improve the efficiency of subtree splitting.

[0101] In a possible implementation, the first candidate node includes the target node and each first child node corresponding to the target node, and the second candidate node includes the root node and the second child node corresponding to the root node. The second candidate node is determined from the first candidate node according to the comparison relationship. The following schemes are included:

[0102] When the comparison relationship shows that the number of nodes is within an expected range, the target node is determined as the root node, and each of the first child nodes is determined as the second child node.

[0103] If the comparison relationship between the number of nodes of the first alternative node and the expected interval is that the number of nodes is within the expected interval, indicating that the size of the subtree formed by the first alternative node is appropriate, the first alternative node is used as the second alternative node, that is, the target node included in the first alternative node is determined as the root node of the subtree, and each first child node under the target node included in the first alternative node is determined as the second child node of the root node of the subtree, and the second child node is the leaf node of the subtree.

[0104] like Figure 6 As shown, Figure 6 An exemplary splitting diagram of a subtree is shown. Figure 6 The first candidate node includes nodes A, B and C, A is the target node, B and C are the first child nodes of A. Among them, the number of annotations carried by A is within the expected interval, that is, the number of nodes of the first candidate node is within the expected interval, then A is the root node of the subtree, B and C are the second child nodes of the root node, that is, the leaf nodes of the subtree, and A, B and C can be divided into a subtree later.

[0105] In a possible implementation, the first candidate node includes the target node and each first child node corresponding to the target node, and the second candidate node includes the root node and the second child node corresponding to the root node. The second candidate node is determined from the first candidate node according to the comparison relationship. The following schemes are included:

[0106] When the comparison relationship is that the number of nodes is greater than the maximum value of the expected interval, obtaining the number of annotations carried by each of the first child nodes;

[0107] When the number of annotations carried by the first child node is within the above-mentioned expected interval, based on the above-mentioned depth-first traversal principle, the first child node whose number of annotations carried is within the above-mentioned expected interval is used as the above-mentioned target node, and the step of determining the node number of the above-mentioned first candidate node according to the number of annotations of the above-mentioned target node is executed to select the above-mentioned root node and the above-mentioned second child node from the above-mentioned target node and each first child node corresponding to the above-mentioned target node.

[0108] If the comparison relationship between the number of nodes of the first candidate node and the expected interval is that the number of nodes is greater than the maximum value of the expected interval, it means that the subtree formed by the first candidate node is larger, and the second candidate node needs to be selected from the first child nodes under the target node included in the first candidate node. Then, the number of annotations carried by each first child node under the target node included in the first candidate node is obtained. If the number of annotations carried by the first child node is within the expected interval, the first child node with the number of annotations carried within the expected interval is used as the target node, and then the step of determining the number of nodes of the first candidate node according to the number of annotations of the target node is returned to execute. After the first child node with the number of annotations carried within the expected interval is used as the target node, if the number of nodes of the first candidate node required to construct the current target node is within the expected interval, the root node of the subtree and the second child node under the root node are selected from the current target node and the first child nodes under the current target node.

[0109] like Figure 7-1 As shown, Figure 7-1 Another exemplary splitting diagram of a subtree is shown. Figure 7-1 The first candidate node includes nodes A, B, B1, B2 and C, A is the target node, B, B1, B2 and C are the first child nodes of A. Among them, the number of annotations carried by A is greater than the maximum value of the expected interval, that is, the number of nodes of the first candidate node is greater than the maximum value of the expected interval, then the number of annotations carried by B and C is obtained, the number of annotations carried by B is within the expected interval, and the number of annotations carried by C is not within the expected interval. Based on the depth-first traversal principle, B is taken as the target node, that is, the first candidate nodes that need to be passed through to construct B include B, B1 and B2, and then B is taken as the root node of the subtree, B1 and B2 are taken as the second child nodes under the root node, and B1 and B2 are specifically the leaf nodes of the subtree.

[0110] like Figure 7-2 As shown, Figure 7-2 Another exemplary splitting diagram of a subtree is shown. Figure 7-2The first candidate node includes nodes A, B, C, C1 and C2, A is the target node, B, C, C1 and C2 are the first child nodes of A. Among them, the number of annotations carried by A is greater than the maximum value of the expected interval, that is, the number of nodes of the first candidate node is greater than the maximum value of the expected interval, then the number of annotations carried by B and C is obtained, the number of annotations carried by B is not within the expected interval, and the number of annotations carried by C is within the expected interval. Based on the depth-first traversal principle, C is taken as the target node, that is, the first candidate nodes that need to be passed through to construct C include C, C1 and C2, and then C is taken as the root node of the subtree, C1 and C2 are taken as the second child nodes under the root node, and C1 and C2 are specifically the leaf nodes of the subtree.

[0111] In a possible implementation, after the step of obtaining the number of annotations carried by each of the first child nodes when the comparison relationship is that the number of nodes is greater than the maximum value of the expected interval, the directed acyclic graph splitting method further includes the following scheme:

[0112] When the number of annotations carried by each of the above-mentioned first child nodes is smaller than the minimum value of the above-mentioned expected interval, the above-mentioned target node is determined as the above-mentioned root node, and each of the above-mentioned first child nodes is determined as the above-mentioned second child nodes.

[0113] When the comparison relationship between the number of nodes of the first candidate node and the expected interval is that the number of nodes is greater than the maximum value of the expected interval, if the number of annotations carried by each first child node under the target node included in the first candidate node is less than the minimum value of the expected interval, it means that if the second candidate node is selected from each first child node under the target node included in the first candidate node, the subtree formed by the second candidate node is smaller, then it is not necessary to select the second candidate node from each first child node under the target node included in the first candidate node, but each first child node of the target node included in the first candidate node is the root node of the subtree, and each first child node under the target node is determined as the second child node under the root node. Among them, the second child node under the root node includes the intermediate node and leaf node of the subtree.

[0114] like Figure 8 As shown, Figure 8 Another exemplary splitting diagram of a subtree is shown. Figure 8The first candidate node includes nodes A, B, C, B1 and C1, A is the target node, B, C, B1 and C1 are the first child nodes of A. Among them, the number of annotations carried by A is greater than the maximum value of the expected interval, that is, the number of nodes of the first candidate node is greater than the maximum value of the expected interval, then the number of annotations carried by B, C, B1 and C1 is obtained, and the number of annotations carried by B, C, B1 and C1 is less than the minimum value of the expected interval, then A is used as the root node of the subtree, B, C, B1 and C1 are used as the second child nodes under the root node, B and C are specifically the middle nodes of the subtree, and B1 and C1 are specifically the leaf nodes of the subtree.

[0115] S140: Divide the second candidate node into the subtrees, and nodes between different subtrees are not repeated.

[0116] Through the above-mentioned different selection methods for the first candidate node, a different second candidate node can be obtained each time the traversal is performed, and the second candidate node obtained each time is divided into a subtree, that is, the root node included in the second candidate node and the second child node under the root node constitute a subtree, so that multiple subtrees can be obtained, thereby realizing the splitting of the complete DAG into multiple subtrees. Among them, for the splitting of the DAG, each time a subtree is split out, the number of annotations of the parent nodes at all levels of the subtree needs to be reduced accordingly to ensure the correctness of the splitting of the upper subtree of the split subtree, so as to complete the entire recursive splitting algorithm. For example, Figure 7-1 In the example, the number of annotations of B1 is 1, the number of annotations of B2 is 1, the number of annotations of B is 3, the number of annotations of C is 1, and the number of annotations of A is 5. After B, B1, and B2 are divided into a subtree, the number of annotations of B is updated to 1, the number of annotations of C remains 1, and the number of annotations of A is updated to 3.

[0117] After the complete DAG is completely split into multiple subtrees, the status of each subtree can be unexecuted, executing, or completed during actual execution. The real-time status of all subtrees is shared among all instances of the Ninja build system. When any instance is idle, it searches the subtree list to find the next subtree that is unexecuted and all dependencies are ready, marks its status as executing, and then starts the build process with the root node of the subtree as the build target. When all subtrees have completed execution, the entire DAG is built.

[0118] After the complete DAG is split into multiple subtrees, a subtree without dependencies can be obtained, and a subtree with dependencies can also be obtained. When building a project based on the data structure of DAG, the tasks corresponding to the subtrees without dependencies can be executed in parallel. After the tasks corresponding to the subtrees without dependencies are completed, the tasks corresponding to the subtrees with dependencies are executed, which is conducive to improving the project construction speed and reducing the project construction time. The directed acyclic graph splitting method provided by this application is a key step for the Ninja construction system to transform distributed construction. It is also a leap from the single-machine single-process single-instance construction mode of the Ninja construction system to the multi-machine multi-process multi-instance construction mode, which lays a solid and reliable foundation for realizing distributed construction.

[0119] The following is a data evaluation of the directed acyclic graph splitting method provided in this application.

[0120] Taking AOSP 13 as an example to evaluate the effect of the directed acyclic graph splitting method, the complete DAG of AOSP 13 version has 155205 build targets and 108649 edges. When different expected intervals are used as splitting parameters, the analysis results are shown in Table 1:

[0121] Table 1

[0122]

[0123] In Table 1, subtree parallelism refers to the average value of how many subtrees are allowed to be executed in parallel at most during the entire construction period. It can be seen that the finer the splitting granularity, the more subtrees are split out, and the more subtrees can be executed in parallel at the same time.

[0124] The shortest execution sequence refers to the number of edges that can complete the entire construction process at the fastest speed based on the maximum parallelism. Compared with the 108,649 edges of normal execution, it can be seen that the finer the splitting granularity and the more subtrees there are, the fewer edges are required for the fastest execution.

[0125] The number of exchange files refers to the interdependence between subtrees. The number of intermediate files that need to be passed from one subtree to another subtree. It can be seen that the finer the split granularity and the more subtrees there are, the more files need to be exchanged for the build.

[0126] From the numbers in Table 1, we can see that the split granularity is neither the finer the better nor the coarser the better. The final split granularity should be determined comprehensively based on the build needs and build environment. However, from the perspective of the shortest execution sequence, it is entirely possible to reduce the time consumption of the entire build process as long as the appropriate split granularity is selected.

[0127] In summary, the directed acyclic graph splitting method provided in this application is an industrial-grade splitting algorithm based on Ninja DAG, which has the characteristics of versatility, efficiency, flexibility and robustness:

[0128] As for universality, not only AOSP projects, but any large-scale projects based on the Ninja build system as the underlying build tool can use the directed acyclic graph splitting method provided in this application to complete the splitting of the entire DAG, even for operating systems such as Windows and Linux.

[0129] As for efficiency, from the application practice of the AOSP project, it only takes a few seconds to split a DAG of the order of 100,000 to complete. Compared with the time saved by distributed construction, this time consumption is completely negligible, which greatly reduces the project construction time.

[0130] As for flexibility, the splitting results can be adjusted very flexibly by setting different expected intervals, and the splitting effect comparison corresponding to different expected intervals can be displayed very intuitively through evaluation data, which is convenient for selecting the optimal splitting strategy.

[0131] As for robustness, for the DAG of a large project like AOSP, no matter what splitting parameters are adopted, the integrity and parallelism of the splitting results can be guaranteed.

[0132] In a possible implementation, after the step of dividing the second candidate node into the subtrees, the directed acyclic graph splitting method further includes the following scheme:

[0133] Get the tasks corresponding to each of the above subtrees;

[0134] Based on the dependency relationships between the different subtrees, determine the execution order of the tasks corresponding to each of the subtrees;

[0135] Scheduling the target execution object according to the execution order to sequentially execute the tasks corresponding to each of the subtrees; wherein the target execution object is a virtual machine in an idle state among the multiple virtual machines;

[0136] When the tasks corresponding to the above subtrees are completed, a project constructed according to the above directed acyclic graph is obtained.

[0137] After splitting the complete DAG into multiple subtrees, you can get subtrees without dependencies or subtrees with dependencies. When implementing a project built based on DAG, the Ninja build system obtains the tasks corresponding to each subtree. The tasks corresponding to each subtree can be understood as the source files required to build the project. After obtaining the tasks corresponding to each subtree, the execution order of the tasks corresponding to each subtree is determined according to the dependencies between different subtrees, that is, which tasks corresponding to the subtrees without dependencies can be executed in parallel, and which tasks corresponding to the subtrees with dependencies need to be executed after the dependent tasks are completed. Get the execution order, schedule the target execution object, and execute the tasks corresponding to each subtree in turn according to the execution order through the target execution object. After the tasks corresponding to all subtrees are executed, the project is built. Among them, the target execution object is a virtual machine in an idle state among the multiple virtual machines provided by the Ninja build system, and the virtual machine is the above-mentioned Worker.

[0138] When building a project, each task corresponding to a subtree has a task status, including unexecuted, executing, and completed. The task status of tasks corresponding to all subtrees is shared among all Workers. When any Worker is idle, it searches the task list corresponding to the subtree list to find the task corresponding to the subtree with a task status of unexecuted. If the tasks that the task corresponding to the found subtree depends on have been completed, the task status of the found task is marked as executing, and then the found task is used as the build target to start the build process. When the tasks corresponding to all subtrees are completed, the project build is completed.

[0139] For example, the workers provided by the Ninja build system are Worker1, Worker2, Worker3, ..., Worker N. Assume that, Figure 2The figure shows the multiple subtrees obtained after the complete DAG is split, which are subtree 1, subtree 2 and subtree 3. Subtree 1 and subtree 2 have no dependency relationship, and subtree 3 needs to depend on subtree 1 and subtree 2. Worker 1, Worker 2 and Worker 3 are all in idle state, then Worker 1 executes the task corresponding to subtree 1, and Worker 2 executes the task corresponding to subtree 2. When Worker 1 and Worker 2 execute tasks, the Ninja build system marks the task status of the tasks corresponding to subtree 1 and subtree 2 in the task list as being executed, and marks the task status of the task corresponding to subtree 3 as not being executed. After Worker1 and Worker2 complete their tasks, the Ninja build system marks the task status of the tasks corresponding to subtree 1 and subtree 2 in the task list as completed. The task corresponding to subtree 3 needs to be completed after the tasks corresponding to subtree 1 and subtree 2 are completed. That is, after the tasks corresponding to subtree 1 and subtree 2 are completed, the task status of the task corresponding to subtree 3 is found in the task list as unexecuted. At this time, Worker1, Worker2 and Worker3 are all idle. You can select any one of Worker1, Worker2 and Worker3 to execute the task corresponding to subtree 3. For example, select Worker3 to execute the task corresponding to subtree 3, and mark the task status of the task corresponding to subtree 3 in the task list as being executed. After Worker3 completes the task corresponding to subtree 3, the project build is completed. By selecting idle workers to execute the tasks corresponding to each subtree, load balancing and full utilization of resources are achieved.

[0140] According to the above technical scheme, this embodiment adopts a technical scheme of selecting one target node each time from the directed acyclic graph based on the depth-first traversal principle, determining the first candidate node that needs to be passed through to construct the target node, and then determining the second candidate node that needs to be passed through to construct the root node of the subtree from the first candidate node, and then dividing the second candidate node into subtrees. This realizes the splitting of the complete DAG into multiple subtrees without duplicate nodes, and can obtain subtrees without dependencies and subtrees with dependencies. When constructing a project implemented based on the DAG data structure, tasks corresponding to subtrees without dependencies can be executed in parallel. After the tasks corresponding to the subtrees without dependencies are completed, the tasks corresponding to the subtrees with dependencies are executed, which is conducive to improving the project construction speed and reducing the project construction time.

[0141] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0142] Fig. 9FIG. 1 shows a schematic diagram of a structure of a directed acyclic graph splitting device provided in an embodiment of the present application. Fig. 9 As shown, the directed acyclic graph splitting device 900 includes:

[0143] A selection module 910 is used to select a target node from the directed acyclic graph one at a time based on a depth-first traversal principle;

[0144] A first determination module 920 is used to determine a first candidate node that needs to be passed through to construct the above-mentioned target node;

[0145] A second determination module 930 is used to determine a second candidate node from the first candidate node, where the second candidate node is a node that needs to be passed through to construct the root node of the subtree;

[0146] The splitting module 940 is used to divide the second candidate node into the subtrees, and the nodes between different subtrees are not repeated.

[0147] In a possible implementation, the second determining module 920 includes:

[0148] A node quantity acquisition unit, used to determine the node quantity of the first candidate node according to the labeled quantity of the target node;

[0149] A data comparison unit, used to determine the comparison relationship between the above node quantity and the expected interval;

[0150] The candidate node selection unit is used to determine the second candidate node from the first candidate node according to the comparison relationship.

[0151] In a possible implementation, the first candidate node includes the target node and each first child node corresponding to the target node, the second candidate node includes the root node and the second child node corresponding to the root node, and the candidate node selection unit includes:

[0152] The first node selection subunit is used to determine the target node as the root node and determine each of the first child nodes as the second child node when the comparison relationship is that the number of nodes is within the expected range.

[0153] In a possible implementation, the first candidate node includes the target node and each first child node corresponding to the target node, the second candidate node includes the root node and the second child node corresponding to the root node, and the candidate node selection unit further includes:

[0154] The annotation quantity acquisition subunit is used to acquire the annotation quantity carried by each of the first subnodes when the comparison relationship is that the number of nodes is greater than the maximum value of the expected interval;

[0155] The second node selection sub-unit is used to, when the number of annotations carried by the first child node is within the above-mentioned expected interval, based on the above-mentioned depth-first traversal principle, take the first child node whose number of annotations carried is within the above-mentioned expected interval as the above-mentioned target node, and execute the step of determining the node number of the above-mentioned first candidate node according to the number of annotations of the above-mentioned target node, so as to select the above-mentioned root node and the above-mentioned second child node from the above-mentioned target node and each first child node corresponding to the above-mentioned target node.

[0156] In a possible implementation, the candidate node selection unit further includes:

[0157] The third node selection subunit is used to determine the target node as the root node and determine the first subnodes as the second subnodes when the number of annotations carried by each of the first subnodes is less than the minimum value of the expected interval.

[0158] In a possible implementation, the directed acyclic graph splitting device 900 further includes:

[0159] A quantity counting unit, used to obtain the total number of first child nodes required to construct the above target node;

[0160] The labeling unit is used to use the total number as a label of the target node.

[0161] In a possible implementation, the directed acyclic graph splitting device 900 further includes:

[0162] A task acquisition unit, used to acquire the task corresponding to each of the above subtrees;

[0163] An order determination unit, used to determine the execution order of the tasks corresponding to each of the above subtrees based on the dependency relationship between the different above subtrees;

[0164] A target scheduling unit, used for scheduling the target execution object to execute the tasks corresponding to each of the above subtrees in sequence according to the above execution order; wherein the above target execution object is a virtual machine in an idle state among the multiple virtual machines;

[0165] The project construction unit is used to obtain a project constructed according to the directed acyclic graph when the tasks corresponding to the subtrees are completed.

[0166] It should be noted that when the directed acyclic graph splitting device provided in the above-mentioned embodiment executes the directed acyclic graph splitting method, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the directed acyclic graph splitting device provided in the above-mentioned embodiment and the directed acyclic graph splitting method embodiment belong to the same concept. Therefore, for details not disclosed in the embodiment of the device disclosed in the present invention, please refer to the embodiment of the directed acyclic graph splitting method disclosed in the present invention, which will not be repeated here.

[0167] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0168] Fig.10 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown.

[0169] For example, Fig.10 As shown, the electronic device 1000 includes: a memory 1001 and a processor 1002, wherein the memory 1001 stores an executable program code 10011, and the processor 1002 is used to call and execute the executable program code 10011 to perform a directed acyclic graph splitting method.

[0170] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be corresponded to, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0171] In the case of dividing each functional module according to each function, the electronic device may include: a selection module, a first determination module, a second determination module, a splitting module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0172] The electronic device provided in this embodiment is used to execute the above-mentioned directed acyclic graph splitting method, and thus can achieve the same effect as the above-mentioned implementation method.

[0173] In the case of an integrated unit, the electronic device may include a processing module and a storage module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device in executing mutual program codes and data.

[0174] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits disclosed in the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0175] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a directed acyclic graph splitting method in the above-mentioned embodiment.

[0176] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement a directed acyclic graph splitting method in the above-mentioned embodiment.

[0177] In addition, the electronic device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the electronic device may include a connected processor and a memory; wherein the memory is used to store instructions, and when the electronic device is running, the processor may call and execute instructions so that the chip executes a directed acyclic graph splitting method in the above-mentioned embodiment.

[0178] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0179] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0180] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0181] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A directed acyclic graph splitting method, characterized in that: The directed acyclic graph splitting method comprises: Based on the depth-first traversal principle, select one target node at a time from the directed acyclic graph; Determine a first candidate node that needs to be passed through to build the target node; Determine a second candidate node from the first candidate nodes, where the second candidate node is a node that needs to be passed through to construct the root node of the subtree; The second candidate node is divided into the subtrees, and nodes between different subtrees are not repeated.

2. The directed acyclic graph splitting method according to claim 1, characterized in that: The step of determining the second candidate node from the first candidate nodes comprises: Determine the number of the first candidate nodes according to the number of annotations of the target node; Determine a comparison relationship between the number of nodes and an expected interval; The second candidate node is determined from the first candidate nodes according to the comparison relationship.

3. The directed acyclic graph splitting method according to claim 2, characterized in that: The first candidate nodes include the target node and each first child node corresponding to the target node, and the second candidate nodes include the root node and the second child node corresponding to the root node; The step of determining the second candidate node from the first candidate nodes according to the comparison relationship includes: When the comparison relationship is that the number of nodes is within an expected range, the target node is determined as the root node, and each of the first child nodes is determined as the second child node.

4. The directed acyclic graph splitting method according to claim 2, characterized in that: The first candidate nodes include the target node and each first child node corresponding to the target node, and the second candidate nodes include the root node and the second child node corresponding to the root node; The step of determining the second candidate node from the first candidate nodes according to the comparison relationship includes: When the comparison relationship is that the number of nodes is greater than the maximum value of the expected interval, obtaining the number of annotations carried by each of the first child nodes; When the number of annotations carried by the first child node is within the expected interval, based on the depth-first traversal principle, the first child node carrying the number of annotations within the expected interval is used as the target node, and the step of determining the number of nodes of the first candidate node according to the number of annotations of the target node is performed to select the root node and the second child node from the target node and each first child node corresponding to the target node.

5. The directed acyclic graph splitting method according to claim 4, characterized in that: After the step of obtaining the number of annotations carried by each of the first child nodes when the comparison relationship is that the number of nodes is greater than the maximum value of the expected interval, the directed acyclic graph splitting method further includes: When the number of annotations carried by each of the first child nodes is smaller than the minimum value of the expected interval, the target node is determined as the root node, and each of the first child nodes is determined as the second child node.

6. The directed acyclic graph splitting method according to any one of claims 3 to 5, characterized in that: Before the step of selecting a target node from the directed acyclic graph each time based on the depth-first traversal principle, the directed acyclic graph splitting method further includes: Obtain the total number of first child nodes required to construct the target node; The total number is used as the label of the target node.

7. The directed acyclic graph splitting method according to any one of claims 1 to 5, characterized in that: After the step of dividing the second candidate node into the subtrees, the directed acyclic graph splitting method further includes: Obtaining the tasks corresponding to each of the subtrees; Determining the execution order of the tasks corresponding to each of the subtrees based on the dependency relationships between the different subtrees; Scheduling the target execution object according to the execution order to sequentially execute the tasks corresponding to each of the subtrees; wherein the target execution object is a virtual machine in an idle state among the multiple virtual machines; When the tasks corresponding to the subtree are completed, a project constructed according to the directed acyclic graph is obtained.

8. A directed acyclic graph splitting device, characterized in that: The directed acyclic graph splitting device comprises: A selection module is used to select a target node from the directed acyclic graph one at a time based on the depth-first traversal principle; A first determination module, used to determine a first candidate node that needs to be passed through to construct the target node; A second determination module is used to determine a second candidate node from the first candidate nodes, where the second candidate node is a node that needs to be passed through to construct the root node of the subtree; The splitting module is used to divide the second candidate node into the subtrees, and the nodes between different subtrees are not repeated.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images