A method and system for mixed task scheduling based on double dependency graph reconstruction in computing power network

Abstract

The application provides a kind of computing power network in double dependence graph reconstruction hybrid task scheduling method and system, by analyzing the task submission information and historical execution record in computing power network, the direct dependence relationship and indirect dependence relationship between tasks are obtained, the initial task dependence graph is constructed, real-time running state data and resource occupation information are collected to construct real-time resource dependence graph, the task topology feature set and resource topology feature set are obtained by feature extraction of dependence graph, the topology difference feature is obtained by comparison, the dependence relationship correction parameter set is generated, and the dependence graph is iteratively reconstructed based on this, to obtain the reconstructed double dependence graph, to construct hybrid task scheduling strategy, to allocate tasks to corresponding computing power nodes, and generate an optimized task allocation scheme. The method of the application can improve the overall efficiency and reliability of task scheduling.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, and in particular to a hybrid task scheduling method and system for reconstructing dual dependency graphs in a computing power network. Background Technology

[0002] In computing power networks, hybrid task scheduling is a key technology that improves resource utilization and task execution efficiency by rationally allocating computing resources and scheduling task execution order. Currently, task dependency graphs are typically constructed based on the logical dependencies between tasks, or resource allocation tables are generated based on the resource status of computing nodes, and then scheduling decisions are made in conjunction with task priority rules or resource matching algorithms. However, traditional methods often use a layered modeling approach when dealing with task and resource dependencies, treating the logical execution relationship of tasks and the physical supply relationship of underlying resources as independent scheduling criteria, failing to form a unified dependency model. This makes it difficult for task scheduling decisions to simultaneously adapt to the logical constraints of task execution and the physical constraints of resource supply. Furthermore, in existing technologies, once the dependency graph structure is constructed, it is statically applied to the scheduling process and cannot be adjusted according to dynamic changes in task status and resources. When node load fluctuates or link status changes in the computing power network, the static dependency graph is prone to path conflicts or resource matching imbalances. Therefore, how to improve the adaptability of task scheduling in computing power networks to complex dependencies has become an urgent technical problem to be solved. Summary of the Invention

[0003] In view of this, the present invention provides a hybrid task scheduling method and system for dual dependency graph reconstruction in computing power networks.

[0004] The technical solution of this invention is implemented as follows: On one hand, embodiments of the present invention provide a hybrid task scheduling method for reconstructing a dual dependency graph in a computing power network. The method includes: parsing task submission information and historical execution records in the computing power network to obtain direct and indirect dependencies between tasks; constructing an initial task dependency graph based on the direct and indirect dependencies; and collecting real-time running status data and resource occupancy information of each node in the computing power network to construct a real-time resource dependency graph, thus obtaining an initial structure of the dual dependency graph; extracting topological features from the initial task dependency graph and the real-time resource dependency graph to obtain a task topological feature set and a resource topological feature set; and comparing the task topological feature set with the resource topological feature set. The topological difference features are obtained, and a dependency correction parameter set is generated based on these features. The initial structure of the dual-dependency graph is iteratively reconstructed based on the dependency correction parameter set. In each iteration, the edge weights and node connections in the graph structure are adjusted according to the corresponding parameters in the dependency correction parameter set to obtain a reconstructed dual-dependency graph. Based on the reconstructed dual-dependency graph, a hybrid task scheduling strategy is constructed. This strategy includes a task priority evaluation mechanism and a resource collaborative matching mechanism. The task priority evaluation mechanism determines the execution order of tasks, and the resource collaborative matching mechanism allocates tasks to corresponding computing power nodes, generating an optimized task allocation scheme.

[0005] On the other hand, embodiments of the present invention provide a computer system including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement the steps in the above-described method.

[0006] The hybrid task scheduling method for reconstructing a dual dependency graph in a computing power network provided by this invention constructs an initial task dependency graph by parsing task submission information and historical execution records, and constructs a real-time resource dependency graph by combining the real-time running status and resource occupancy information of nodes, resulting in an initial structure of a dual dependency graph containing task dependency subgraphs and resource dependency subgraphs. This dual dependency graph construction method can associate and integrate the direct and indirect dependencies between tasks with the resource supply relationships of computing power nodes, avoiding the limitation of isolated modeling of task logical dependencies and resource physical dependencies in traditional scheduling, and improving the comprehensiveness of dependency modeling. By extracting and comparing the topological structure features of the initial task dependency graph and the real-time resource dependency graph to generate topological difference features, a set of dependency correction parameters is generated based on the topological difference features to achieve dynamic correction of dependencies. The system is accurate, avoiding the problem of static dependency graphs being out of touch with actual scheduling requirements and improving the accuracy of dependency descriptions. It iteratively reconstructs the initial structure of the dual dependency graph based on a dependency correction parameter set, eliminating conflicting and redundant dependency paths and strengthening critical task chains by adjusting edge weights and node connections. This dynamically optimizes the graph structure to adapt to dynamic changes in the computing power network, enhancing its adaptability to complex scheduling scenarios. Based on the reconstructed dual dependency graph, a hybrid task scheduling strategy is constructed, incorporating both task priority evaluation and resource collaborative matching mechanisms. The task priority evaluation mechanism determines the execution order, while the resource collaborative matching mechanism allocates computing power nodes, achieving collaborative decision-making between task logical priority and resource physical matching degree. This avoids the problem of priority and resource allocation being out of touch in traditional scheduling, improving the overall efficiency and reliability of task scheduling. Attached Figure Description

[0007] Figure 1 This is a schematic diagram illustrating the implementation process of a hybrid task scheduling method for reconstructing a dual dependency graph in a computing network, as provided in an embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of the hardware entity of a computer system provided in an embodiment of the present invention. Detailed Implementation

[0009] This invention provides a hybrid task scheduling method for reconstructing a dual dependency graph in a computing power network, which can be executed by a processor of a computer system. The computing power network includes, for example, cloud computing networks, edge computing networks, high-performance computing (HPC) networks, etc. The computer system includes, for example, servers in a data center system, computer devices in a distributed computing system, etc.

[0010] Figure 1 This is a schematic diagram illustrating the implementation process of a hybrid task scheduling method for dual dependency graph reconstruction in a computing power network, provided by an embodiment of the present invention. The method includes the following steps: Step S100: Analyze the task submission information and historical execution records in the computing power network to obtain the direct and indirect dependencies between tasks. Construct an initial task dependency graph based on the direct and indirect dependencies. Collect real-time running status data and resource usage information of each node in the computing power network to construct a real-time resource dependency graph and obtain the initial structure of the dual dependency graph.

[0011] Task submission information includes details such as the task's basic description and input / output data, provided by the user when submitting a task to the computing power network. Historical execution records contain detailed information about past task execution processes, such as execution order, execution time, and execution results. Direct dependencies refer to direct associations between tasks, such as the input of one task depending on the output of another. Indirect dependencies are derived from direct dependencies. The initial task dependency graph is a graph structure built with tasks as nodes and dependencies as directed edges, used to represent the dependencies between tasks. Real-time runtime status data reflects the current running status of each node in the computing power network, such as CPU utilization and memory utilization. Resource usage information refers to the resource usage of each node, such as storage resource usage and network bandwidth usage. The real-time resource dependency graph is a graph built with nodes as vertices and resource supply relationships as directed edges, used to represent resource dependencies between nodes. The initial structure of the dual dependency graph is a graph structure obtained by merging the initial task dependency graph and the real-time resource dependency graph after mapping nodes.

[0012] As one implementation method, step S100 can be specifically implemented as the following steps S110~S160: Step S110: Perform structured parsing on the task submission information, extract the task identifier, input data items and output data items, and determine the data dependency relationship between tasks by associating the generation relationship between the input data items and output data items of different tasks, as a component of the direct dependency relationship.

[0013] Structured parsing involves breaking down and analyzing task submission information according to certain rules to extract useful information. A task identifier is a unique identifier for each task, used to distinguish different tasks. Input data items are the input data required for task execution, and output data items are the output data generated after task execution. Data dependencies refer to the dependencies between tasks due to the input and output relationships of their data.

[0014] When performing structured parsing of task submission information, the process begins with field parsing. The task submission information is broken down into individual fields, extracting the unique task identifier, input data items, and output data items for each task, establishing a correspondence between tasks and data items. For example, in an image processing task, the input data item for task E might be an image file, and the output data item might be the processed image file. Next, all output data items for each task are traversed, constructing a data item-task mapping table to record the task corresponding to each output data item, thus obtaining the source task index for each data item. For each task's input data item, the data item-task mapping table is queried to determine the generating task for that input data item. When a corresponding generating task exists for an input data item, a directed connection is created from the generating task to the current task, marked as the data dependency direction. For example, if the input data item for task F is the output data item for task G, then a directed connection is created from task G to task F. Finally, all data dependency directions are deduplicated, duplicate directed connections are deleted, and by analyzing the transmission time of data items in historical execution, a weight value representing the degree of dependency is assigned to each directed connection to obtain the data dependency table between tasks.

[0015] Specifically, step S110 can be implemented as the following steps S111~S114: Step S111: Parse the fields of the task submission information, extract the unique task identifier for each task, as well as the input and output data items related to the task, and establish the correspondence between tasks and data items.

[0016] Field parsing is the process of splitting and identifying task submission information according to its fields. A unique task identifier is used to uniquely distinguish each task, while input and output data items represent the data before and after task execution, respectively. Establishing a correspondence between tasks and data items clearly shows the data flow for each task. When parsing task submission information, regular expression matching can be used to extract the task identifier, input data items, and output data items from the submission information based on a predefined field format. For example, task submission information might be a text containing a task description; regular expressions can accurately extract the task identifier, the filename of the input data, and the filename of the output data, thus establishing a correspondence between tasks and data items.

[0017] Step S112: Traverse all output data items of all tasks, construct a data item-task mapping table, record the task corresponding to each output data item, and obtain the source task index of the data item.

[0018] The data item-task mapping table is used to record the correspondence between output data items and the tasks that generated those data items. The source task index allows for quick location of the task that generated the data item.

[0019] When iterating through the output data items of all tasks, a loop can be used to process each task's output data item sequentially. For each output data item, its corresponding task identifier is recorded in a data item-task mapping table. For example, if task H outputs data I, then the mapping table records the task corresponding to data I as task H. In this way, when querying the source task of a certain data item, it can be directly looked up in the mapping table.

[0020] Step S113: For each input data item of a task, query the data item-task mapping table to determine the generation task of the input data item. When there is a corresponding generation task for the input data item, create a directed connection from the generation task to the current task and mark it as the data dependency direction.

[0021] Querying the data item-task mapping table is to find the generating tasks for the input data items. A directed connection is a type of connection in a graph structure used to represent the direction of dependencies between tasks. The direction of data dependencies clearly defines the flow of data between tasks.

[0022] For each input data item of a task, a lookup is performed in the data item-task mapping table. If a corresponding generating task is found, a directed connection is created from the generating task to the current task, and the direction of this connection is marked as the data dependency direction. For example, if the input data item of task J is data K, and the mapping table shows that data K is generated by task L, then a directed connection is created from task L to task J, and marked as the data dependency direction.

[0023] Step S114: Perform deduplication on all data dependency directions, delete duplicate directed connections, and by analyzing the transmission time of data items in the past, assign a weight value representing the degree of dependency to each directed connection to obtain the data dependency table between tasks.

[0024] Deduplication is performed to prevent duplicate dependencies from affecting subsequent analysis and processing. The weight values ​​for dependency tightness reflect the strength of the dependency relationship between tasks. The data dependency table records the data dependencies between tasks and their weight values.

[0025] When deduplicating all data dependencies, all directed connections can be stored in a list. Duplicate connections are then determined by comparing their start and end points, and these are deleted. Analyzing the transmission time of data items in historical executions can be achieved by statistically analyzing the time it takes for data items to be transmitted from the generation task to dependent tasks. Each directed connection is assigned a weight based on its transmission time; shorter transmission times result in larger weights, indicating a tighter dependency. Finally, the processed directed connections and their weights are compiled into a data dependency table.

[0026] Step S120: Extract the task call sequence from the historical execution record, identify the preceding call relationship in the task execution process, generate the call dependency relationship between tasks as a component of the direct dependency relationship, and merge the data dependency relationship and the call dependency relationship to obtain the complete direct dependency relationship.

[0027] A task invocation sequence is the order in which tasks are executed in the historical execution record. A prerequisite invocation relationship means that one task must execute another task before it can be executed. A call dependency relationship is a dependency relationship between tasks generated based on prerequisite invocation relationships. A complete direct dependency relationship is the result of merging data dependencies and call dependencies.

[0028] When extracting task call sequences from historical execution records, the execution time and order information of tasks can be extracted to organize the task call sequence. When identifying prerequisite calls, the order of tasks in the call sequence is analyzed to determine which tasks are prerequisites for others. For example, in a software development task, the compilation task must be executed before the testing task; therefore, the compilation task is a prerequisite for the testing task. When generating call dependencies, directed connections from prerequisite tasks to subsequent tasks are created based on the prerequisite call relationships. Finally, data dependencies and call dependencies are merged to obtain the complete direct dependencies.

[0029] Step S130: Perform transitive closure calculation on the complete direct dependencies to generate a candidate set of indirect dependencies. Combine the fluctuation of task execution time intervals and task success rates in historical execution records to filter the candidate set of indirect dependencies and obtain indirect dependencies that meet the credibility requirements.

[0030] Transitive closure computation is used to compute all possible indirect paths in the graph. The candidate set of indirect dependencies is a set containing all possible indirect dependencies. The fluctuation in task execution time intervals reflects the stability of task execution time. Task success rate refers to the proportion of tasks that are successfully executed. Indirect dependencies that meet the credibility requirements are filtered indirect dependencies with higher credibility.

[0031] When calculating the transitive closure of complete direct dependencies, the complete direct dependencies are converted into an adjacency matrix. The transitive closure is then calculated using matrix multiplication, generating a candidate matrix containing all possible indirect paths. Matrix elements indicate whether indirect dependency paths exist between tasks. The candidate matrix is ​​then converted into a candidate set of indirect dependencies, recording the source task, target task, and the number of direct dependency edges contained in each candidate path. The continuous execution time intervals of the source and target tasks in historical execution records are extracted, and the fluctuation degree of the time interval is calculated. When the fluctuation degree is within a preset stable range, the candidate path of indirect dependency is retained. The ratio of the number of successful executions of tasks executed through candidate indirect dependency paths in historical execution records to the total number of executions is calculated to obtain the path success rate. When the success rate is higher than a preset confidence threshold, the candidate path is marked as a valid indirect dependency.

[0032] As one implementation method, step S130 can be specifically implemented as the following steps S131~S134: Step S131: Convert the complete direct dependencies into an adjacency matrix, calculate the transitive closure through matrix multiplication, and generate a candidate matrix containing all possible indirect paths. The matrix elements indicate whether there are indirect dependency paths between tasks.

[0033] An adjacency matrix is ​​a matrix used to represent the structure of a graph; its elements indicate whether there are connections between nodes. Transitive closure computation uses matrix multiplication to find all possible indirect paths in the graph. A candidate matrix is ​​a matrix containing information about all possible indirect dependencies.

[0034] When converting complete direct dependencies into an adjacency matrix, a square matrix is ​​created based on the number of tasks, with rows and columns corresponding to different tasks. If two tasks have a direct dependency, the corresponding element in the matrix is ​​1; otherwise, it is 0. The transitive closure is calculated using matrix multiplication, repeatedly multiplying the adjacency matrix by itself until the matrix no longer changes. The resulting matrix is ​​a candidate matrix containing all possible indirect paths. A matrix element of 1 indicates the existence of an indirect dependency path between tasks, while 0 indicates its absence.

[0035] Step S132: Convert the candidate matrix into a candidate set of indirect dependencies, and record the source task, target task, and number of direct dependency edges contained in each candidate path.

[0036] The indirect dependency candidate set is a collection that stores all possible indirect dependencies. The source task is the starting task of an indirect dependency path, and the target task is the ending task of an indirect dependency path. The number of direct dependency edges refers to the number of direct dependencies contained in an indirect dependency path.

[0037] When converting the candidate matrix into a candidate set of indirect dependencies, the elements of the candidate matrix are traversed. When an element is 1, it indicates the existence of an indirect dependency path. The source task, target task, and number of direct dependency edges in the path are recorded. For example, if there is an indirect dependency path from task M to task N with 2 direct dependency edges, then the candidate set of indirect dependencies records task M as the source task, task N as the target task, and 2 direct dependency edges.

[0038] Step S133: Extract the continuous execution time interval between the source task and the target task from the historical execution record, calculate the fluctuation degree of the time interval, and retain the indirect dependency candidate path when the fluctuation degree is within the preset stable range.

[0039] The continuous execution time interval refers to the time difference between the continuous execution of the source task and the target task in their historical execution. The degree of fluctuation reflects the stability of the time interval. The preset stability range is a pre-defined allowable range for fluctuations in the time interval.

[0040] When extracting the consecutive execution time intervals between the source and target tasks from historical execution records, the execution times of the source and target tasks are identified from the historical execution records, and the time difference between them is calculated. To calculate the fluctuation of the time intervals, the standard deviation method can be used to calculate the standard deviation of multiple time intervals. When the fluctuation is within a preset stable range, it indicates that the execution time of the indirect dependency candidate path is relatively stable, and this path is retained.

[0041] Step S134: Calculate the ratio of the number of successful tasks executed through indirect dependency candidate paths in the historical execution records to the total number of times to obtain the path success rate. When the success rate is higher than the preset trust threshold, mark the candidate path as a valid indirect dependency relationship.

[0042] Path success rate refers to the proportion of tasks successfully executed through indirect dependency candidate paths. The preset trust threshold is a pre-defined minimum requirement for the success rate. A valid indirect dependency is an indirect dependency with high trust. When calculating the ratio of the number of successful tasks executed through indirect dependency candidate paths to the total number of successful tasks in historical execution records, tasks executed through that candidate path are identified from the historical execution records, and the number of successful tasks and the total number of successful tasks are counted. The ratio is then used to calculate the path success rate. When the success rate is higher than the preset trust threshold, it indicates that the candidate path has high trust and is marked as a valid indirect dependency.

[0043] Step S140: Based on the complete direct and indirect dependencies, construct an initial task dependency graph with task identifiers as nodes and dependencies as directed edges. Assign dependency type identifiers and dependency strength weights to the directed edges using task execution history data.

[0044] The initial task dependency graph is a graph structure built with tasks as nodes and dependencies as directed edges. Dependency type identifiers are used to distinguish different types of dependencies, such as data dependencies and invocation dependencies. Dependency strength weights reflect the strength of the dependency relationship.

[0045] When constructing the initial task dependency graph, task identifiers are used as nodes, and complete direct and indirect dependencies are used as directed edges. Based on historical task execution data, each directed edge is assigned a dependency type identifier and a dependency strength weight. For example, based on the dependency frequency and duration between tasks in historical data, directed edges are assigned different dependency strength weights; the higher the dependency frequency and the shorter the dependency duration, the greater the weight.

[0046] Step S150: Collect the computing resource load, storage resource load, and communication link load of each node in the computing power network as real-time operating status data. Combine the communication link delay and resource allocation history between nodes as resource occupancy information. Construct a real-time resource dependency graph with node identifiers as vertices and resource supply relationships as directed edges. Assign resource transmission rate and link stability indicators to the directed edges through resource transmission test data.

[0047] Computational resource load refers to the utilization of a node's computing power, such as CPU utilization. Storage resource load refers to the utilization of a node's storage resources, such as disk utilization. Communication link load refers to the utilization of communication links between nodes, such as network bandwidth utilization. Communication link latency refers to the time delay in data transmission between nodes. Resource allocation history records the past resource allocation of nodes. A real-time resource dependency graph is a graph constructed with nodes as vertices and resource supply relationships as directed edges. Resource transmission rate refers to the speed at which data is transmitted between nodes. Link stability metrics reflect the stability of the communication link.

[0048] When collecting real-time operational status data of each node in the computing network, system monitoring tools such as `top` and `vmstat` can be used to obtain the node's computing resource load, storage resource load, and communication link load. Combining the communication link latency and resource allocation history between nodes, a real-time resource dependency graph is constructed with node identifiers as vertices and resource supply relationships as directed edges. Through resource transmission test data, such as latency measured by the `ping` command and transmission rate measured by the `iperf` command, directed edges are assigned resource transmission rate and link stability metrics.

[0049] Step S160: Map the initial task dependency graph and the real-time resource dependency graph to nodes, establish the association between task nodes and resource nodes, and merge them to obtain the initial structure of a dual dependency graph containing task dependency subgraphs and resource dependency subgraphs.

[0050] Node mapping maps task nodes in the initial task dependency graph to resource nodes in the real-time resource dependency graph. The association between task nodes and resource nodes represents the correspondence between a task and the resources required to execute that task. The initial structure of the dual dependency graph is a graph structure resulting from merging the initial task dependency graph and the real-time resource dependency graph, containing both task dependency subgraphs and resource dependency subgraphs.

[0051] When mapping nodes between the initial task dependency graph and the real-time resource dependency graph, the association between task nodes and resource nodes is established based on the resource requirements of the tasks and the resource availability of the nodes. For example, if task O needs to use the computing resources of node P, then an association is established between task O and node P. The two graphs are then merged to obtain an initial structure of a dual dependency graph containing both task dependency subgraphs and resource dependency subgraphs.

[0052] Step S200: Extract topological structure features from the initial task dependency graph and the real-time resource dependency graph to obtain a task topological feature set and a resource topological feature set. Compare the task topological feature set with the resource topological feature set to obtain topological difference features. Generate a dependency relationship correction parameter set based on the topological difference features.

[0053] Topology feature extraction extracts features from a graph structure that reflect its topological characteristics. The task topology feature set is a collection of initial task dependency graph topology features. The resource topology feature set is a collection of real-time resource dependency graph topology features. Topology difference features represent the differences between the task topology feature set and the resource topology feature set. The dependency correction parameter set is a set of parameters generated based on the topology difference features to correct dependencies.

[0054] When extracting topological features from the initial task dependency graph, node centrality analysis and path feature analysis can be used. Node centrality analysis calculates the degree centrality, betweenness centrality, and compact centrality of task nodes, reflecting the importance of nodes in the graph. Path feature analysis identifies critical paths in the graph and extracts features such as path length, number of nodes, and average weight. When extracting topological features from the real-time resource dependency graph, the resource distribution characteristics of resource nodes, such as remaining computing resources, remaining storage resources, and available communication resources, are analyzed, as well as link quality characteristics such as transmission efficiency and stability of inter-node links.

[0055] When comparing the task topology feature set with the resource topology feature set, feature values ​​are compared along corresponding dimensions to calculate the degree of difference, thus obtaining topology difference features. Based on these topology difference features, a correction coefficient is assigned to each feature dimension, generating a dependency correction parameter set. For example, when the centrality of a node in the task topology features differs significantly from the resource supply capacity of the corresponding node in the resource topology features, a larger correction coefficient is assigned to that feature dimension.

[0056] As one implementation method, step S200 can be specifically implemented as the following steps S210~S260: Step S210: Perform node centrality analysis on the initial task dependency graph. By calculating the number of direct dependency edges of task nodes, the frequency of being on the shortest dependency path, and the average path length to other nodes, generate node centrality features that describe the importance of nodes.

[0057] The number of direct dependency edges reflects the direct connections between a node and other nodes. The frequency of a node on the shortest dependency path indicates how often a node appears in the shortest path of the graph. The average path length to other nodes is the average of the shortest path lengths from a node to all other nodes in the graph. Node centrality is a feature used to describe the importance of a node.

[0058] When performing node centrality analysis on the initial task dependency graph, each node in the graph is traversed, and the number of its direct dependency edges is calculated. Using a shortest path algorithm, such as Dijkstra's algorithm, the frequency of each node being on the shortest dependency path and the average path length to other nodes are calculated. For example, for task node Q, the number of other task nodes it directly connects to is calculated, the number of times it appears on all shortest dependency paths is counted, and the average shortest path length to all other task nodes is calculated. These values ​​are used as node centrality features.

[0059] Step S220: Traverse the directed edges of the initial task dependency graph, calculate the distribution characteristics of the edge weights, including the central tendency, dispersion and distribution pattern of the weights, identify the critical path with the largest sum of edge weights in the graph, and extract the path length, number of nodes and average weight as path features.

[0060] The distribution characteristics of edge weights reflect their statistical properties. Central tendency refers to the average level of edge weights, such as the arithmetic mean. Dispersion refers to the degree of dispersion of edge weights, such as variance. Distribution shape refers to the shape of the edge weight distribution, such as skewness. The critical path is the path in the graph with the largest sum of edge weights. Path length is the number of directed edges contained in the critical path. Number of nodes is the total number of task nodes contained in the critical path. Average weight is the average value of the edge weights in the critical path.

[0061] When traversing the directed edges of the initial task dependency graph, edge weights are collected, and their arithmetic mean is calculated to represent central tendency. The mean of the sum of squares of deviations of each weight from the mean is calculated to represent dispersion, and the direction and degree of skewness of the weight distribution are calculated to represent the distribution pattern. The initial task dependency graph is traversed using a depth-first search to identify start task nodes with an in-degree of 0 and end task nodes with an out-degree of 0, establishing start-end node pairs. For each start-end node pair, the cumulative edge weights of all possible paths are calculated, and the path with the largest cumulative value is marked as a candidate critical path. The number of directed edges on the critical path is extracted as the path length. The total number of task nodes contained in the path is counted, and the ratio of the cumulative edge weights to the path length is calculated as the average weight. These are combined to obtain path features describing the path characteristics.

[0062] As one implementation method, step S220 can be specifically implemented as the following steps S221~S224: Step S221: Traverse all directed edges in the initial task dependency graph, collect edge weight values ​​to obtain a weight set, calculate the arithmetic mean of the weight set to represent the central tendency, calculate the mean of the sum of squares of the deviations of each weight value from the mean to represent the dispersion, and calculate the skew direction and degree of the weight value distribution to represent the distribution pattern.

[0063] The weight set is a collection that stores the weight values ​​of all directed edges. The arithmetic mean is the sum of all values ​​in the weight set divided by the number of values. The mean of the sum of squared deviations is a measure of the dispersion of the weight values, also known as variance. The direction and degree of skewness reflect the asymmetry of the weight value distribution.

[0064] When traversing all directed edges in the initial task dependency graph, the weight value of each directed edge is collected into a weight set. The arithmetic mean of the weight set is calculated by summing all weight values ​​and then dividing by the number of weight values. To calculate the dispersion, the difference between each weight value and the mean is calculated first, the squares of these differences are summed, and then divided by the number of weight values. To calculate the distribution shape, the skewness coefficient method can be used, calculating the degree of skewness by the ratio of the third central moment to the cube of the standard deviation.

[0065] Step S222: Traverse the initial task dependency graph using depth-first search, identify the starting task node with an in-degree of 0 and the ending task node with an out-degree of 0, and establish a starting-ending node pair combination.

[0066] A starting task node with an in-degree of 0 is a task node to which no other task points. A ending task node with an out-degree of 0 is a task node that does not point to any other task node. A start-end node pair is a set of paired start and end task nodes. When traversing the initial task dependency graph using depth-first search, starting from a node, we continuously visit adjacent nodes along directed edges until no further visits are possible. During the traversal, the in-degree and out-degree of each node are counted. A node with an in-degree of 0 is marked as a starting task node; a node with an out-degree of 0 is marked as a ending task node. All start and end task nodes are then paired to create start-end node pairs.

[0067] Step S223: For each start-end node pair, calculate the cumulative edge weight of all possible paths, and mark the path with the largest cumulative weight as a critical path candidate.

[0068] The cumulative edge weight is the sum of the weights of all directed edges in a path. Critical path candidates are paths that are likely to become critical paths.

[0069] For each start-end node pair, use a backtracking algorithm or a dynamic programming algorithm to calculate the cumulative edge weights of all possible paths. For example, from the start task node R to the end task node S, find all possible paths and calculate the cumulative edge weights of each path. Mark the path with the largest cumulative weight as a critical path candidate.

[0070] Step S224: Extract the number of directed edges of the critical path as the path length, count the total number of task nodes contained in the path, calculate the ratio of the cumulative edge weight to the path length as the average weight, and combine them to obtain the path features describing the path characteristics.

[0071] Path length is the number of directed edges in the critical path. Total number of task nodes is the number of task nodes in the critical path. Average weight is the average weight of the edges in the critical path. Path features are the set of features describing the characteristics of the critical path.

[0072] When extracting the number of directed edges in the critical path, the total number of directed edges in the critical path is directly counted. When counting the total number of task nodes in the path, the critical path is traversed, and the number of task nodes it contains is recorded. When calculating the average weight, the cumulative edge weights of the critical path are divided by the path length. Combining the path length, the total number of task nodes, and the average weight yields the path features describing the path's characteristics.

[0073] Step S230: Merge node centrality features and path features to obtain a set of task topology features, and unify feature dimensions through feature normalization.

[0074] The task topology feature set is a collection containing node centrality features and path features. Feature normalization unifies the dimensions of different features to facilitate subsequent comparison and analysis. When merging node centrality features and path features, they are combined into a single set. Through feature normalization, such as Min-Max normalization or Z-Score normalization, the values ​​of different features are mapped to the same range, unifying the feature dimensions. For example, the values ​​of features such as node degree centrality and path length are mapped to the range [0,1].

[0075] Step S240: Perform resource distribution characteristic analysis on the real-time resource dependency graph, calculate the remaining computing resources, remaining storage resources, and available communication resources of resource nodes, analyze the transmission efficiency and stability of links between nodes, and generate resource node characteristics and link quality characteristics.

[0076] Resource distribution characteristic analysis analyzes the resource distribution of resource nodes in a real-time resource dependency graph. Remaining computing resources refer to the unused portion of a node's computing resources. Remaining storage resources refer to the unused portion of a node's storage resources. Available communication resources refer to the available bandwidth of communication links between nodes. Transmission efficiency refers to the speed at which data is transmitted between nodes. Stability refers to the reliability of communication links. Resource node characteristics describe the resource supply capacity of resource nodes. Link quality characteristics describe the transmission capacity of links between nodes.

[0077] As one implementation method, step S240 can be specifically implemented as the following steps S241~S244: Step S241: Collect the total resource capacity and used resource amount of each resource node in the real-time resource dependency graph, calculate the difference between the total resource capacity and the used resource amount, and obtain the remaining computing resources and remaining storage resources.

[0078] Total resource capacity refers to the total size of the computing and storage resources of a resource node. Used resources refer to the amount of computing and storage resources that a resource node has already used. Remaining computing and storage resources are the remaining portion after subtracting the used resources from the total resource capacity.

[0079] When collecting the total resource capacity and used resource amount of each resource node in the real-time resource dependency graph, system management tools or monitoring software can be used to obtain information on the total resource capacity of the node, such as the total number of CPU cores, total memory capacity, and total hard disk capacity, as well as information on the used resource amount, such as CPU utilization, memory utilization, and hard disk utilization. The difference between the total resource capacity and the used resource amount is calculated to obtain the remaining computing resources and remaining storage resources.

[0080] Step S242: Obtain the real-time transmission rate and data transmission success rate of the communication link between resource nodes, calculate the ratio of the actual transmission rate of the link to the theoretical maximum rate to represent the transmission efficiency, and use the ratio of the number of successful transmissions to the total number of transmissions in continuous transmission tests to represent the stability.

[0081] Real-time transmission rate refers to the current transmission speed of data in the communication link between resource nodes. Data transmission success rate refers to the proportion of data successfully transmitted within a certain period of time. Transmission efficiency is the ratio of actual transmission rate to theoretical maximum rate, reflecting the utilization efficiency of the link. Stability is the ratio of the number of successful transmissions to the total number of transmissions in continuous transmission tests, reflecting the reliability of the link.

[0082] To obtain the real-time transmission rate and data transmission success rate of communication links between resource nodes, network monitoring tools such as netstat and nload can be used to obtain the real-time transmission rate of the link. Through continuous transmission tests, such as using the ping or iperf command to perform multiple data transmissions, the number of successful transmissions and the total number of transmissions are counted to calculate the data transmission success rate. When calculating transmission efficiency, the actual transmission rate is divided by the theoretical maximum rate. When calculating stability, the number of successful transmissions is divided by the total number of transmissions.

[0083] Step S243: Normalize the remaining computing resources and remaining storage resources of each resource node according to resource type to obtain resource node characteristics and describe the resource supply capacity of the node.

[0084] Normalization maps the remaining resource values ​​of different resource types to the same range for comparison and analysis. Resource node characteristics describe the resource supply capacity of a resource node.

[0085] When normalizing the remaining computing and storage resources of each resource node by resource type, the Min-Max normalization method can be used to map the values ​​of the remaining computing and storage resources to the range [0,1]. For example, for the remaining computing resources, subtract the minimum value from its value, and then divide by the difference between the maximum and minimum values ​​to obtain the normalized remaining computing resource value. Combining the normalized remaining computing and storage resources yields the resource node characteristics, which describe the node's resource supply capacity.

[0086] Step S244: Group the transmission efficiency and stability of each communication link according to the link identifier to obtain the link quality characteristics, which are used to describe the resource transmission capacity of the link.

[0087] A link identifier is a unique identifier for each communication link. Link quality characteristics are features that describe the link's transmission capabilities.

[0088] When grouping the transmission efficiency and stability of each communication link by link identifier, the transmission efficiency and stability values ​​of links with the same identifier are grouped together. For example, for link T, its transmission efficiency and stability values ​​are recorded as the link quality characteristics. These link quality characteristics can be used to describe the link's resource transmission capacity.

[0089] Step S250: Calculate the network density and clustering degree of the real-time resource dependency graph, merge resource node features, link quality features, network density and clustering degree to obtain a set of resource topology features, and unify the feature dimensions through feature normalization processing.

[0090] Network density refers to the ratio of the actual number of edges in a graph to the number of possible edges, reflecting the tightness of the graph's connections. Clustering degree refers to the degree to which nodes in a graph are clustered together. The resource topology feature set is a collection that includes resource node features, link quality features, network density, and clustering degree. Feature normalization is the process of unifying the dimensions of different features.

[0091] When calculating the network density of a real-time resource dependency graph, the number of actual edges and the number of possible edges are counted, and the network density is obtained by dividing the number of actual edges by the number of possible edges. When calculating the clustering degree, clustering coefficients can be used to calculate the tightness of connections between each node's neighboring nodes. By combining resource node features, link quality features, network density, and clustering degree, a set of resource topology features is obtained. Feature normalization, such as Min-Max normalization or Z-Score normalization, is then applied to unify the feature dimensions.

[0092] Step S260: Compare the task topology feature set and the resource topology feature set according to the corresponding dimensions, calculate the degree of difference in feature values, generate topology difference features, assign a correction coefficient to each feature dimension based on the degree of difference, and obtain the dependency correction parameter set.

[0093] Corresponding dimension comparison compares features of the same type in the task topology feature set and the resource topology feature set. Feature value difference refers to the difference between corresponding feature values. Topology difference features reflect the differences between task topology features and resource topology features. Correction coefficients are coefficients used to correct dependencies. The dependency correction parameter set is a set containing all correction coefficients.

[0094] When comparing the task topology feature set and the resource topology feature set according to corresponding dimensions, the process iterates through both sets, comparing feature values ​​in the same dimension. To calculate the degree of difference between feature values, methods such as absolute value or Euclidean distance can be used. Based on the degree of difference, a correction coefficient is assigned to each feature dimension; the greater the difference, the larger the correction coefficient. All correction coefficients are then combined to obtain the dependency correction parameter set.

[0095] Step S300: Perform iterative reconstruction of the initial structure of the dual dependency graph based on the dependency correction parameter set. In each iteration, adjust the edge weights and node connections in the graph structure according to the corresponding parameters in the dependency correction parameter set, eliminate conflicting redundant dependency paths, strengthen the connection strength of the critical task chain, and obtain the reconstructed dual dependency graph.

[0096] Iterative reconstruction is a process of adjusting and optimizing the initial structure of a dual dependency graph through multiple iterations. The dependency correction parameter set is a collection of parameters used to correct dependencies. Edge weights are measures of the importance of edges in the graph. Node connections are the ways in which nodes are connected in the graph. Conflicting redundant dependency paths refer to unreasonable or repetitive dependency paths in the graph. Critical task chains are sequences of tasks in the graph that are crucial to task execution. Reconstructing the dual dependency graph results in the optimized dual dependency graph obtained after iterative reconstruction.

[0097] During each iteration, the edge weights and node connections in the graph structure are adjusted based on the corresponding parameters in the dependency correction parameter set. For example, the centrality features and directed edge weights of corresponding nodes in the initial task dependency graph are adjusted according to the task topology correction coefficient, improving the matching degree between task topology features and resource topology features. The resource capacity features and link quality features of corresponding nodes in the real-time resource dependency graph are adjusted according to the resource topology correction coefficient, improving the matching degree between resource supply capacity and task requirements.

[0098] When eliminating conflicting redundant dependency paths, the node pairs in the initial structure of the dual dependency graph are traversed to identify conflicting paths where the task dependency direction is opposite to the resource dependency direction. By comparing the weight values ​​of conflicting edges, conflicting edges with smaller weights are deleted to eliminate path conflicts. Redundant dependency paths with shorter alternative paths and smaller weight differences are identified, and the path structure is simplified by merging the weight values ​​of redundant edges, preserving core dependencies. When strengthening the connection strength of critical task chains, the weight values ​​of directed edges on critical paths in the task topology feature set are increased. This process is repeated iteratively until a preset termination condition is met, such as reaching the upper limit of the number of iterations or the change in topology features being less than a threshold, resulting in a reconstructed dual dependency graph.

[0099] As one implementation method, step S300 can be implemented as the following steps S310~S360: Step S310: Initialize the iteration counter and termination conditions. The termination conditions include the upper limit of the number of iterations and the threshold of the change in topological features. The iteration stops when either condition is met.

[0100] An iteration counter records the number of iterations. The maximum number of iterations is a pre-defined limit. The topology change threshold is a pre-defined minimum allowed value for topology feature changes. Iteration stops when the maximum number of iterations is reached or the topology change is less than the threshold. The iteration counter is initialized to 0. The maximum number of iterations and the topology change threshold are set; for example, the maximum number of iterations is set to 100, and the topology change threshold is set to 0.01. After each iteration, the termination condition is checked; if it is met, iteration stops.

[0101] Step S320: In the current iteration, extract the task topology correction coefficients from the dependency correction parameter set, adjust the centrality features and directed edge weights of the corresponding nodes in the initial task dependency graph, and improve the matching degree between the task topology features and the resource topology features.

[0102] Task topology correction coefficients are coefficients in the dependency correction parameter set used to correct task topology features. Centrality features describe the importance of a node in the graph. Directed edge weights are measures of the importance of edges in the graph. The matching degree between task topology features and resource topology features refers to the similarity between the task topology features and resource topology features. In the current iteration, task topology correction coefficients are extracted from the dependency correction parameter set. The centrality features and directed edge weights of the corresponding nodes in the initial task dependency graph are adjusted according to the correction coefficients. For example, if the task topology correction coefficient of a node is large, the centrality feature value and directed edge weight of that node are increased to make the task topology features better match the resource topology features.

[0103] Step S330: Extract the resource topology correction coefficients from the dependency correction parameter set, and adjust the resource capacity characteristics and link quality characteristics of the corresponding nodes in the real-time resource dependency graph to improve the matching degree between resource supply capacity and task requirements.

[0104] Resource topology correction coefficients are coefficients in the dependency correction parameter set used to correct resource topology characteristics. Resource capacity characteristics describe the resource supply capacity of resource nodes. Link quality characteristics describe the link transmission capacity between nodes. The matching degree between resource supply capacity and task requirements refers to the degree to which the resource supply capacity of resource nodes matches the resources required by the task. Resource topology correction coefficients are extracted from the dependency correction parameter set, and the resource capacity and link quality characteristics of corresponding nodes in the real-time resource dependency graph are adjusted according to these coefficients. For example, if a resource node has a large resource topology correction coefficient, the resource capacity and link quality characteristic values ​​of that node are increased to better match the resource supply capacity with the task requirements.

[0105] Step S340: Traverse the node pairs in the initial structure of the dual dependency graph, identify conflicting paths where the task dependency direction and resource dependency direction are opposite, and delete conflicting edges with smaller weights to eliminate path conflicts by comparing the weight values ​​of the conflicting edges.

[0106] A node pair is any combination of two nodes in the graph. The task dependency direction is the direction of the edges in the initial task dependency graph. The resource dependency direction is the direction of the edges in the real-time resource dependency graph. A conflict path is a path where the task dependency direction is opposite to the resource dependency direction. A conflict edge is an edge within a conflict path.

[0107] As one implementation method, step S340 can be specifically implemented as the following steps S341~S343: Step S341: Traverse all node pairs (u, v) in the initial structure of the dual dependency graph, check whether there is a task dependency edge from u to v and a resource dependency edge from v to u at the same time, or a resource dependency edge from u to v and a task dependency edge from v to u, and mark them as conflict path candidates.

[0108] A node pair (u, v) is any combination of two nodes in the initial structure of the dual dependency graph. Task dependency edges are edges in the initial task dependency graph, and resource dependency edges are edges in the real-time resource dependency graph. Conflict path candidates are paths that may conflict.

[0109] When traversing all node pairs (u, v) in the initial structure of the dual dependency graph, two nested loops are used to check each pair of nodes sequentially. For each pair of nodes, it is checked whether there exists both a task dependency edge from u to v and a resource dependency edge from v to u, or a resource dependency edge from u to v and a task dependency edge from v to u. If such a situation exists, the path is marked as a conflict path candidate.

[0110] Step S342: For each candidate conflict path, extract the weight values ​​of the task dependency edge and the resource dependency edge. When the weight value of the task dependency edge is less than the weight value of the resource dependency edge, determine the task dependency edge as a conflict edge to be deleted; when the weight value of the resource dependency edge is less than the weight value of the task dependency edge, determine the resource dependency edge as a conflict edge to be deleted; when the two weight values ​​are equal, determine the deletion object based on the success rate of the path in historical execution, and delete the edge with the lower success rate.

[0111] Task dependency edge weights measure the importance of task-dependent edges. Resource dependency edge weights measure the importance of resource-dependent edges. Conflicting edges to be deleted are those that need to be removed from the graph. The successful execution rate is the ratio of the number of times a path was successfully executed to the total number of executions in the historical execution history.

[0112] For each candidate conflict path, extract the weight values ​​of the task-dependent edges and resource-dependent edges. Compare these two values; if the weight value of the task-dependent edge is less than that of the resource-dependent edge, the task-dependent edge is identified as a conflict edge to be deleted; if the weight value of the resource-dependent edge is less than that of the task-dependent edge, the resource-dependent edge is identified as a conflict edge to be deleted. When the two weight values ​​are equal, query historical execution records, count the number of successful executions and the total number of executions for that path, calculate the success rate, and delete the edge with the lower success rate.

[0113] Step S343: Perform the conflict edge deletion operation, update the node connection relationship and edge attribute list of the dual dependency graph, record the deletion operation information for iterative verification, and ensure that there are no bidirectional conflict paths in the graph structure after deletion.

[0114] The conflict edge removal operation removes conflicting edges from the graph. Node connectivity refers to the way nodes are connected in the graph. The edge attribute list is a list recording the attribute information of the edges in the graph. Iterative verification is the process of checking whether the graph structure meets the requirements after each iteration. A bidirectional conflict path is a path in the graph that has two conflicting edges in opposite directions.

[0115] When performing a conflict edge deletion operation, remove the conflict edge to be deleted from the dual dependency graph. Update the node connectivity and edge attribute list of the dual dependency graph to ensure the correctness of the graph structure. Record deletion operation information, such as the start point, end point, and weight of the deleted edge, for iterative verification. After each iteration, check if there are any bidirectional conflicting paths in the graph structure; if so, make further adjustments.

[0116] Step S350: Identify redundant dependency paths with shorter alternative paths and smaller weight differences, simplify the path structure by merging the weight values ​​of redundant edges, and retain the core dependencies.

[0117] A shorter alternative path is one where a shorter path exists in the graph than the current dependent path. Small weight differences mean that the edge weights of the two paths are not significantly different. Redundant dependent paths are unnecessary dependent paths in the graph. Core dependencies are dependencies in the graph that are crucial to task execution.

[0118] When identifying redundant dependency paths with shorter alternative paths and small weight differences, a shortest path algorithm, such as Dijkstra's algorithm, is used to calculate the shortest path between any two nodes in the graph. The lengths and edge weights of the current dependency path are compared with those of the shortest path. If a shorter alternative path exists with a small weight difference, the dependency path is identified as redundant. By merging the weights of redundant edges, the weights of the redundant dependency path are incorporated into the alternative path, simplifying the path structure and preserving the core dependencies.

[0119] Step S360: For the critical path in the task topology feature set, increase the weight value of the directed edge on the path to strengthen the connection strength of the critical task chain, complete the graph structure adjustment of the current iteration, repeat the iteration until the preset termination condition is met, and obtain the reconstructed dual dependency graph.

[0120] The critical path is the path with the largest sum of edge weights in the task topology feature set. Directed edge weights are measures of the importance of edges in the graph. The critical task chain is the sequence of tasks represented by the critical path. Connection strength is the degree of dependency between tasks in the critical task chain. The reconstructed dual dependency graph is the optimized dual dependency graph obtained after iterative reconstruction.

[0121] For critical paths in the task topology feature set, weight values ​​are increased for directed edges along these paths. The node sequence and directed edge set of the critical path are extracted from the task topology feature set. Combined with the current load fluctuation characteristics of the corresponding resource nodes in the real-time resource dependency graph, a basic range for weight adjustment is determined. The weight sensitivity of the critical path under different load states is analyzed through historical execution records, generating a load-weight adjustment coefficient mapping table. The greater the load fluctuation, the more dynamic the adjustment coefficient. Based on the load-weight adjustment coefficient mapping table, a differentiated weight increase value is assigned to each directed edge on the critical path. The increase value is increased when the resource node load is low and decreased when the load is high. A path reinforcement feedback mechanism is introduced, comparing the execution efficiency improvement rate of the critical path after historical reinforcement. When the improvement rate is lower than a preset threshold, the weight increase value is dynamically adjusted to ensure the reinforcement effect. The adjusted edge weight values ​​are merged with the current edge weight values ​​of the reconstructed dual dependency graph, updating the dependency strength weight in the edge attributes, and synchronously recording the weight adjustment log for the next iteration optimization. The graph structure adjustment for the current iteration is completed, and the iteration is repeated until a preset termination condition is met, such as reaching the upper limit of the number of iterations or the change in topology features being less than a threshold, resulting in a reconstructed dual dependency graph.

[0122] As one implementation method, step S360 can be specifically implemented as the following steps S361~S365: Step S361: Extract the node sequence and directed edge set of the critical path from the task topology feature set, and determine the basic range of weight adjustment by combining the current load fluctuation characteristics of the corresponding resource nodes in the real-time resource dependency graph.

[0123] The node sequence of the critical path is the order of the task nodes contained in the critical path. The set of directed edges is the set of directed edges contained in the critical path. The current load fluctuation characteristic is the load fluctuation of the corresponding resource nodes in the real-time resource dependency graph. The base interval for weight adjustment is the range used to determine the weight increase value.

[0124] When extracting the node sequence and directed edge set of the critical path from the task topology feature set, the node sequence and directed edge set are extracted based on the critical path information recorded in the task topology feature set. Combined with the current load fluctuation characteristics of the corresponding resource nodes in the real-time resource dependency graph, the magnitude and trend of load fluctuation are analyzed. Based on the load fluctuation characteristics, a base interval for weight adjustment is determined. For example, when the load fluctuation is small, the base interval for weight adjustment can be set smaller; when the load fluctuation is large, the base interval for weight adjustment can be set larger.

[0125] Step S362: Analyze the weight sensitivity of the critical path under different load conditions through historical execution records, and generate a load-weight adjustment coefficient mapping table. The greater the load fluctuation, the more dynamic the adjustment coefficient.

[0126] Weight sensitivity refers to how sensitive the execution efficiency of the critical path is to changes in edge weights. The load-weight adjustment coefficient mapping table records the corresponding weight adjustment coefficients under different load conditions. Dynamic adjustment coefficients refer to coefficients that are adjusted in real time according to load fluctuations.

[0127] When analyzing the weight sensitivity of the critical path under different load conditions using historical execution records, the execution efficiency and edge weight values ​​of the critical path under different load conditions are extracted from the historical execution records. The impact of weight changes on execution efficiency is analyzed, and weight sensitivity is calculated. Based on weight sensitivity and load fluctuations, a load-weight adjustment coefficient mapping table is generated. The greater the load fluctuation, the more dynamic the adjustment coefficient, that is, the weight increase value is adjusted in real time according to the real-time load situation.

[0128] Step S363: Based on the load-weight adjustment coefficient mapping table, assign differentiated weight increment values ​​to each directed edge on the critical path. Increase the increment value when the resource node load is low and decrease the increment value when the load is high.

[0129] Differentiated weight increments refer to the different weight increments assigned to each directed edge on the critical path. Low resource node load means the load on the corresponding resource node in the real-time resource dependency graph is at a low level. High resource node load means the load on the corresponding resource node in the real-time resource dependency graph is at a high level. Based on the load-weight adjustment coefficient mapping table, differentiated weight increments are assigned to each directed edge on the critical path. The load-weight adjustment coefficient mapping table is queried, and the weight adjustment coefficient for each directed edge is determined based on the current load of the corresponding resource node. When the resource node load is low, the weight increment is increased to strengthen the connection strength of the critical task chain; when the resource node load is high, the weight increment is decreased to avoid excessively burdening the resource node.

[0130] Step S364: Introduce a path reinforcement feedback mechanism, compare the execution efficiency improvement rate of the critical path after historical reinforcement, and dynamically adjust the weight increase value when the improvement rate is lower than the preset threshold to ensure the reinforcement effect.

[0131] The path reinforcement feedback mechanism is a mechanism that adjusts the weight increment based on the execution efficiency feedback of the critical path. The execution efficiency improvement rate refers to the ratio of the execution efficiency of the critical path after reinforcement to its execution efficiency before reinforcement. The preset threshold is a pre-defined minimum requirement for execution efficiency improvement. Dynamically adjusting the weight increment is the process of adjusting the weight increment in real time based on execution efficiency feedback.

[0132] When introducing a path reinforcement feedback mechanism, the execution efficiency of the critical path after historical reinforcement is recorded. By comparing the efficiency improvement rate of the critical path after historical reinforcement, if the improvement rate is lower than a preset threshold, it indicates that the current weight increase value may be insufficient, and the weight increase value needs to be dynamically adjusted. For example, the weight increase value is increased, and iterative adjustments are made again until the execution efficiency improvement rate reaches the preset threshold to ensure the reinforcement effect.

[0133] Step S365: Merge the adjusted edge weight values ​​with the current edge weight values ​​of the reconstructed dual dependency graph, update the dependency strength weight in the edge attributes, and synchronously record the weight adjustment log for the next iteration optimization.

[0134] The adjusted edge weights are the edge weights after adjustment using the weight increment values. The current edge weights in the reconstructed dual dependency graph are the edge weights in the currently reconstructed dual dependency graph. Dependency strength weights are weights in the edge attributes that represent the tightness of the dependency between edges. The weight adjustment log is a log recording the weight adjustment process and results.

[0135] When merging the adjusted edge weights with the current edge weights in the reconstructed dual-dependency graph, the adjusted edge weights are added to the current edge weights to obtain the new edge weights. The dependency strength weights in the edge attributes are updated to ensure that the tightness of the edge dependencies is correctly reflected. A weight adjustment log is recorded synchronously, including information such as the start and end points of the adjusted edges, their original weights, and the adjusted weights, for use in the next iteration of optimization.

[0136] Step S400: Based on the reconstruction of the dual dependency graph, construct a hybrid task scheduling strategy. The hybrid task scheduling strategy includes a task priority evaluation mechanism and a resource coordination matching mechanism. The task priority evaluation mechanism determines the execution order of tasks, and the resource coordination matching mechanism allocates tasks to the corresponding computing power nodes to generate an optimized task allocation scheme.

[0137] The reconstructed dual dependency graph is the optimized dual dependency graph obtained after iterative reconstruction. The hybrid task scheduling strategy is a scheduling strategy that comprehensively considers task priority and resource matching. The task priority evaluation mechanism is used to determine the task execution order. The resource coordination matching mechanism is used to match tasks with suitable computing power nodes. The optimized task allocation scheme is the optimized task allocation plan.

[0138] As one implementation method, step S400 can be specifically implemented as the following steps S410~S450: Step S410: Based on the task dependency subgraph of the reconstructed dual dependency graph, construct a task priority evaluation mechanism. The task priority evaluation mechanism includes a basic priority calculation module and an adjustment module. The basic priority calculation module generates an initial priority by combining the centrality characteristics of task nodes and the critical path position. The adjustment module dynamically corrects the priority according to the real-time status of the resource dependency subgraph.

[0139] The task dependency subgraph in the reconstructed dual dependency graph represents the task dependencies within the graph. The task priority evaluation mechanism determines the task execution order. The basic priority calculation module calculates the initial task priority. The adjustment module dynamically adjusts task priorities based on resource status. Centrality describes the importance of a task node in the graph. Critical path location indicates whether a task node is located on the critical path. Initial priority is the initial priority score for a task. Real-time state is the current state of resource nodes in the resource dependency subgraph.

[0140] When constructing a task priority evaluation mechanism based on the task dependency subgraph of a reconstructed dual dependency graph, the initial priority is first generated in the basic priority calculation module by combining the centrality characteristics and critical path positions of task nodes. Centrality characteristics can include degree centrality, betweenness centrality, and tight centrality. Degree centrality reflects the number of connections a task node has with other nodes; the more connections, the more important the node is in the task network and the greater its impact on the overall task execution. Betweenness centrality measures the frequency with which a task node appears in the shortest paths between other nodes; the higher the frequency, the more critical the node's role in information transmission and task coordination. Tight centrality reflects the average shortest path length from a task node to all other nodes; the shorter the average path length, the closer the node is to other nodes, enabling faster interaction and collaboration.

[0141] In one implementation, step S410 can be specifically implemented as the following steps S411~S414: Step S411: In the basic priority calculation module, extract the degree centrality, betweenness centrality and tight centrality features of the task nodes, generate a centrality score by weighting them according to preset weights, and at the same time determine whether the task is located on the critical path. Tasks located on the critical path are given a critical path bonus coefficient, and the initial priority score is generated by merging them.

[0142] Degree centrality refers to the number of direct edges connecting a task node, reflecting its local influence in the graph. Betweenness centrality refers to the frequency with which a node appears in all shortest paths, reflecting its bridging role in information transmission and resource flow. Tight centrality is the reciprocal of the average shortest path length from a node to all other nodes in the graph, reflecting the proximity of the node to other nodes. Preset weights are weight values ​​pre-defined based on the importance of different centrality features to task priority. Centrality score is a comprehensive score obtained by multiplying each centrality feature value by its preset weight and then summing the results. Critical path bonus is an additional coefficient assigned to tasks located on the critical path to increase their priority. Initial priority score is the preliminary task priority obtained by comprehensively considering the centrality score and critical path factors.

[0143] In the basic priority calculation module, the degree centrality, betweenness centrality, and compact centrality features of task nodes are first extracted. When calculating degree centrality, the number of direct connecting edges between task nodes is counted. For example, in the task dependency graph of a software development project, the coding task may be directly connected to multiple tasks such as testing and debugging tasks, resulting in a high degree centrality. When calculating betweenness centrality, a shortest path algorithm (such as the Floyd-Warshall algorithm) is used to find the shortest path between all pairs of nodes in the graph. The number of times the task node appears on these shortest paths is counted, and then divided by the total number of shortest paths to obtain the betweenness centrality value. The compact centrality is calculated by first calculating the shortest path length from the task node to all other nodes, then taking the average of these shortest path lengths, and finally taking its reciprocal.

[0144] Next, these centrality features are weighted according to preset weights, and it is determined whether the task is on the critical path. The critical path is the path with the largest sum of edge weights in the task dependency graph, and it determines the shortest completion time for the entire task sequence. Critical path algorithms (such as Critical Path Method, CPM) can be used to find the critical path. If a task is on the critical path, it is assigned a critical path bonus coefficient, and the centrality score is multiplied by the critical path bonus coefficient to obtain the initial priority score.

[0145] Step S412: In the adjustment module, monitor the load fluctuation of resource nodes associated with tasks in the resource dependency subgraph. When the load of associated resource nodes exceeds a preset threshold, adjust the priority of tasks that depend on the node by lowering it. When the load is below the threshold, adjust it by raising it.

[0146] The resource dependency subgraph is a subgraph that reconstructs the dual dependency graph to represent the dependencies between resource nodes and between resource nodes and task nodes. Resource node load fluctuation refers to the change in the load of resource nodes (such as CPU utilization, memory utilization, network bandwidth utilization, etc.) over time. The preset threshold is a load limit pre-set based on the performance of resource nodes and the needs of task execution. Downward adjustment refers to lowering the priority of a task, while upward adjustment refers to raising the priority of a task.

[0147] The adjustment module continuously monitors the load fluctuations of resource nodes associated with tasks in the resource dependency subgraph. System monitoring tools and sensors can be used to obtain real-time load information of resource nodes. For example, for a big data processing task running on a server cluster, each server node has corresponding CPU and memory usage. When the CPU usage of a server node consistently exceeds 80% (a preset threshold), it indicates that the node is overloaded and may not be able to provide sufficient computing resources to tasks that depend on it in a timely manner. In this case, the priority of tasks that depend on that node is adjusted downwards.

[0148] The specific method for adjusting priority downwards can be based on the degree to which the load exceeds a threshold. For example, when CPU utilization exceeds the threshold by 10%-20%, the task priority is lowered by one level; if it exceeds 20%-30%, it is lowered by two levels. Conversely, when the load on a resource node is below the threshold, such as when CPU utilization is below 30%, it indicates that the node has more idle resources and can execute tasks more efficiently. In this case, the priority of tasks that depend on that node is adjusted upwards. The adjustment magnitude can also be determined based on the degree to which the load is below the threshold; for example, if it is 10%-20% below the threshold, the task priority is increased by one level; if it is 20%-30% below the threshold, it is increased by two levels.

[0149] Step S413: Introduce a task dependency chain blocking risk assessment, analyze the waiting time of downstream tasks and the completion progress of dependent tasks, and when the blocking risk is higher than the preset risk threshold, increase the priority adjustment of the current task to reduce the blocking probability.

[0150] Task dependency chain blocking risk assessment evaluates the risk that downstream tasks may fail to execute on time due to delays in the execution of certain tasks within a task dependency chain. Downstream task waiting time refers to the time a downstream task spends waiting for its preceding tasks to complete. Dependent task completion progress refers to the proportion of work completed by dependent tasks relative to the total workload. The preset risk threshold is a blocking risk limit pre-set based on the importance of the tasks and the overall time requirements of the tasks. Priority adjustment magnitude refers to the degree to which task priorities are adjusted. Blocking probability refers to the likelihood of a blocking situation occurring in the task dependency chain.

[0151] When assessing the blocking risk of a task dependency chain, the first step is to analyze the waiting time of downstream tasks. This waiting time can be calculated by recording the start time of downstream tasks and the estimated completion time of their predecessor tasks. For example, in a construction project, the decoration task can only begin after the main structure construction is completed. If the main structure construction is delayed for some reason, the decoration task will enter a waiting state, and the waiting time will continuously increase. Simultaneously, the completion progress of dependent tasks should be analyzed to understand the execution status of predecessor tasks. The completion progress can be determined by monitoring the comparison between the actual completed workload and the planned workload.

[0152] When the blocking risk exceeds a preset risk threshold, it indicates that the execution delay of the current task may significantly impact the entire task dependency chain, causing downstream tasks to wait for extended periods and reducing overall task execution efficiency. In this case, the priority adjustment level for the current task should be increased. For example, if the original plan was to increase the task priority by one level, when the blocking risk is high, it could be increased by two levels to speed up the execution of the current task and reduce the probability of blocking. A risk-adjustment level mapping table can be used to determine the specific adjustment level based on the specific degree of blocking risk. This mapping table can be established using historical task execution data and simulation experiments, recording the priority adjustment levels corresponding to different blocking risk levels.

[0153] Step S414: Integrate the initial priority score and adjustment correction value to generate a dynamic priority score. The task execution order is obtained by sorting the scores. The sorting result is synchronized to the task scheduling queue in real time to ensure that the priority is dynamically updated with the resource status.

[0154] The initial priority score is a preliminary task priority score generated in the basic priority calculation module. The adjustment correction value is a value used in the adjustment module to adjust the task priority based on resource status and blocking risk. The dynamic priority score is a real-time task priority score obtained by comprehensively considering the initial priority score and the adjustment correction value. The task execution order is the execution sequence obtained by sorting tasks according to their dynamic priority scores. The task scheduling queue is a queue used to store tasks to be executed and schedule them according to their priority.

[0155] When integrating the initial priority score and the adjustment correction value, the adjustment correction value is added to the initial priority score to obtain the dynamic priority score. The task execution order is obtained by sorting all tasks according to their dynamic priority scores. Sorting algorithms (such as quicksort, heapsort, etc.) can be used to sort the tasks, arranging them in descending order of dynamic priority score. The sorting result is synchronized to the task scheduling queue in real time. The task scheduling queue will schedule tasks for execution sequentially according to their dynamic priority order. As resource status continuously changes, the adjustment module will continuously correct task priorities, generate new dynamic priority scores and task execution orders, and update them to the task scheduling queue in a timely manner, ensuring that task priorities always match resource status and improving the efficiency and rationality of task scheduling.

[0156] Step S420: Based on the resource dependency subgraph of the reconstructed dual dependency graph, a multi-dimensional resource collaborative matching mechanism is constructed. This mechanism establishes a collaborative matching model between tasks and resource nodes through the supply characteristics, link transmission characteristics and historical task adaptation characteristics of resource nodes. The resource dependency subgraph of the reconstructed dual dependency graph reflects the dependencies between resource nodes and between resource nodes and task nodes. Based on this, a multi-dimensional resource collaborative matching mechanism can be constructed to more comprehensively and accurately match tasks with resource nodes. The supply characteristics of resource nodes refer to their resource supply capabilities, including the dynamic trend of remaining resources, resource elastic expansion capabilities, and resource fragmentation rate. The dynamic trend of remaining resources reflects the change of remaining resources (such as computing resources and storage resources) of a resource node over time; understanding this trend helps predict the amount of available resources a resource node will have in the future. Resource elastic expansion capabilities reflect the ability of a resource node to quickly increase resource supply when facing increased resource demand, such as by dynamically allocating additional computing cores or storage capacity. Resource fragmentation rate indicates the degree to which available resources in a resource node are divided into small pieces; an excessively high fragmentation rate may lead to low resource utilization efficiency.

[0157] Link transmission characteristics describe the link transmission situation between resource nodes and nodes where dependent tasks reside, including link transmission rate fluctuations, latency jitter, and stability decay coefficients. Link transmission rate fluctuations reflect changes in the data transmission speed within the link; unstable transmission rates can affect the data transmission efficiency of tasks. Latency jitter refers to the degree of variation in data transmission delay; excessive latency jitter increases the uncertainty of data arrival time, affecting the real-time performance of tasks. The stability decay coefficient reflects the decreasing trend of link transmission stability over time; as time progresses, the link may be subject to interference from various factors, gradually reducing transmission stability.

[0158] Historical task adaptability characteristics refer to the performance of resource nodes when executing similar tasks, including the success rate of such tasks, resource utilization, and dependency chain collaboration efficiency. The success rate reflects the proportion of times a resource node has successfully completed similar tasks in the past; a higher success rate indicates better adaptability to this type of task. Resource utilization measures the effective use of resources by a resource node when executing tasks; a higher utilization rate indicates less resource waste. Dependency chain collaboration efficiency reflects the efficiency of a resource node in collaborating with other nodes to complete task dependency chains; efficient collaboration can accelerate the execution speed of the entire task chain.

[0159] When establishing a collaborative matching model between tasks and resource nodes, it is necessary to comprehensively consider the features of the above three dimensions. Machine learning algorithms, such as neural networks and decision trees, can be used to take these features as input and train the model to learn the matching relationship between tasks and resource nodes. For example, a multilayer perceptron (MLP) neural network can be used, taking resource supply features, link transmission features, and historical task adaptation features as neurons in the input layer. After nonlinear transformation in the hidden layer, the final output is a matching score between tasks and resource nodes.

[0160] As one implementation method, step S420 can be specifically implemented as the following steps S421~S425: Step S421: Extract the supply characteristics of resource nodes, including the dynamic change trend of remaining resources, resource elastic expansion capability and resource fragmentation rate, and generate a resource supply forecast curve for a future preset period through a time series forecast model.

[0161] The dynamic trend of remaining resources refers to the changing pattern of remaining computing resources and remaining storage resources of resource nodes over time. Resource elastic scaling capability refers to the ability of resource nodes to increase resource supply when needed, such as dynamically allocating virtual machines to increase computing resources in a cloud computing environment. Resource fragmentation rate refers to the degree to which resources are divided into small pieces; for storage resources, an excessively high fragmentation rate can affect data read and write efficiency. Time series forecasting models are models used to predict data trends over time, such as ARIMA (Autoregressive Integral Moving Average) and LSTM (Long Short-Term Memory). The future preset time period refers to the pre-defined time range for which resource supply forecasting will be performed. The resource supply forecast curve is a curve representing the change in resource supply over time within the future preset time period.

[0162] When extracting the supply characteristics of resource nodes, the first step is to collect historical remaining resource data for these nodes, including computing resource utilization and storage resource occupancy rates at different points in time. For the dynamic trends of remaining resources, time series analysis is used to observe the periodicity and trend characteristics of the data. For example, in a data center, the computing resources of some servers may exhibit a periodicity, with higher utilization rates during the day and lower utilization rates at night. This pattern can be identified through analysis of historical data.

[0163] Resource elastic scaling capability can be determined through testing and evaluation. For example, for a cloud computing platform that supports elastic scaling, testing can be conducted to determine the platform's ability to quickly allocate additional computing resources under different pressures, recording parameters such as allocation time and allocation amount. Resource fragmentation rate can be calculated using different methods for different types of resources. For storage resources, the fragmentation rate can be calculated by counting the number and size of available storage space divided into small blocks.

[0164] A time series forecasting model generates a resource supply prediction curve for a predetermined future time period. Taking the LSTM model as an example, historical remaining resource data is used as input, and the LSTM network is trained to learn the temporal dependencies of the data. The LSTM network has memory units, enabling it to capture long-term time series features. The trained model can predict the resource supply for a predetermined future time period based on the current resource status.

[0165] Step S422: Analyze the link transmission characteristics of resource nodes, including link transmission rate fluctuations, latency jitter, and stability attenuation coefficients with the nodes where dependent tasks are located, construct a time attenuation model of link transmission quality, and predict the change of transmission efficiency over time.

[0166] Link transmission rate fluctuation refers to the change in data transmission rate between resource nodes and nodes containing dependent tasks over different times. Delay jitter refers to the instability of data transmission delay, i.e., the magnitude of delay variation. The stability decay coefficient reflects the decreasing trend of link transmission stability over time. The time decay model of link transmission quality is a mathematical model used to describe the change in link transmission quality over time. The change in transmission efficiency over time refers to the change in data transmission efficiency on the link at different points in time.

[0167] When analyzing the link transmission characteristics of resource nodes, network monitoring tools are used to collect link transmission rate and latency data in real time. For example, the ping command and iperf tool are used to measure link latency and transmission rate. Statistical analysis is performed on the collected data, and the standard deviation of the link transmission rate is calculated to measure the degree of fluctuation; a larger standard deviation indicates a more unstable transmission rate. Latency jitter can be assessed by averaging the absolute values ​​of the differences in latency between adjacent data transmissions. The stability decay coefficient can be determined by long-term monitoring of the link transmission success rate. Assuming that the link transmission success rate is 95% in the initial stage, and gradually decreases to 90% over time, the stability decay coefficient can be obtained by fitting success rate data at multiple time points.

[0168] When constructing a time-degradation model for link transmission quality, factors such as link transmission rate fluctuations, latency jitter, and stability decay coefficients are considered. An exponential decay function can be used to describe the change in transmission efficiency over time. For example, the transmission efficiency E(t) can be expressed as E(t) = E0 × exp(-αt), where E0 is the initial transmission efficiency, α is the stability decay coefficient, and t is time. The model is then modified to incorporate link transmission rate fluctuations and latency jitter, making it more accurate in predicting the change in transmission efficiency over time. For example, when transmission rate fluctuations are large, the predicted transmission efficiency can be appropriately reduced.

[0169] Step S423: Query the historical task adaptation features of resource nodes, including the execution success rate, resource utilization rate and dependency chain collaboration efficiency of similar tasks, and convert the historical data into an adaptation score vector through feature engineering.

[0170] Historical task fit features refer to the performance of resource nodes when executing similar tasks in the past. Success rate of similar tasks refers to the proportion of times a resource node successfully completes a task similar to the current one; a higher success rate indicates better adaptability of the resource node to this type of task. Resource utilization rate refers to the ratio of the actual resources used by a resource node during task execution to the total resources; high resource utilization rate indicates that resources are being fully utilized. Dependency chain collaboration efficiency refers to the efficiency of a resource node in collaborating with other nodes to complete a task dependency chain; efficient collaboration can accelerate the execution speed of the entire task chain. Feature engineering refers to the preprocessing and transformation of raw data to extract more valuable feature information. The fit score vector converts historical task fit features into a vector form for subsequent matching calculations.

[0171] When querying the historical task adaptation characteristics of resource nodes, relevant data is extracted from the task execution record database. For the success rate of similar tasks, the total number of times similar tasks were executed and the number of successful executions are statistically analyzed to calculate the success rate. For example, in a software development project, the success rate of compilation tasks is obtained by statistically analyzing the number of successful and total executions of a code compilation task on a specific server node. Resource utilization can be calculated based on resource usage records during task execution, such as CPU usage and memory usage, to determine the average resource utilization. Dependency chain collaboration efficiency can be evaluated by analyzing the interaction time between nodes in the task dependency chain and the task completion time.

[0172] Feature engineering transforms historical data into a fit score vector. First, the fit features of each historical task are normalized, mapping feature values ​​from different ranges to the same interval, such as [0,1]. Then, weights are assigned to each feature, determined by its importance. For example, resource utilization might have a higher weight for computationally intensive tasks, while dependency chain efficiency might have a higher weight for tasks requiring multi-node collaboration. The normalized feature values ​​are multiplied by their corresponding weights and then summed to obtain the fit score for each resource node. The fit scores of all resource nodes are combined into a fit score vector for subsequent task-resource matching.

[0173] Step S424: Input the resource supply prediction curve, the link transmission quality attenuation model output, and the fit score vector into the collaborative matching model. Generate a comprehensive matching score through multimodal feature fusion. The higher the score, the better the collaborative fit between the task and the resource node.

[0174] The resource supply forecast curve reflects the resource supply situation of resource nodes within a preset future time period, helping to determine whether resource nodes have sufficient resources to execute tasks. The link transmission quality degradation model output describes the change in link transmission quality between resource nodes and nodes where dependent tasks reside over time; good link transmission quality is crucial for tasks requiring large amounts of data transmission. The fit score vector reflects the performance of resource nodes in historical execution of similar tasks, reflecting their adaptability to tasks. The collaborative matching model is used to comprehensively consider various features to match tasks with resource nodes. Multimodal feature fusion is the process of integrating different types of features (such as time series features, numerical features, etc.). The comprehensive matching score is a comprehensive evaluation of the collaborative fit between tasks and resource nodes; the higher the score, the better the collaborative effect.

[0175] When inputting the resource supply prediction curve, the link transmission quality degradation model output, and the fit score vector into the collaborative matching model, these features are first preprocessed to ensure they have the same dimensions and format. For example, the resource supply prediction curve and the link transmission quality degradation model output are sampled to obtain vectors of the same length. The fit score vector, however, retains its original dimensions.

[0176] In collaborative matching models, deep learning models such as Convolutional Neural Networks (CNNs) or Multilayer Perceptrons (MLPs) can be used for multimodal feature fusion. Taking the MLP model as an example, preprocessed feature vectors are concatenated together to form neurons in the input layer. The MLP model performs nonlinear transformations on the input features through multiple hidden layers, learning the complex relationships between features. In the output layer, the model outputs a comprehensive matching score.

[0177] For example, for a data analysis task requiring significant computing resources and data transmission, the resource supply prediction curve shows that a certain resource node has sufficient computing resources during task execution. The link transmission quality degradation model output indicates that the link transmission quality between this node and the task's node is good, and the fit score vector shows that this node has performed well in historical executions of similar data analysis tasks. After inputting these features into the collaborative matching model, the model will comprehensively consider these factors and give a high overall fit score, indicating that the resource node has good collaborative fit with the task.

[0178] Step S425: Sort the resource nodes based on the comprehensive matching score and generate a candidate list of resource nodes. The list includes the matching score and the contribution ratio of each dimension feature, which is used as the basis for conflict correction during task-resource pre-allocation simulation.

[0179] The comprehensive matching score is a holistic evaluation of the collaborative adaptability between tasks and resource nodes. The resource node candidate list is a list obtained by ranking resource nodes according to the comprehensive matching score, containing resource nodes that are potentially suitable for executing the task. The contribution percentage of each dimension feature refers to the proportion of resource supply features, link transmission features, and historical task adaptability features in the comprehensive matching score, reflecting the degree of influence of each feature on the final matching result. Task-resource pre-allocation simulation is the process of simulating and evaluating the allocation scheme of tasks and resource nodes before the actual task allocation. Conflict correction basis refers to the reference information used to adjust the allocation scheme when task and resource node allocation conflicts occur during the pre-allocation simulation.

[0180] When sorting resource nodes based on their overall matching score, a sorting algorithm (such as bubble sort or quicksort) is used to arrange the resource nodes from highest to lowest according to their overall matching score. A candidate list of resource nodes is generated, and each resource node in the list includes not only its overall matching score but also the contribution percentage of each dimension of its features.

[0181] The contribution percentage of each dimension feature can be calculated using sensitivity analysis or feature importance assessment methods. Taking sensitivity analysis as an example, the value of each dimension feature is changed individually, and the changes in the overall matching score are observed.

[0182] The resource node candidate list is used for task-resource pre-allocation simulation. During the simulation, based on the task requirements and the status of resource nodes, attempts are made to assign tasks to resource nodes in the candidate list. When conflicts occur, such as multiple tasks competing for a single resource node, or a resource node's resources being insufficient to meet the needs of all assigned tasks, conflict correction can be performed based on the contribution ratio of each dimension feature. For example, if the contribution ratio of the resource supply feature is high, the remaining resource situation of the resource node is given priority, and the task allocation scheme is adjusted to ensure that tasks receive sufficient resource support.

[0183] Step S430: Process all tasks to be scheduled through the task priority evaluation mechanism to generate a task execution order table containing a dynamic priority sorting sequence and adjustment basis. The sorting sequence is updated in real time as the resource status changes.

[0184] The task priority evaluation mechanism is used to determine the execution order of tasks. It comprehensively considers factors such as task centrality, critical path location, and the real-time state of the resource dependency subgraph. Tasks awaiting scheduling are those waiting to be assigned to resource nodes for execution. The dynamic priority ranking sequence is a sequence obtained by sorting tasks according to their dynamic priorities. As resource status changes, task priorities are adjusted accordingly, and the ranking sequence is updated in real time. The adjustment basis refers to the information used when adjusting task priorities, such as resource node load fluctuations and the blocking risk of task dependency chains. The task execution order table is a table that records the task execution order, containing information such as task identifiers, dynamic priority ranking sequences, and adjustment basis.

[0185] When processing all tasks to be scheduled using the task priority evaluation mechanism, the initial priority is first generated in the basic priority calculation module by combining the centrality characteristics of the task nodes and the critical path position. As mentioned earlier, centrality characteristics include degree centrality, betweenness centrality, and compact centrality, which reflect the importance of task nodes in the task network from different perspectives. The critical path position indicates whether the task is located on a path that plays a crucial role in the completion time of the entire task sequence.

[0186] The adjustment module continuously monitors the load fluctuations of resource nodes associated with tasks in the resource dependency subgraph. When the load of an associated resource node exceeds a preset threshold, the priority of tasks dependent on that node is adjusted downwards; when the load is below the threshold, it is adjusted upwards. Simultaneously, a task dependency chain blocking risk assessment is introduced, analyzing the waiting time of downstream tasks and the completion progress of dependent tasks. When the blocking risk exceeds a preset risk threshold, the priority adjustment magnitude for the current task is increased.

[0187] The initial priority score and adjustment values ​​are integrated to generate a dynamic priority score. All tasks to be scheduled are sorted according to the dynamic priority score to obtain a dynamic priority ranking sequence. The ranking results and corresponding adjustment criteria are recorded in the task execution order table.

[0188] As resource status changes, such as the load of resource nodes or the transmission status of network links, the task priority evaluation mechanism updates task priorities in real time. For example, if the CPU utilization of a resource node suddenly increases and exceeds a preset threshold, the priority of tasks that depend on that node will be downgraded, and the sorting sequence in the task execution order table will be adjusted accordingly to ensure that the execution order of tasks always matches the resource status, thereby improving the efficiency and rationality of task scheduling.

[0189] Step S440: Based on the task execution order table, the multi-dimensional resource collaboration matching mechanism is called sequentially to evaluate the multi-dimensional matching degree of each task execution resource node and generate a candidate list of resource nodes with weighted matching degree.

[0190] The task execution order table records the execution order of tasks, providing a basis for the priority of tasks in resource allocation. The multi-dimensional resource collaborative matching mechanism comprehensively considers the supply characteristics of resource nodes, link transmission characteristics, and historical task adaptability characteristics to establish a collaborative matching model between tasks and resource nodes. Multi-dimensional matching degree evaluation assesses the adaptability of resource nodes to tasks across multiple dimensions. The matching degree-weighted candidate list of resource nodes is a list obtained by weighting and sorting resource nodes according to their matching degree with tasks, containing resource nodes suitable for executing tasks.

[0191] When the multi-dimensional resource collaborative matching mechanism is invoked sequentially based on the task execution sequence list, the matching degree of resource nodes for each task is evaluated according to the order in the task execution sequence list. For each task, its resource requirement information, such as computing resource requirements, storage resource requirements, and data transmission requirements, is extracted first.

[0192] In the multi-dimensional resource collaborative matching mechanism, the supply characteristics of resource nodes are analyzed. As mentioned earlier, supply characteristics include the dynamic trend of remaining resources, resource elastic expansion capability, and resource fragmentation rate. A time series prediction model is used to generate resource supply prediction curves for a preset future time period, predicting the resource availability of resource nodes during task execution.

[0193] Analyze the link transmission characteristics of resource nodes, including link transmission rate fluctuations, latency jitter, and stability attenuation coefficients with nodes where dependent tasks reside. Construct a time-degradation model for link transmission quality to predict how transmission efficiency changes over time, ensuring efficient data transmission during task execution.

[0194] Query the historical task adaptation characteristics of resource nodes, including the success rate of similar tasks, resource utilization, and dependency chain collaboration efficiency. Through feature engineering, convert historical data into adaptation score vectors to reflect the performance of resource nodes when executing similar tasks.

[0195] The resource supply prediction curve, the link transmission quality attenuation model output, and the fit score vector are input into the collaborative matching model. A comprehensive matching score is generated through multimodal feature fusion. Resource nodes are then weighted and ranked according to the comprehensive matching score to generate a candidate list of resource nodes with weighted matching score.

[0196] Step S450: Based on the candidate list of resource nodes and the task dependencies, perform task-resource pre-allocation simulation, verify the resource coordination of the dependency chain of the pre-allocation scheme, correct conflicting pre-allocations, and generate the final optimized task allocation scheme.

[0197] The resource node candidate list is a list sorted according to the matching degree between tasks and resource nodes, containing resource nodes suitable for task execution. Task dependency refers to the order and dependencies between tasks, such as the execution of one task depending on the completion of another. Task-resource pre-allocation simulation is the process of simulating and evaluating the allocation scheme of tasks and resource nodes before actual task allocation. Dependency chain resource coordination refers to the coordination of resource usage among tasks in the task dependency chain, ensuring that each task receives sufficient resource support and that resource allocation does not cause blockage in the dependency chain. Conflict pre-allocation refers to unreasonable allocation situations that occur during the pre-allocation process, such as multiple tasks competing for a resource node simultaneously, or a resource node having insufficient resources to meet the needs of all allocated tasks. The final optimized task allocation scheme is the best task and resource node allocation scheme obtained after pre-allocation simulation and conflict correction.

[0198] When performing task-resource pre-allocation simulation based on the resource node candidate list and task dependencies, tasks are first retrieved sequentially from the task execution order list. Then, a suitable resource node is selected for each task according to the resource node candidate list. During the selection process, task dependencies are considered to ensure that dependent tasks are executed before the tasks they depend on.

[0199] For example, in a task chain comprising three tasks: data acquisition, data processing, and data analysis, the data acquisition task must be completed before the data processing task can proceed, and the data analysis task can only proceed after the data processing task is completed. During the pre-allocation simulation, resource nodes are first selected for the data acquisition task. Then, based on the completion status of the data acquisition task and the status of the resource nodes, suitable resource nodes are selected for the data processing task. Finally, resource nodes are selected for the data analysis task.

[0200] Verify the resource coordination of the dependency chain in the pre-allocation scheme. Check whether the resource requirements of each task in the dependency chain can be met, and whether the resource allocation of resource nodes is reasonable. For example, check whether a resource node is simultaneously allocated to tasks in multiple dependency chains, leading to resource shortages. If it is found that a task's dependent tasks do not receive sufficient resource support, it may cause the task to fail to execute on time, affecting the execution efficiency of the entire dependency chain.

[0201] Correcting Conflict Pre-allocation: When conflict pre-allocation occurs, adjustments are made based on the contribution percentage of each dimension of the resource node candidate list. As mentioned earlier, if the contribution percentage of the resource supply characteristic is high, the remaining resource situation of the resource node is given priority, and the task allocation scheme is adjusted to allocate tasks to nodes with more abundant resources. Alternatively, based on task priority, the resource needs of high-priority tasks are prioritized, and the allocation of low-priority tasks is adjusted accordingly.

[0202] After multiple pre-allocation simulations and conflict corrections, an optimized task allocation scheme is finally generated. This scheme ensures that tasks execute smoothly according to their dependencies while making full use of the resources of resource nodes, thus improving the efficiency and reliability of task execution. For example, in a large-scale cloud computing environment, this optimized task allocation scheme can rationally distribute multiple complex tasks across different server nodes, maximizing the resource utilization of the entire system while ensuring that tasks are completed on time and efficiently.

[0203] Figure 2 A hardware entity diagram of a computer system provided as an embodiment of the present invention, such as... Figure 2 As shown, the hardware entity of the computer system 1000 includes a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can run on the processor 1001, and the processor 1001 executes the program to implement the steps in the method of any of the above embodiments.

[0204] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A hybrid task scheduling method for reconstructing a dual dependency graph in a computing power network, characterized in that, The method includes: The task submission information and historical execution records in the computing power network are analyzed to obtain the direct and indirect dependencies between tasks. An initial task dependency graph is constructed based on the direct and indirect dependencies. Real-time running status data and resource usage information of each node in the computing power network are collected to construct a real-time resource dependency graph, resulting in the initial structure of the dual dependency graph. The initial task dependency graph and the real-time resource dependency graph are subjected to topological structure feature extraction to obtain a task topological feature set and a resource topological feature set. The task topological feature set and the resource topological feature set are compared to obtain topological difference features. A dependency relationship correction parameter set is generated based on the topological difference features. Based on the dependency correction parameter set, the initial structure of the dual dependency graph is iteratively reconstructed. In each iteration, the edge weights and node connections in the graph structure are adjusted according to the corresponding parameters in the dependency correction parameter set to obtain the reconstructed dual dependency graph. Based on the reconstructed dual dependency graph, a hybrid task scheduling strategy is constructed. The hybrid task scheduling strategy includes a task priority evaluation mechanism and a resource coordination matching mechanism. The task priority evaluation mechanism determines the execution order of tasks, and the resource coordination matching mechanism allocates tasks to corresponding computing power nodes to generate an optimized task allocation scheme.

2. The method according to claim 1, characterized in that, The process involves analyzing task submission information and historical execution records in the computing power network to obtain direct and indirect dependencies between tasks. An initial task dependency graph is constructed based on these dependencies. Additionally, real-time running status data and resource usage information of each node in the computing power network are collected to construct a real-time resource dependency graph, resulting in an initial structure of a dual dependency graph, including: The task submission information is structured and parsed to extract task identifiers, input data items, and output data items. By associating the generation relationships between the input data items and output data items of different tasks, the data dependencies between tasks are determined as components of the direct dependencies. Extract the task call sequence from the historical execution record, identify the preceding call relationship in the task execution process, generate the call dependency relationship between tasks as a component of the direct dependency relationship, and merge the data dependency relationship and the call dependency relationship to obtain the complete direct dependency relationship; Transitive closure calculation is performed on the complete direct dependency relationship to generate a candidate set of indirect dependencies. The candidate set of indirect dependencies is filtered by combining the fluctuation of task execution time interval and task success rate in the historical execution record to obtain indirect dependencies that meet the credibility requirements. Based on the complete direct dependency relationship and the indirect dependency relationship, an initial task dependency graph is constructed with task identifiers as nodes and dependency relationships as directed edges. The directed edges are assigned dependency type identifiers and dependency strength weights through task execution history data. The computing resource load, storage resource load, and communication link load of each node in the computing power network are collected as real-time operating status data. The communication link delay and resource allocation history between nodes are combined as resource occupancy information. A real-time resource dependency graph is constructed with node identifiers as vertices and resource supply relationships as directed edges. Resource transmission test data is used to assign resource transmission rate and link stability indicators to the directed edges. The initial task dependency graph and the real-time resource dependency graph are mapped to nodes to establish the association between task nodes and resource nodes, and then merged to obtain an initial structure of a dual dependency graph containing task dependency subgraphs and resource dependency subgraphs.

3. The method according to claim 2, characterized in that, The step involves performing structured parsing of the task submission information to extract task identifiers, input data items, and output data items. By associating the generation relationships between the input and output data items of different tasks, the data dependencies between tasks are determined as components of direct dependencies, including: The task submission information is parsed to extract the unique task identifier for each task, as well as the input and output data items related to the task, and to establish the correspondence between tasks and data items. Iterate through all the output data items of all tasks, build a data item-task mapping table, record the task corresponding to each output data item, and obtain the source task index of the data item; For each input data item of a task, the data item-task mapping table is queried to determine the generation task of the input data item. When there is a corresponding generation task for the input data item, a directed connection from the generation task to the current task is created and marked as the data dependency direction. All data dependency directions are deduplicated, duplicate directed connections are deleted, and by analyzing the transmission time of data items in the past, a weight value representing the degree of dependency is assigned to each directed connection to obtain the data dependency table between tasks.

4. The method according to claim 2, characterized in that, The transitive closure calculation is performed on the complete direct dependencies to generate a candidate set of indirect dependencies. This candidate set is then filtered by combining the fluctuations in task execution time intervals and task success rates from the historical execution records to obtain indirect dependencies that meet the reliability requirements, including: The complete direct dependencies are converted into an adjacency matrix, and the transitive closure is calculated through matrix multiplication to generate a candidate matrix containing all possible indirect paths. The matrix elements indicate whether there are indirect dependency paths between tasks. The candidate matrix is ​​converted into a candidate set of indirect dependencies, and the source task, target task, and number of direct dependency edges contained in each candidate path are recorded. Extract the continuous execution time interval between the source task and the target task from the historical execution record, calculate the fluctuation degree of the time interval, and retain the indirect dependency candidate path when the fluctuation degree is within a preset stable range. The success rate of a path is obtained by calculating the ratio of the number of successful tasks executed through candidate paths in the historical execution records to the total number of times. When the success rate is higher than a preset confidence threshold, the candidate path is marked as a valid indirect dependency.

5. The method according to claim 1, characterized in that, The process involves extracting topological structure features from the initial task dependency graph and the real-time resource dependency graph to obtain a task topological feature set and a resource topological feature set. The task topological feature set is then compared with the resource topological feature set to obtain topological difference features. Based on these topological difference features, a dependency correction parameter set is generated, including: Node centrality analysis is performed on the initial task dependency graph. By calculating the number of direct dependency edges of task nodes, the frequency of being on the shortest dependency path, and the average path length to other nodes, node centrality features describing the importance of nodes are generated. Traverse the directed edges of the initial task dependency graph, calculate the distribution characteristics of the edge weights, including the central tendency, dispersion and distribution pattern of the weights, identify the critical path with the largest sum of edge weights in the graph, and extract the path length, number of nodes and average weight as path features. The node centrality feature and the path feature are combined to obtain the task topology feature set; The resource distribution characteristics of the real-time resource dependency graph are analyzed, the remaining computing resources, remaining storage resources, and available communication resources of the resource nodes are calculated, the transmission efficiency and stability of the links between nodes are analyzed, and resource node characteristics and link quality characteristics are generated. Calculate the network density and clustering degree of the real-time resource dependency graph, and combine the resource node features, link quality features, network density, and clustering degree to obtain a set of resource topology features; The task topology feature set and the resource topology feature set are compared according to the corresponding dimensions, the degree of difference in feature values ​​is calculated, topology difference features are generated, and a correction coefficient is assigned to each feature dimension based on the degree of difference to obtain a dependency correction parameter set.

6. The method according to claim 5, characterized in that, The process involves traversing the directed edges of the initial task dependency graph, calculating the distribution characteristics of the edge weights, including the central tendency, dispersion, and distribution pattern of the weights, identifying the critical path with the largest sum of edge weights, and extracting the path length, number of nodes, and average weight as path features, including: Traverse all directed edges in the initial task dependency graph, collect edge weights to obtain a weight set, calculate the arithmetic mean of the weight set to represent the central tendency, calculate the mean of the sum of squares of the deviations of each weight value from the mean to represent the dispersion, and calculate the skew direction and degree of the weight value distribution to represent the distribution shape. By traversing the initial task dependency graph using depth-first search, the starting task node with an in-degree of 0 and the ending task node with an out-degree of 0 are identified, and a combination of starting and ending node pairs is established. For each start-end node pair, calculate the cumulative edge weights of all possible paths, and mark the path with the largest cumulative weight as a critical path candidate; The number of directed edges in the critical path is extracted as the path length. The total number of task nodes contained in the path is counted. The ratio of the cumulative edge weight to the path length is calculated as the average weight. These are combined to obtain the path features that describe the path characteristics.

7. The method according to claim 5, characterized in that, The process of performing resource distribution characteristic analysis on the real-time resource dependency graph, calculating the remaining computing resources, remaining storage resources, and available communication resources of resource nodes, analyzing the transmission efficiency and stability of links between nodes, and generating resource node characteristics and link quality characteristics includes: Collect the total resource capacity and used resource amount of each resource node in the real-time resource dependency graph, calculate the difference between the total resource capacity and the used resource amount, and obtain the remaining computing resources and remaining storage resources; The real-time transmission rate and data transmission success rate of the communication link between resource nodes are obtained. The ratio of the actual transmission rate of the link to the theoretical maximum rate is used to represent the transmission efficiency. The stability is represented by the ratio of the number of successful transmissions to the total number of transmissions in continuous transmission tests. The remaining computing resources and remaining storage resources of each resource node are normalized according to resource type to obtain resource node characteristics, which describe the resource supply capacity of the node. The transmission efficiency and stability of each communication link are grouped according to the link identifier to obtain the link quality characteristics.

8. The method according to claim 1, characterized in that, The iterative reconstruction operation of the initial structure of the dual dependency graph based on the dependency correction parameter set involves adjusting the edge weights and node connections in the graph structure according to the corresponding parameters in the dependency correction parameter set during each iteration to obtain the reconstructed dual dependency graph, including: Initialize the iteration counter and the termination condition, which includes an upper limit on the number of iterations and a threshold for the amount of topological feature change. The iteration stops when either condition is met. In the current iteration, the task topology correction coefficients in the dependency correction parameter set are extracted, and the centrality features and directed edge weights of the corresponding nodes in the initial task dependency graph are adjusted to improve the matching degree between the task topology features and the resource topology features. Extract the resource topology correction coefficients from the dependency correction parameter set, and adjust the resource capacity characteristics and link quality characteristics of the corresponding nodes in the real-time resource dependency graph to improve the matching degree between resource supply capacity and task requirements. Traverse the node pairs in the initial structure of the dual dependency graph, identify conflicting paths where the task dependency direction and resource dependency direction are opposite, and eliminate path conflicts by comparing the weight values ​​of conflicting edges and deleting conflicting edges with smaller weights. Identify redundant dependency paths with shorter alternative paths and smaller weight differences, simplify the path structure by merging the weight values ​​of redundant edges, and preserve the core dependencies; For the critical paths in the task topology feature set, increase the weight value of the directed edges on the path to strengthen the connection strength of the critical task chain, complete the graph structure adjustment of the current iteration, repeat the iteration until the preset termination condition is met, and obtain the reconstructed dual dependency graph.

9. The method according to claim 8, characterized in that, The process of traversing node pairs in the initial structure of the dual dependency graph, identifying conflicting paths where the task dependency direction and resource dependency direction are opposite, and eliminating path conflicts by comparing the weight values ​​of conflicting edges and deleting conflicting edges with smaller weights includes: Traverse all node pairs (u, v) in the initial structure of the dual dependency graph, check if there is a task dependency edge from u to v and a resource dependency edge from v to u at the same time, or a resource dependency edge from u to v and a task dependency edge from v to u, and mark them as conflict path candidates. For each candidate conflict path, extract the weight values ​​of the task-dependent edge and the resource-dependent edge. When the weight value of the task-dependent edge is less than the weight value of the resource-dependent edge, the task-dependent edge is determined as a conflict edge to be deleted; when the weight value of the resource-dependent edge is less than the weight value of the task-dependent edge, the resource-dependent edge is determined as a conflict edge to be deleted; when the two weight values ​​are equal, the object to be deleted is determined based on the success rate of historical paths in progress, and the edge with the lower success rate is deleted. Perform the conflict edge deletion operation, update the node connection relationship and edge attribute list of the dual dependency graph, and record the deletion operation information for iterative verification to ensure that there are no bidirectional conflict paths in the graph structure after deletion.

10. A computer system comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 9.

Images