A cooperative scheduling method

Abstract

This invention discloses a collaborative scheduling method, comprising the following steps: 1) Deploying a central node controller in the target network, and deploying a collaborative scheduler on each node in the target network; 2) The central node controller periodically sends monitoring requests to each node; the collaborative scheduler reports the status information and link load of its node to the central node controller; 3) The central node controller constructs a global resource view based on the status information and link load data, and pushes it to each node; 4) The collaborative scheduler on the source node calculates multiple candidate paths based on the destination IP address of the data flow, the global routing link table of the target network, and the network topology, and sends them to the central node controller; 5) The central node controller calculates the comprehensive score of each candidate path, selects the optimal path, and sends it to the source node for data transmission. This invention significantly improves the efficiency and reliability of network communication.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of network communication technology and information security, and relates to a collaborative scheduling method. Background Technology

[0002] With the rapid development of cloud computing, big data, and IoT technologies, the architecture and performance requirements of data center networks (DCNs) are increasingly demanding. In highly connected DCNs, servers frequently engage in one-to-one and many-to-many communication. Efficiently utilizing multiple paths between servers is crucial for improving network performance. Shortest path-based transmission methods suffer from low multi-path utilization efficiency, uneven network load, and frequent data communication congestion, leading to degraded data scheduling performance. Therefore, a method is needed that can analyze node status and link load in real time and dynamically select the optimal transmission path accordingly.

[0003] Cooperative scheduling currently faces multiple challenges closely related to network models. First, sudden data surges place higher demands on the system's storage and computing capabilities, requiring scheduling systems to possess efficient data processing and distribution capabilities to avoid data backlog and processing delays. Second, the dynamic nature of network bandwidth increases scheduling complexity, requiring scheduling algorithms to adapt to bandwidth changes in real time, optimize resource allocation, and ensure the smooth execution of critical tasks. Simultaneously, the randomness of network congestion exacerbates the uncertainty of network conditions, requiring cooperative scheduling algorithms to possess intelligent prediction and adaptive adjustment capabilities to effectively cope with network congestion and improve overall system performance and stability. Furthermore, the diversification of network objectives necessitates more flexible and adaptable scheduling strategies to meet the diverse needs of different users and achieve optimal resource allocation and maximum utilization. Summary of the Invention

[0004] To address the problems existing in the prior art, the purpose of this invention is to provide an efficient collaborative scheduling method. This invention focuses on developing more intelligent and adaptive scheduling algorithms based on node status and link load conditions, improving the system's ability to cope with data, bandwidth, and congestion, while also emphasizing the optimization of resource management and task allocation strategies to ensure efficient and stable collaborative scheduling in various network environments.

[0005] To achieve the objective, the technical solution adopted by this invention is as follows:

[0006] (1) System initialization: This invention is based on a mesh structure to achieve separation of control and forwarding and centralized global control. The system includes a central node controller and data source / destination nodes. The central node controller is responsible for collecting network resource information, including node status and link load, and calculating the optimal transmission path accordingly.

[0007] (2) Global Resource View Construction: The central node controller periodically collects network node status information and link load information. Node status information includes CPU utilization, memory usage, and port status, while link load information includes bandwidth utilization, latency, and packet loss rate. This information is used to evaluate the processing capacity and health status of nodes, as well as the transmission capacity and reliability of links, providing a basis for path selection.

[0008] (3) Optimal path calculation: The central node controller calculates the optimal path between a pair of nodes based on the real-time node status and link load information and the preset routing algorithm.

[0009] (4) Data transmission and monitoring: The central node controller sends the calculated optimal path to the source node, which then forwards it to the next-hop node according to the optimal path. At the same time, the central node controller continuously monitors the network status and dynamically adjusts the transmission path as needed to adapt to network changes.

[0010] The key points or inventive aspects of this invention are mainly reflected in the following aspects:

[0011] (1) Real-time dynamic monitoring and analysis

[0012] This invention introduces a real-time dynamic monitoring mechanism that can continuously and in real-time collect node status and link load information in the network. This real-time capability enables the system to respond quickly to changes in network status, thereby making more accurate path selection decisions.

[0013] (2) Calculation of the optimal path considering multiple factors

[0014] When selecting the optimal transmission path, this invention not only considers single factors such as traditional path length and bandwidth utilization, but also comprehensively considers multiple factors such as available link bandwidth, path latency, link load balancing, and node processing capabilities. This multi-objective optimization algorithm makes the calculated optimal path more comprehensive and reasonable, and can better meet the diverse needs of network transmission.

[0015] (3) Intelligent dynamic adjustment capability

[0016] This invention possesses intelligent dynamic adjustment capabilities, enabling it to rapidly recalculate and distribute new optimal path information to devices along the route when network conditions change (such as link failures or node congestion). This dynamic adjustment mechanism allows the network to maintain efficient and stable operation, reducing service interruptions and data loss caused by network failures and performance issues.

[0017] The technical solution of this invention is as follows:

[0018] A cooperative scheduling method, comprising the following steps:

[0019] 1) Deploy a central node controller in the target network, and deploy a cooperative scheduler on each node in the target network;

[0020] 2) The central node controller periodically sends monitoring requests to each node in the target network; each time the cooperative scheduler receives the monitoring request, it reports the status information and link load of its node to the central node controller.

[0021] 3) The central node controller constructs a global resource view based on the status information and link load data of each node, and pushes it to each node;

[0022] 4) The coordinating scheduler on the source node in the target network obtains the global routing link table and network topology of the target network from the global resource view; then, based on the destination IP address of the data flow and the global routing link table and network topology of the target network, it calculates multiple candidate paths and sends them to the central node controller.

[0023] 5) The central node controller calculates the comprehensive score of each candidate path, selects the optimal path based on the comprehensive score of each candidate path, and sends it to the source node for data transmission.

[0024] Furthermore, the central node controller calculates a comprehensive score for each candidate path based on latency, bandwidth capacity, reliability, security, and cost-effectiveness.

[0025] Furthermore, the central node controller is based on an evaluation function. Calculate the i-th candidate path P i The overall score result is Score (P). i ); where D max D min These are the maximum and minimum delay times among the multiple candidate paths, w D Weights for delay time; B max B min These are the maximum and minimum bandwidth capacities among the multiple candidate paths, w B The weight of bandwidth capacity; R max ,R min These are the maximum and minimum reliability values ​​among the multiple candidate paths, w R Weights for reliability; S max w is the maximum security value among the multiple candidate paths. S Weights for security; C max C minThese are the maximum and minimum cost-effectiveness values ​​among the multiple candidate paths, w C Weighting based on cost-effectiveness.

[0026] Furthermore, the status information includes, but is not limited to, CPU utilization, memory usage, and port status.

[0027] Furthermore, the link load data of the node is obtained using the Simple Network Management Protocol (SNMP).

[0028] Furthermore, the link load data includes bandwidth utilization, latency, and packet loss rate.

[0029] Furthermore, the central node controller monitors the amount of data received and sent by each node, the frequency of transmission and reception, the data transmission delay time, the bandwidth rate, the response time, and the number of data transmission interruptions. Based on the monitoring data, it constructs and updates an anomaly curve in real time, and then identifies bottlenecks or potential problems in the target network based on the anomaly curve.

[0030] Furthermore, the central node controller dynamically evaluates the effectiveness of the current optimal path based on the monitored network state changes. If the current optimal path is no longer effective, it recalculates and selects a new optimal path.

[0031] Furthermore, the method for calculating multiple candidate paths is as follows: First, multiple indicator factors are obtained from the global resource view and the importance value of each indicator factor is determined. Multiple candidate paths from the source node to the destination node are generated based on the global routing link table and network topology of the target network. Then, a weighted graph of candidate paths is generated by assigning weights to each candidate path according to the importance value of each indicator factor. Then, an algorithm in graph theory is used to search and evaluate the merits of each candidate path in the weighted graph of candidate paths, and multiple candidate paths are generated based on the evaluation results.

[0032] Furthermore, the indicator factors include link status, bandwidth usage, and historical transmission records; the algorithm in the graph theory is Dijkstra's algorithm, A* algorithm, Bellman-Ford algorithm, or a variant thereof.

[0033] The advantages of this invention are as follows:

[0034] (1) This method can intelligently allocate network traffic, avoiding overload of some links or nodes due to excessive traffic, while other links or nodes remain relatively idle. By balancing network load, the overall throughput and stability of the network can be effectively improved.

[0035] (2) When a node is in poor condition (such as failure, maintenance or performance degradation), the path selection algorithm based on node status and link load can quickly replan the transmission path, bypass the problematic node or link, and ensure the continuity and reliability of data transmission.

[0036] This invention calculates the optimal transmission path from source to destination based on the status of each node and the load of each link, thereby achieving efficient and accurate transmission of massive amounts of data and greatly improving the efficiency and reliability of network communication. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0039] (1) System initialization

[0040] First, a central node controller is deployed at the core or logical center of the network, serving as the management and decision-making hub for the entire network. This controller is responsible for monitoring global resources, data analysis, path calculation and optimization, and the unified scheduling and distribution of path information. The central node controller must possess high availability, high reliability, and powerful data processing capabilities to handle the complex and ever-changing data flows and requests in a large-scale network environment.

[0041] Secondly, this method employs a distributed deployment, deploying a collaborative scheduler on each node in the network. These nodes, located in different physical spaces, form crucial nodes on the data transmission path, directly impacting data transmission efficiency and reliability. The collaborative schedulers work closely with the central node controller, receiving and executing path information and scheduling instructions issued by the controller. Simultaneously, they are responsible for collecting local node status information and link load data, reporting them to the central node controller in real time for global analysis and optimization. By deploying collaborative schedulers on nodes, the status information and link load of the local transmission node can be proactively reported in real time. The central node controller updates the global routing links in real time based on the information reported by the transmission nodes and promptly distributes the information to the collaborative schedulers on each node, ensuring that each transmission node always maintains the latest global routing link table. This enables fine-grained control and dynamic adjustment of the data transmission path, further improving the flexibility and efficiency of data transmission.

[0042] Finally, during system deployment, reasonable network planning and policy configuration ensure that at least one reachable path exists between any two nodes. This requires full consideration of network connectivity and redundancy during network design to avoid network outages caused by single points of failure or link congestion. Leveraging the global perspective and intelligent algorithms of the central node controller, the network topology can be dynamically evaluated and optimized, potential path problems can be identified and resolved in a timely manner, and the continuity and stability of data transmission can be ensured.

[0043] (2) Construction of global resource view

[0044] First, we established a precise monitoring period (e.g., every second or every minute). Through this periodic monitoring mechanism, the central node controller periodically sends detailed monitoring requests to every node in the network. These requests aim to collect critical status information, including but not limited to CPU utilization, memory usage, and port status, thereby providing a comprehensive understanding of the operational status of each node.

[0045] Secondly, in order to accurately capture the real-time performance of the link, we use technologies such as real-time data stream monitoring and SNMP (Simple Network Management Protocol) to obtain the link load status in real time. Specific indicators include bandwidth utilization, latency, and packet loss rate. These indicators play an irreplaceable role in evaluating the transmission efficiency and reliability of the link.

[0046] Finally, the central node controller integrates the collected node information and link load data to construct a comprehensive and accurate global resource view, which is then pushed to all nodes. This aims to enable node managers to clearly understand their node's scheduling status in the global resource allocation process, even in complex network environments. This view not only provides a solid foundation for subsequent path calculations but also allows network administrators to intuitively understand the resource allocation and operational status of the entire network, providing strong support for network optimization and decision-making.

[0047] (3) Optimal path calculation

[0048] First, the source node's coordinator obtains the global routing link table and network topology from the global resource view. Based on the destination IP address of the data flow, it calls the intelligent routing algorithm, as shown in the pseudocode below. Combining the global routing link table, it obtains the best link. This algorithm comprehensively considers various factors such as the link length of the network topology, the number of routing points, the link health status, the bandwidth utilization rate, and the historical transmission timeliness, and calculates multiple candidate paths.

[0049] The intelligent routing algorithm collects necessary indicator data from the global resource view and network topology, including link status, bandwidth usage, and historical transmission records. This data is then cleaned, integrated, and preprocessed for algorithm use. Based on the importance of each indicator factor, and according to the target network's global routing link table and network topology, multiple candidate paths from the source node to the destination node are generated. Then, a weighted graph of candidate paths is generated by assigning weights to each candidate path based on the importance of each indicator factor. Finally, graph theory algorithms (such as Dijkstra's algorithm, A* algorithm, Bellman-Ford algorithm, or their variants) are used to search for the optimal path in the weighted graph. These algorithms can evaluate the quality of paths based on weights (i.e., the result after comprehensively considering multiple factors) and generate multiple candidate paths.

[0050] The alternative path weighted graph identifies key performance indicators (such as link status, bandwidth usage, and historical transmission records) and assigns each an importance value (a weight between 0 and 1, representing the proportion of importance of that factor in the overall evaluation). For each alternative path, the overall weight is calculated based on the importance values ​​of each indicator and the specific values ​​of that path for each indicator, and is achieved through a weighted summation. Here, n represents the number of indicator factors involved in the path. Since different alternative paths involve different numbers of indicator factors, default values ​​(such as 0 or a neutral value) are assigned to uninvolved indicator factors to ensure that all paths can be compared within the same framework. The weighted graph of alternative paths uses the calculated weights of each alternative path to assign corresponding weights or labels to each node (source node, intermediate node, destination node) and edge (path segment) in the graph.

[0051] Secondly, in order to select the most suitable path for the current transmission needs from many candidate paths, a multi-dimensional evaluation function is used to quantify and score each candidate path, including but not limited to path latency, bandwidth capacity, reliability (such as link failure rate), security (such as encryption support), and cost-effectiveness. The evaluation function can objectively reflect the advantages and disadvantages of each path.

[0052] The evaluation function defines the metrics and corresponding weights for each dimension, with latency (D) in milliseconds (ms) and weights w. D Bandwidth capacity (B): unit Mbps, weight w B Reliability (R): can be represented by the reciprocal of the link failure rate (i.e., MTBF, Mean Time Between Failures) or the negative logarithm of the failure rate, with a weight w. R Security (S): A score based on factors such as encryption support and security protocols (e.g., 0-10 points), with a weight w. SCost-effectiveness (C): Considering deployment and maintenance costs, this can be quantified as a cost-performance ratio, with a weight w. C Let path Pi be rated D in each dimension. i B i ,R i ,S i C i Then path P i The overall score is:

[0053]

[0054] Among them, D max D min B max B min ,R max ,R min ,S max C max C min These are the maximum and minimum values ​​of the corresponding dimensions in all candidate paths, used for normalization to ensure that the scores of all dimensions are between 0 and 1 (cost-effectiveness is an inverse indicator, so subtraction and inversion are used).

[0055] Then, based on the comprehensive score results from the evaluation function, the central node controller sorts and filters the data, ultimately selecting the optimal path. This optimal path not only provides the best data transmission performance under current conditions but also adapts to future network changes to a certain extent, ensuring the continuity and stability of data transmission. Through this process, the central node controller can intelligently manage network traffic, optimize data transmission efficiency, and improve overall network performance.

[0056] (4) Data transmission and monitoring

[0057] First, the source node transmits data according to the optimal path calculated by the central node controller, ensuring that the data can reach the destination node along the most efficient and reliable path.

[0058] Secondly, the central node controller continuously and proactively monitors the entire network status, including link load, bandwidth utilization, latency variations, and any potential security threats. By collecting and analyzing this data in real time, such as node overload in receiving and sending data, excessively high transmission frequency, severe data transmission latency, bandwidth rates below 50% of normal network speeds, response timeouts, and data transmission interruptions, combined with historical data from each node, the central node controller can quickly identify any bottlenecks or potential problems in the network based on abnormal curves of these indicators.

[0059] Finally, based on the monitored changes in network status, the central node controller dynamically evaluates the effectiveness of the current optimal path. If it detects that the current path is no longer the optimal choice (e.g., due to link failure, bandwidth congestion, or increased costs), the controller immediately triggers a path adjustment mechanism. It re-invokes the routing algorithm, incorporates the latest network status information, and recalculates and selects a new optimal path.

[0060] The intelligent network optimal transmission path selection system and method proposed in this invention, based on node status and link load, achieves reasonable, efficient, and balanced allocation of network resources by monitoring network status in real time and dynamically selecting the optimal path. This improves the efficiency, reliability, and stability of data transmission and has significant technological innovation and broad application prospects.

[0061] In a network bandwidth environment of 10Mbps, by simulating a network scenario with 50 transmission nodes, we conducted a comparative experiment between our algorithm (an intelligent network optimal transmission path algorithm based on node status and link load) and the traditional shortest path-based scheduling algorithm. The comparison results are as follows:

[0062] Table 1 Comparison Results

[0063] Shortest path scheduling algorithm Intelligent scheduling algorithm Throughput 6Mbps 8Mbps Transmission delay 1.2s 0.8s Bandwidth utilization 60％ 80％

[0064] In summary, this patent, by comprehensively considering the current state of nodes and link load, can dynamically select the optimal transmission path, effectively avoiding congestion on high-load links and thus significantly improving the overall network throughput. In contrast, traditional shortest path algorithms select routes based solely on path length, without considering actual load conditions, leading to some links becoming bottlenecks during high-load periods and limiting further throughput improvements. Experimental results show that, under the same network conditions, the throughput of this algorithm is approximately 33% higher than that of traditional algorithms. Furthermore, because this patent can intelligently avoid congested links and select smoother paths for data transmission, it significantly reduces data transmission latency. Traditional shortest path algorithms, when facing network congestion, may choose seemingly shortest paths that actually suffer from severe delays due to excessive load, leading to increased transmission latency. Experimental data shows that this algorithm performs exceptionally well in reducing transmission latency, with an average latency reduction of approximately 25% compared to traditional algorithms. Finally, by intelligently allocating network resources, this patent ensures more balanced and efficient utilization of network resources. It dynamically adjusts transmission strategies based on the real-time status of nodes and links, avoiding excessive concentration and waste of resources. In contrast, traditional shortest path algorithms may lead to resource strain on some links while others remain relatively idle due to a lack of consideration for global resource allocation, resulting in uneven bandwidth utilization. Experimental results show that our proposed algorithm improves bandwidth utilization by approximately 20% compared to traditional algorithms, achieving a more optimal allocation of network resources.

[0065] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.

Claims

1. A cooperative scheduling method, comprising the following steps: 1) Deploy a central node controller in the target network, and deploy a cooperative scheduler on each node in the target network; 2) The central node controller periodically sends monitoring requests to each node in the target network; each time the cooperative scheduler receives the monitoring request, it reports the status information and link load of its node to the central node controller; wherein, the status information includes CPU utilization, memory usage, and port status; 3) The central node controller constructs a global resource view based on the status information and link load data of each node, and pushes it to each node; 4) The coordinating scheduler on the source node in the target network obtains the global routing link table and network topology of the target network from the global resource view; then, based on the destination IP address of the data flow and the global routing link table and network topology of the target network, it calculates multiple candidate paths and sends them to the central node controller; wherein, the method for calculating multiple candidate paths is as follows: first, multiple indicator factors are obtained from the global resource view and the importance value of each indicator factor is determined; multiple candidate paths from the source node to the destination node are generated based on the global routing link table and network topology of the target network; then, a weighted graph of candidate paths is generated by assigning weights to each candidate path based on the importance value of each indicator factor; then, an algorithm in graph theory is used to search and evaluate the merits of each candidate path in the weighted graph of candidate paths, and multiple candidate paths are generated based on the evaluation results; 5) The central node controller calculates the comprehensive score of each candidate path, selects the optimal path based on the comprehensive score of each candidate path, and sends it to the source node for data transmission.

2. The method according to claim 1, characterized in that, The central node controller calculates a comprehensive score for each candidate path based on latency, bandwidth capacity, reliability, security, and cost-effectiveness.

3. The method according to claim 2, characterized in that, The central node controller is based on an evaluation function. Calculate the i-th candidate path P i The overall score result is Score (P). i ); where D max D min These are the maximum and minimum delay times among the multiple candidate paths, respectively. Weights for delay time; B max B min These are the maximum and minimum bandwidth capacities among the multiple candidate paths, respectively. The weight of bandwidth capacity; R max ,R min These are the maximum and minimum reliability values ​​among the multiple candidate paths, respectively. Weights for reliability; S max It is the maximum security value among the multiple candidate paths. Weights for security; C max C min These are the maximum and minimum cost-effectiveness values ​​among the multiple candidate paths, respectively. Weighting based on cost-effectiveness.

4. The method according to claim 1, 2, or 3, characterized in that, The link load data of the node is obtained using the Simple Network Management Protocol (SNMP).

5. The method according to claim 1, 2, or 3, characterized in that, The link load data includes bandwidth utilization, latency, and packet loss rate.

6. The method according to claim 1, 2, or 3, characterized in that, The central node controller monitors the amount of data received and sent by each node, the frequency of transmission and reception, the data transmission delay time, the bandwidth rate, the response time, and the number of data transmission interruptions. Based on the monitoring data, it constructs and updates an anomaly curve in real time, and then identifies bottlenecks or potential problems in the target network based on the anomaly curve.

7. The method according to claim 1, characterized in that, The central node controller dynamically evaluates the effectiveness of the current optimal path based on the monitored changes in network status. If the current optimal path is no longer effective, it recalculates and selects a new optimal path.

8. The method according to claim 1, characterized in that, The metrics include link status, bandwidth usage, and historical transmission records; the algorithms in the graph theory are Dijkstra's algorithm, A* algorithm, Bellman-Ford algorithm, or their variants.

Images