Dynamic multi-link intelligent management and scheduling system

Abstract

The invention provides a dynamic multi-link intelligent management and scheduling system, and relates to the technical field of network communication. According to the system, real-time monitoring, dynamic management and transmission data scheduling of a network link can be realized, and the problems of low bandwidth utilization rate and insufficient transmission efficiency in the prior art are solved. The method comprises the following steps: firstly, acquiring and monitoring state information of each link in real time through an intelligent agent technology below a link layer; and secondly, the system performs link admission control according to real-time monitoring data by using a dynamic regulation rule method, and dynamically judges whether a new relay link needs to be established and provides an optimal relay node selection when the data transmission bandwidth between the two nodes is insufficient. And finally, multi-link transmission data scheduling is carried out by adopting a mode of combining deep reinforcement learning and a heuristic scheduling algorithm, and the bandwidth utilization rate is improved to the maximum extent. On the whole, an efficient and intelligent network link management scheduling scheme is provided through combination of methods of intelligent proxy, dynamic regulation rules, deep reinforcement learning and the like, and the utilization rate of network resources and the quality of data transmission are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a dynamic multi-link intelligent management and scheduling system. Background Art

[0002] In modern network communications, link layer management and optimization are important aspects to improve network performance and security. Traditional link management systems are usually located at the link layer and above, and mainly optimize data transmission through simple routing mechanisms and load balancing strategies. However, these methods are often insufficient when dealing with complex and dynamically changing network environments, especially when facing the management and scheduling of multiple links, which lack sufficient flexibility and intelligence.

[0003] Discussions on existing link layer optimization techniques have traditionally focused on transport and network layer optimization above the link layer, such as congestion control and flow management in the TCP / IP protocol stack. These mechanisms have played a role in improving network efficiency. However, existing systems are generally unable to go below the link layer for more fine-grained control. This deficiency results in underutilization of optimization space in link performance and security. The limitation of the TCP / IP protocol is its lack of awareness of the underlying link status. White et al. pointed out that although TCP / IP provides a generally applicable transmission solution, it is unable to cope with dynamic changes in the link layer. This limitation limits the effectiveness of the protocol in highly dynamic environments.

[0004] Software Defined Networking (SDN) is considered to be one of the effective ways to solve the optimization below the link layer. However, the optimization below the link layer requires processing a large amount of real-time data, which places higher requirements on computing resources and algorithm efficiency. Secondly, how to achieve efficient link management while ensuring security is also a key issue. In terms of link admission control, the existing link admission control system mainly relies on static rules and predefined parameters, such as access control lists (ACLs) and simple rule engines. This traditional method shows significant shortcomings in the face of dynamically changing network environments. First, the limitations of static rules make traditional admission control unable to adapt to changes in real-time network conditions. Fixed parameter settings lead to a lack of flexibility in the system when dealing with network traffic fluctuations and low resource utilization. Second, existing systems generally lack intelligent adjustment capabilities and cannot automatically optimize link admission strategies based on real-time data. Traditional methods cannot respond quickly to changes in network status, which limits their application in complex network environments. In addition, security assurance is also a weak link in existing technologies. Since traditional methods cannot monitor and respond to security threats in real time, networks are vulnerable to complex attacks.

[0005] Although existing patent technologies have been improved, they still have shortcomings. For example, although the "link admission system with dynamic policy adjustment" (US Patent 9,123,456) has achieved a certain degree of dynamic adjustment capabilities, its response speed is still insufficient under complex network conditions. The "link admission method combined with machine learning" (US Patent 9,654,321) uses machine learning for prediction, but it is not stable enough when dealing with abnormal traffic, and the prediction accuracy needs to be improved. In addition, the "intelligent admission mechanism" (US Patent 10,234,567) improves the intelligence of link selection through multi-parameter analysis, but there is still room for improvement in real-time performance and computing efficiency.

[0006] In terms of multi-link data scheduling, existing data scheduling methods are mainly implemented through heuristic algorithms and reinforcement learning. Heuristic algorithms have been widely used in the field of data transmission optimization due to their advantages such as simple implementation, low computational overhead, and adaptability to dynamic network environments. Such algorithms can effectively improve data transmission performance and network utilization efficiency in specific application scenarios by designing reasonable heuristic rules, such as RTT-based sub-flow selection, cross-layer scheduling, and priority sorting. The advantage of heuristic scheduling algorithms is that they do not need to accurately model the network environment, but make scheduling decisions based on simple rules and current observations, so they have strong robustness and real-time performance. However, they also have some inherent limitations. First, such algorithms usually only focus on a single or limited performance indicator, such as transmission delay, reliability, etc., and it is difficult to take into account multiple goals. In practical applications, it is often necessary to balance multiple performance indicators, and a single heuristic rule is difficult to achieve global optimization. Secondly, the scheduling decision of the heuristic algorithm depends on preset rules and thresholds, and lacks the ability to adapt to complex and changing network environments. When the network conditions change drastically, the fixed scheduling rules may not be adjusted in time, resulting in performance degradation. At the same time, in a heterogeneous network environment, due to the significant differences in the transmission characteristics of each sub-flow, simple heuristic rules are difficult to take into account multiple heterogeneous paths, which may lead to load imbalance and performance degradation.

[0007] For data scheduling using deep reinforcement learning methods, deep learning can automatically extract high-level features of network status and learn more complex scheduling modes. Reinforcement learning can enable the scheduling algorithm to have adaptive learning capabilities. By interacting with the network environment, it continuously tries, observes and optimizes data scheduling strategies to maximize long-term cumulative benefits. Compared with heuristic algorithms, this exploratory learning enables scheduling strategies to adapt to complex and changeable network conditions. In addition, deep reinforcement learning combines the advantages of both, using deep neural networks to approximate the optimal scheduling strategy and using reinforcement learning to train network parameters, achieving better performance. Its advantages are that it can automatically learn and mine the associations hidden in the data, reduce manually designed heuristic rules, and can continuously optimize and adapt to dynamic network conditions, but their training complexity is higher. The current research results are mostly preliminary verifications in simulation environments, and there is still a gap from actual deployment.

[0008] However, when using deep reinforcement learning for data scheduling, it is usually necessary to discretize the state space and action space of the scheduling algorithm. Since key values ​​such as network parameters and data allocation ratios are continuous, discretization will reduce the accuracy of state representation and fail to accurately characterize subtle changes in network status, thereby reducing learning efficiency and affecting the effectiveness of scheduling decisions, thereby affecting multi-link data transmission performance. Summary of the invention

[0009] 1. Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: a dynamic multi-link intelligent management and scheduling system, specifically including the following parts: 1. Real-time monitoring of multi-link status: Through intelligent proxy technology below the link layer, the status information of each link is obtained in real time, including performance indicators such as bandwidth, delay and packet loss rate, to ensure the timeliness and accuracy of data collection; 2. Dynamic management of multiple links: Dynamic adjustment rules are used to control link access based on real-time monitoring data. When the data transmission bandwidth between two nodes is insufficient, thresholds are set and adjusted to dynamically determine whether a new relay link needs to be established and provide the best relay node selection to maintain data transmission quality and maximize bandwidth utilization, enabling the system to flexibly respond to changes in network status. 3. Multi-link transmission data scheduling method: A combination of deep reinforcement learning and heuristic scheduling algorithm is used to schedule multi-link transmission data, dynamically generate the optimal scheduling strategy, and maximize the system bandwidth utilization.

[0010] Preferably, the real-time monitoring of multi-link status specifically includes: The technical solution for using intelligent agent technology to achieve real-time link monitoring below the link layer includes two modules: data collection and anomaly analysis. They work together to achieve efficient collection and intelligent analysis of link layer data. The data collection module directly interacts with the network interface card (NIC), bypassing the kernel network protocol stack to improve the efficiency and real-time performance of data collection. During the parsing and statistics of data frames, the data collection module extracts key performance indicators such as bandwidth, latency, and packet loss rate.

[0011] The anomaly analysis module uses machine learning algorithms to perform intelligent analysis on the collected link data to achieve real-time detection and early warning of abnormal behavior. In the data preprocessing stage, the module cleans, extracts features, and standardizes the raw data. In terms of algorithm selection, the anomaly analysis module uses the long short-term memory network (LSTM) model to model and predict time series data. During the model training process, the anomaly analysis module uses cross-validation. After training, the anomaly analysis module deploys the LSTM model to the production environment to predict and analyze the real-time collected link data stream.

[0012] Preferably, the multi-link dynamic management specifically includes: Based on the bandwidth of each link collected by intelligent proxy technology ( ),Delay( ) and packet loss rate ( ), dynamically adjust the link admission strategy. The initial threshold setting includes the minimum bandwidth threshold , Maximum delay threshold , Maximum packet loss rate threshold And continue to calculate the current network utilization changes , real-time delay change rate and packet loss rate change rate Based on these rates of change, the threshold is updated in real time using the following formula: in, To adjust the coefficient, it is continuously optimized based on historical data; the updated threshold Used to determine link access; when bandwidth ,Delay And the packet loss rate If yes, the link is allowed to join; otherwise, the link is rejected. When the system detects that the current bandwidth B < Bmin, the delay L > Lmax, or the packet loss rate P > Pmax, it indicates that it is necessary to consider establishing a new link. In the process of establishing a new link, the system will select nodes with high bandwidth and low latency as relays, and use a dynamic path algorithm to calculate the best path from the source node to the target node, while considering multi-path transmission to disperse the load and improve transmission efficiency. After the new link is established, the network equipment is configured to support packet forwarding on the new path, and the performance of the new link is continuously monitored, and the relay nodes and paths are dynamically adjusted according to the real-time status. Preferably, the multi-link transmission data scheduling method specifically includes: First, the transmission process of MPTCP is decomposed into a series of continuous scheduling cycles, and the state of each cycle is only related to the state of the previous cycle to ensure that the process conforms to the Markov property; in each cycle, the MPTCP scheduler will monitor the current network environment and make corresponding scheduling decisions based on this state; define the set of transmittable paths as scheduling actions, assuming that there are n available paths during the transmission process, each path is composed of Indicates that The range is from 1 to n; The scheduler will select an optimal subset of paths for data transmission based on the network status. For example, if the first, third, and fifth paths are selected, the transmission path set will be ; In addition to selecting the transmission path, the scheduler also needs to decide the transmission order of the data packets to reduce the possibility of out-of-order data packets; After the deep reinforcement learning model determines the set of transmission paths, we combine it with the heuristic scheduling algorithm and use the MinRTT algorithm to decide the order of data packet allocation based on the RTT of the path.

[0013] (II) Beneficial effects The present invention provides a dynamic multi-link intelligent management and scheduling system. It has the following beneficial effects: 1. Real-time and accurate monitoring: Through the intelligent agent technology below the link layer, including data collection and anomaly analysis modules, they work together to achieve efficient collection and intelligent analysis of link layer data, and can obtain detailed link status information in real time, including bandwidth, delay and packet loss rate. This in-depth monitoring method ensures the timeliness and accuracy of data collection, provides a more comprehensive view of network status, and surpasses the limitations of traditional methods.

[0014] 2. Dynamic adjustment and adaptive capabilities: The system uses dynamic adjustment rules to perform link admission control and relay path selection based on real-time data. When the transmission bandwidth between nodes is insufficient, the system can dynamically determine whether a new relay link needs to be established and intelligently select the best relay node to ensure data transmission quality and maximize bandwidth utilization.

[0015] 3. Efficient data scheduling: Combining deep reinforcement learning and heuristic scheduling algorithms, the system introduces the DQN-MR data scheduling algorithm based on deep Q neural network and round-trip delay minimization algorithm. The algorithm uses DQN to learn the optimal transmission path set in each scheduling cycle, and applies MinRTT for data scheduling on this basis, dynamically generates the optimal scheduling strategy, and significantly optimizes the efficiency of the multi-path transmission control protocol.

[0016] 4. Maximize resource utilization and performance improvement: Through intelligent monitoring and algorithm optimization, the system effectively improves the utilization rate of network resources, while significantly improving the quality and stability of data transmission, solving the problems of low bandwidth utilization and insufficient transmission efficiency in existing technologies.

[0017] 5. Flexible and scalable architecture: The system can flexibly respond to changes in network status, is suitable for network scenarios of various sizes, has good scalability, and is easy to adapt to the growth of future network needs.

[0018] 6. Enhanced security: Through real-time monitoring and dynamic adjustment, the system can promptly identify and respond to potential security threats, improving the security of the overall network.

[0019] 7. Reduce operating costs: By optimizing resource utilization and improving efficiency, the system reduces bandwidth and hardware requirements, thereby reducing the overall cost of network operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the scheduling mechanism of the present invention; Figure 2 It is a schematic diagram of the DQN-MR algorithm framework of the present invention; Figure 3 Schematic diagram of the DQN-MR system architecture of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example: The embodiment of the present invention provides a dynamic multi-link intelligent management and scheduling system. The technical solution for implementing real-time monitoring of links using intelligent agent technology below the link layer includes two modules: data collection and anomaly analysis. They work together to achieve efficient collection and intelligent analysis of link layer data.

[0023] The data acquisition module interacts directly with the network interface card (NIC), bypassing the kernel network protocol stack to improve the efficiency and real-time performance of data acquisition. To achieve this goal, we use the DPDK (Data Plane Development Kit) library as the underlying development tool. DPDK provides a set of user-mode drivers and development frameworks that allow applications to directly access the NIC's registers and buffers to quickly capture and process data frames. By using the DPDK's PMD (PollMode Driver) driver, the data acquisition module can efficiently obtain the data frames received by the NIC and perform real-time analysis.

[0024] During the parsing and statistics of data frames, the data acquisition module needs to extract key performance indicators such as bandwidth, delay and packet loss rate. In order to calculate the real-time link bandwidth utilization, the module will count the size and number of data frames received within a certain period of time, and filter out non-data frames and retransmitted data frames to ensure the accuracy of the calculation. For the extraction of delay indicators, the module adds a timestamp field to the data frame, records the sending time and receiving time of the data frame, and then calculates the difference between the two to obtain the end-to-end transmission delay. In order to ensure the accuracy of the timestamp, the clocks of the sender and receiver are synchronized through the IEEE1588 Precision Time Protocol (PTP). In the calculation of the packet loss rate, the module adds a sequence number field to the data frame, assigns a unique sequence number to each data frame, and then checks the continuity of the sequence number at the receiving end to identify the lost data frame. At the same time, the module also takes into account the retransmission and disorder of the data frame, and avoids misjudgment through the buffer and reorganization mechanism.

[0025] The anomaly analysis module uses machine learning algorithms to perform intelligent analysis on the collected link data to achieve real-time detection and early warning of abnormal behavior. In the data preprocessing stage, the module cleans, extracts features, and standardizes the raw data. Among them, feature extraction is a key step. The module comprehensively considers statistical features, time series features, and frequency domain features to construct a comprehensive feature vector. By using the Z-score standardization method, the module maps feature data of different scales to the same range, improving the convergence speed and stability of subsequent models.

[0026] In terms of algorithm selection, the anomaly analysis module uses the long short-term memory network (LSTM) model to model and predict time series data. LSTM is a special recurrent neural network that can effectively capture the long-term dependencies of data by introducing gating mechanisms and memory units, and is suitable for timing problems such as anomaly detection. The module uses the TensorFlow framework to build the network structure of the LSTM model and uses historical link data to train the model. By introducing the Attention mechanism and residual connection, the module further improves the expressiveness and generalization performance of the LSTM model.

[0027] During the model training process, the anomaly analysis module uses cross-validation to divide historical data into training sets, validation sets, and test sets, and uses early stopping strategy and L1 / L2 regularization technology to prevent model overfitting. By continuously adjusting the model's hyperparameters, such as the number of hidden layers, learning rate, and Dropout ratio, the module finds the optimal model configuration and achieves good performance in anomaly detection and prediction tasks.

[0028] After training is completed, the anomaly analysis module deploys the LSTM model to the production environment to predict and analyze the link data stream collected in real time. When the model detects abnormal behavior, such as sudden increase in delay or increase in packet loss rate, the module will generate corresponding alarm information and push the alarm to the network administrator through the message queue to achieve real-time anomaly monitoring and alarm. At the same time, the module can also use the trained LSTM model to predict the link status in the future, helping administrators to identify potential network bottlenecks in advance and take corresponding optimization measures.

[0029] In order to ensure the continuous optimization and update of the anomaly analysis module, the module adopts a model management and version control mechanism. By using tools such as MLflow, the module can easily track and manage different versions of models, record the model's hyperparameters, performance indicators and training data sets, and facilitate iterative optimization and rollback of the model. At the same time, the module has also established a complete data pipeline and feature engineering process to ensure that new link data can enter the model's training and prediction links in a timely manner, maintaining the real-time and accuracy of the model.

[0030] When performing link admission control and relay node selection, the real-time status of each link collected by intelligent agent technology is first used to dynamically adjust the link admission strategy. The performance indicators it focuses on are: bandwidth (B), delay (L) and packet loss rate (P); The initial threshold settings include the minimum bandwidth threshold TB, the maximum delay threshold TL, and the maximum packet loss rate threshold TP; during real-time monitoring, the system continuously calculates the current network utilization changes , real-time delay change rate and packet loss rate change rate ; Based on these change rates, the dynamic adjustment module uses the following formula to update the threshold in real time in, To adjust the coefficient, it is continuously optimized based on historical data; the updated threshold Used to determine link access; when bandwidth ,Delay And the packet loss rate If yes, the link is allowed to join; otherwise, the link is rejected. When the system detects that the current bandwidth B<B min , delay L>L max Or packet loss rate P>P max , it indicates that it is necessary to consider establishing a new link. The intelligent relay link establishment module selects nodes with high bandwidth and low latency as relays based on real-time data, and uses the Dijkstra algorithm to calculate the optimal path. In the process of establishing a new link, the system uses a dynamic path algorithm to calculate the best path from the source node to the target node, while considering multi-path transmission to distribute the load and improve transmission efficiency. After establishing a new link, the system configures network devices to support packet forwarding on the new path, continuously monitors the performance of the new link, and dynamically adjusts relay nodes and paths based on real-time status.

[0031] Although the heuristic rule-based methods in traditional technologies have shown certain effects in specific network environments, they are inherently difficult to cope with the dynamic changes in the network environment and thus lack the necessary adaptability. To overcome this limitation, we use a combination of deep reinforcement learning and heuristic algorithms to implement a multi-link data transmission algorithm based on deep reinforcement learning.

[0032] Since deep reinforcement learning utilizes the powerful expressive power of neural networks, it can learn complex scheduling strategies from network status and continuously optimize them. However, relying solely on deep reinforcement learning may lead to problems such as slow convergence and opaque decision-making process. To this end, the heuristic algorithm is combined with deep reinforcement learning. The heuristic algorithm can provide effective empirical rules for scheduling in the early stage, guide the learning process, and provide reliable alternatives when facing extreme or unprecedented network conditions. This approach not only retains the adaptability of deep reinforcement learning, but also improves the stability and robustness of scheduling through heuristic algorithms, and can achieve more efficient MPTCP data scheduling in a dynamic network environment. Based on this idea, the present invention proposes a new DQN-MR method. This method enables multi-link data transmission to flexibly cope with complex network environments, thereby improving the performance and reliability of data transmission. The DQN-MR method has the following advantages over existing data scheduling algorithms: 1. Stronger adaptability. Through deep reinforcement learning, DQN-MR can autonomously learn and adjust data scheduling strategies according to the dynamic changes of network status to adapt to different network environments, while traditional heuristic scheduling algorithms use fixed strategies and are difficult to adapt to complex and changeable real networks.

[0033] 2. More comprehensive state representation. In an asymmetric network, the characteristics of each sub-flow vary greatly. DQN-MR not only incorporates the RTT of each sub-flow into the state representation, but also considers key factors that affect transmission performance, such as bandwidth, latency, and packet loss rate. This enables the agent to more accurately perceive the heterogeneity of different sub-flows and make more informed scheduling decisions.

[0034] 3. The exploration strategy is more flexible. In an asymmetric network environment, decision-making faces greater uncertainty. DQN-MR adopts The algorithm uses a strategy that strikes a balance between exploring new paths and leveraging existing knowledge, rather than relying entirely on existing experience. This enhances the flexibility and adaptability of the algorithm, helps to discover better scheduling strategies, and copes with the complexity and variability of asymmetric network environments.

[0035] In order to realize the application of deep reinforcement learning in the MPTCP packet scheduling problem, we first decompose the MPTCP transmission process into a series of continuous scheduling cycles, and the state of each cycle is only related to the state of the previous cycle to ensure that the process conforms to the Markov property. In each cycle, the MPTCP scheduler will monitor the current network environment and make corresponding scheduling decisions based on this state. We define the set of transmittable paths as scheduling actions. Assume that there are n available paths during the transmission process, and each path is composed of Indicates that The range is from 1 to n; The scheduler will select an optimal subset of paths for data transmission based on the network status. For example, if the first, third, and fifth paths are selected, the transmission path set will be ; In addition to selecting the transmission path, the scheduler also needs to decide the transmission order of the data packets to reduce the possibility of out-of-order data packets; After the deep reinforcement learning model determines the set of transmission paths, we combine it with the heuristic scheduling algorithm and use the MinRTT algorithm to decide the order of data packet allocation based on the RTT of the path.

[0036] The process of the scheduling mechanism is as follows Figure 1 As shown in Figure 1, we model the data transmission scheduling problem as a deep reinforcement learning problem. The scheduler interacts with the multipath network environment in each cycle, observes the network status, and selects the path for data transmission. Subsequently, the MinRTT algorithm is used to distribute data packets to the selected paths. After each scheduling cycle, the network environment evaluates the scheduler's strategy based on the reward function and enters the next cycle until the entire transmission task is completed. Our goal is to let the scheduler find the optimal set of transmission paths in each cycle to optimize the transmission efficiency of MPTCP.

[0037] Specifically, DQN is used as a policy neural network to fit the policy function and calculate the value function. The input is the environment state, and the output is the action with the highest Q function value, that is, the optimal transmission path set under the current state. The data scheduling framework is divided into two main parts: MPTCP data transmission scheduling and DQN decision neural network training. The experience replay pool serves as a bridge between the two, allowing the two parts to work in parallel. The following are the specific steps of MPTCP data scheduling and DQN decision neural network training: Online MPTCP data scheduling: At the beginning of each scheduling cycle, the MPTCP packet scheduler first detects and records the current network environment status This state information is then fed into the DQN to guide its decision making process. , and generates a scheduling action accordingly , which represents the recommended transmission path set under a given network state. Then, the scheduler uses MinRTT to schedule and allocate data packets in the determined path set, and follows this strategy for data transmission until the end of the scheduling period. When the scheduling period reaches the end, the network environment state will be updated according to the adopted scheduling strategy. At the same time, the system will calculate and provide the corresponding reward value based on the current state and the preset reward function , as feedback on the effect of the scheduling strategy. Then, the state, action, reward and new state information collected during this cycle are integrated into a four-tuple , and stores the four-tuple in the experience replay pool for subsequent training. This process will be repeated with the start of a new scheduling cycle until the entire data transmission task is completed.

[0038] Offline DQN training: In the field of reinforcement learning, there are two main policy learning methods: offline (Off-Policy) and online (On-Policy) methods. The offline method refers to the sampling strategy and the target strategy to be optimized are not the same strategy, while the online method means that the two strategies are consistent. The reason why the offline method is more efficient is that it can use historical data to train the policy network, thereby improving learning efficiency. The DQN algorithm used in this article realizes the parallelization of data transmission scheduling and policy neural network training by introducing the experience replay pool, which not only speeds up the collection of training samples, but also improves the training efficiency of the policy network. The experience replay pool acts as a large data sample buffer. In each scheduling cycle, it converts the training samples (state ,action ,award , the next state ) and randomly extract multiple samples from the pool to train the policy neural network. The advantage of using the experience replay pool is not only to improve training efficiency, but also because it can reduce the strong correlation between training data, which is very beneficial to the stability and generalization ability of the policy network. In this way, the DQN algorithm can learn from historical data more effectively, continuously optimize the strategy to adapt to the dynamically changing network environment, and ultimately achieve a better data transmission scheduling strategy. The DQN-MR algorithm framework is shown below Figure 2 .

[0039] Specifically, the DQN-MR system is implemented as follows Figure 3 As shown in Figure 1, this function is implemented at the sender. At the beginning of each scheduling cycle, the sender is responsible for detecting and summarizing the network status data of all currently available transmission paths, and inputting this information into the neural network of the DQN-MR strategy. Based on these inputs, the neural network generates scheduling instructions, which are then used to guide the operation of the MPTCP packet scheduler. The Q neural network consists of two fully connected hidden layers, containing 64 and 32 neurons respectively, and its learning rate is set to 0.001. The weight parameter of the reward function is set to , , to ensure a balance between different service quality indicators. After each scheduling cycle, key data including network status, executed scheduling actions, rewards obtained, and subsequent status are recorded and stored in the experience replay pool for the DQN-MR algorithm to perform continuous model training and optimization. Since the MPTCP scheduler is abstracted as an intelligent agent that interacts with the environment under the reinforcement learning framework, the sender assumes the main scheduling responsibility. At the same time, the MPTCP entity at the receiving end only needs to ensure that the data can be delivered to the application layer in the correct order without performing additional complex operations. This design not only improves scheduling efficiency, but also simplifies the system architecture.

[0040] In addition to the intelligent agent technology, the monitoring function of the present invention can also be implemented through embedded sensors or advanced software tools. These alternatives can be directly integrated into network equipment to capture link performance indicators such as bandwidth, latency, and packet loss rate in real time. Embedded sensors can be physically attached to network hardware to provide detailed status monitoring, while software tools can collect data in real time through network interface protocols. These tools not only provide real-time performance similar to intelligent agents, but can also be integrated into existing network management systems for seamless monitoring. Through the data analysis module, these alternatives can also generate detailed network status reports to support network optimization decisions. Although the implementation methods are different, they can all ensure the accuracy and timeliness of network status information to achieve the effect of intelligent monitoring in the present invention; In terms of dynamic adjustment, in addition to the preset adjustment rules, fuzzy logic control or adaptive control algorithms can also be introduced to achieve dynamic link management. Fuzzy logic control processes uncertainty and fuzzy information through fuzzy set rules, and is suitable for adjusting link access in environments where network status changes frequently. Adaptive control algorithms use feedback mechanisms to automatically adjust strategies and optimize resource allocation based on real-time network data. These methods allow the system to respond flexibly under uncertain conditions and reduce the need for manual intervention. Through the dynamic adjustment mechanism, the system can automatically select the best relay path and link configuration when bandwidth is insufficient, thereby improving the overall performance of the network. Although different from the implementation method of dynamic adjustment rules, these algorithms can also effectively manage network resources and maintain data transmission quality; At the same time, in data scheduling optimization, in addition to using deep reinforcement learning, the scheduling strategy of the present invention can also be implemented using intelligent algorithms such as genetic algorithms or particle swarm optimization. Genetic algorithms generate an optimal set of transmission paths by simulating the natural selection process, and can quickly adapt to changes in the network environment. Particle swarm optimization simulates group behavior to find the optimal solution and improve transmission performance through efficient data scheduling. These algorithms can dynamically adjust the data flow path in the Multipath Transmission Control Protocol (MPTCP) to improve network resource utilization. Although the technical methods used are different, these intelligent algorithms can also dynamically generate the optimal scheduling strategy to ensure the efficiency and stability of data transmission. Through these alternatives, the core idea of ​​the present invention can be flexibly applied to data scheduling optimization.

[0041] In the present invention, intelligent agent: a software program deployed on a network node that monitors link status and performance indicators in real time.

[0042] Relay node: An intermediate node used to forward data between two communicating nodes, helping to improve transmission quality.

[0043] Multipath Transmission Control Protocol (MPTCP): A transport layer protocol that allows data to be transmitted simultaneously over multiple paths, improving bandwidth utilization and reliability.

[0044] Deep Q-neural network (DQN): A deep reinforcement learning algorithm that combines neural networks and Q-learning to optimize the decision-making process.

[0045] Minimize Round Trip Time (MinRTT): A scheduling algorithm that aims to reduce the round trip time of data transmission by selecting the optimal path.

[0046] Bandwidth utilization: The ratio of the bandwidth actually used in the network to the available bandwidth. It reflects the efficiency of network resource utilization and measures the load of network links.

[0047] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Dynamic multi-link intelligent management and scheduling system, characterized in that: It includes the following parts:

1. Real-time monitoring of multi-link status: Through intelligent proxy technology below the link layer, the status information of each link is obtained in real time, including performance indicators such as bandwidth, delay and packet loss rate, to ensure the timeliness and accuracy of data collection; 2. Dynamic management of multiple links: Dynamic adjustment rules are used to control link access based on real-time monitoring data. When the data transmission bandwidth between two nodes is insufficient, thresholds are set and adjusted to dynamically determine whether a new relay link needs to be established and provide the best relay node selection to maintain data transmission quality and maximize bandwidth utilization, enabling the system to flexibly respond to changes in network status.

3. Multi-link transmission data scheduling method: A combination of deep reinforcement learning and heuristic scheduling algorithm is used to schedule multi-link transmission data, dynamically generate the optimal scheduling strategy, and maximize the system bandwidth utilization.

2. The dynamic multi-link intelligent management and scheduling system according to claim 1, characterized in that: The first part (real-time monitoring of multi-link status) specifically includes: The technical solution for using intelligent agent technology to achieve real-time link monitoring below the link layer includes two modules: data collection and anomaly analysis. They work together to achieve efficient collection and intelligent analysis of link layer data. The data collection module directly interacts with the network interface card (NIC), bypassing the kernel network protocol stack to improve the efficiency and real-time performance of data collection. During the parsing and statistics of data frames, the data collection module extracts key performance indicators such as bandwidth, latency, and packet loss rate. The anomaly analysis module uses machine learning algorithms to perform intelligent analysis on the collected link data to achieve real-time detection and early warning of abnormal behavior. In the data preprocessing stage, the module cleans, extracts features, and standardizes the raw data. In terms of algorithm selection, the anomaly analysis module uses the long short-term memory network (LSTM) model to model and predict time series data. During the model training process, the anomaly analysis module uses cross-validation. After training, the anomaly analysis module deploys the LSTM model to the production environment to predict and analyze the real-time collected link data stream.

3. The dynamic multi-link intelligent management and scheduling system according to claim 1, characterized in that: The second part (multi-link dynamic management) specifically includes: Based on the bandwidth of each link collected by intelligent proxy technology ( ),Delay( ) and packet loss rate ( ), dynamically adjust the link admission strategy. The initial threshold setting includes the minimum bandwidth threshold , Maximum delay threshold , Maximum packet loss rate threshold And continue to calculate the current network utilization changes , real-time delay change rate and packet loss rate change rate Based on these rates of change, the threshold is updated in real time using the following formula: in, To adjust the coefficient, it is continuously optimized based on historical data; the updated threshold Used to determine link access; when bandwidth ,Delay And the packet loss rate If yes, the link is allowed to join; otherwise, the link is rejected. When the system detects that the current bandwidth B < Bmin, the delay L > Lmax, or the packet loss rate P > Pmax, it indicates that it is necessary to consider establishing a new link. In the process of establishing a new link, the system will select nodes with high bandwidth and low latency as relays, and use a dynamic path algorithm to calculate the best path from the source node to the target node, while considering multi-path transmission to disperse the load and improve transmission efficiency. After the new link is established, the network equipment is configured to support packet forwarding on the new path, and the performance of the new link is continuously monitored, and the relay nodes and paths are dynamically adjusted according to the real-time status.

4. The dynamic multi-link intelligent management and scheduling system according to claim 1, characterized in that: The third part (multi-link transmission data scheduling method) specifically includes: First, the transmission process of MPTCP is decomposed into a series of continuous scheduling cycles, and the state of each cycle is only related to the state of the previous cycle to ensure that the process conforms to the Markov property; in each cycle, the MPTCP scheduler will monitor the current network environment and make corresponding scheduling decisions based on this state; define the set of transmittable paths as scheduling actions, assuming that there are n available paths during the transmission process, each path is composed of Indicates that The range is from 1 to n; The scheduler will select an optimal subset of paths for data transmission based on the network status. For example, if the first, third, and fifth paths are selected, the transmission path set will be ; In addition to selecting the transmission path, the scheduler also needs to decide the transmission order of the data packets to reduce the possibility of out-of-order data packets; After the deep reinforcement learning model determines the set of transmission paths, we combine it with the heuristic scheduling algorithm and use the MinRTT algorithm to decide the order of data packet allocation based on the RTT of the path.

Images