Cooperative congestion control method based on attention-driven multi-path tcp subflow dependency modeling

Abstract

The application provides a kind of collaborative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling, MPTCP connection is established by sending end and receiving end;Subflow running data is read from MPTCP protocol stack by state monitoring module;Each subflow feature of the normalized state matrix input is weighted and fused by using self-attention layer;Action vector A is generated by Actor network t , and the effective congestion window of each subflow is used as the available sending window constraint when sending data;Action value Q is generated by Critic network;Reward R is calculated by reward calculation module t ;Form collaborative congestion control model;Realize online adaptive control;This method can realize unified decision and collaborative control of multiple subflows, can realize collaborative consistency of congestion control effect and scheduling result, and can improve the stability and comprehensive performance of multi-path transmission.

Description

Technical Field

[0001] This invention relates to a collaborative congestion control method based on attention-driven multi-path TCP sub-stream dependency modeling, belonging to the field of communication transmission control technology. Background Technology

[0002] With the widespread adoption of cloud computing, mobile internet, and multi-access terminals, it has become commonplace for terminals to simultaneously support multiple access methods such as cellular networks and wireless LANs. In real-world network environments, different links exhibit significant differences in bandwidth, round-trip time, jitter, and packet loss, and link states change rapidly over time. Multipath transmission control protocols (MTCs) can improve throughput, enhance robustness, and improve link switching continuity to some extent by establishing multiple sub-streams for parallel data transmission over the same connection. Therefore, they have received widespread attention and are widely used in heterogeneous multi-link aggregation transmission scenarios.

[0003] Existing MPTCP schemes typically rely on a congestion control module and a flow scheduling module to jointly handle transmission: the congestion control module adjusts the transmission window or rate of each sub-stream, while the flow scheduling module determines which sub-stream to send the data segment to. Although existing technologies can achieve bandwidth aggregation or stable transmission under certain conditions, the following problems still exist under heterogeneous link and time-varying network conditions: (1) The separation of control and scheduling leads to insufficient coordination. Currently, the divide-and-conquer mechanism is mostly used, in which congestion control determines how much to send and the scheduler determines which path to send to. There is a lack of unified goal constraints and consistent feedback loop between the two. When the link state changes abruptly or queue backlog occurs, the congestion control and scheduling decisions may respond asynchronously, resulting in a mismatch between resource allocation and path selection among sub-flows, thereby causing overall performance degradation.

[0004] (2) Lack of effective adaptation to link time-varying and jitter. The RTT and jitter of heterogeneous links are random and bursty in the wireless environment. Strategies based on fixed rules or single indicators (such as minimum RTT, historical throughput, etc.) are prone to path bias or delayed switching, which leads to increased queue backlog, amplified latency and jitter, and makes it difficult to maintain a stable transmission experience.

[0005] (3) Out-of-order and retransmission overhead amplifies, reducing effective throughput. Multi-path parallel transmission naturally has differences in arrival delay. If window control and scheduling strategies fail to match the differences in links, the degree of out-of-order at the receiving end will increase, triggering waiting for reordering and unnecessary retransmission, resulting in additional overhead and reduced effective throughput, while further worsening end-to-end delay fluctuations.

[0006] (4) Insufficient ability to balance multiple objectives, making it difficult to simultaneously consider throughput and latency stability. Practical applications often require simultaneous optimization of metrics such as throughput, latency, jitter, packet loss, and fairness. Existing solutions mostly rely on empirical rules or static parameter configurations, lacking the ability to adaptively balance multiple objectives in dynamic environments. This can easily lead to problems such as increased throughput but worsened latency / jitter, or decreased latency but insufficient bandwidth utilization, affecting the overall system performance and service experience.

[0007] Therefore, there is an urgent need for an MPTCP sub-stream transmission control method that can achieve coordinated sub-stream control and scheduling and adaptively balance multi-objective performance under time-varying structural network conditions, so as to improve the stability and overall performance of multi-path transmission. Summary of the Invention

[0008] The purpose of this invention is to provide a collaborative congestion control method based on attention-driven multi-path TCP sub-stream dependency modeling to solve the problems in the prior art where the separation of control and scheduling leads to insufficient collaboration and the need to improve the stability and overall performance of multi-path transmission.

[0009] The technical solution of this invention is: A collaborative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling includes the following steps: S1. The sending end and the receiving end establish an MPTCP connection and generate N sub-streams for data transmission; S2. The state monitoring module reads the sub-stream execution data from the MPTCP protocol stack and constructs the state matrix S. t After normalization, the normalized state matrix is ​​obtained. ; S3. Use a self-attention layer to normalize the input state matrix. The features of each sub-flow are weighted and fused to obtain the representation Z of the dependencies between the fused sub-flows. t The Actor network and Critic network are output respectively. S4. Representation of the inter-stream dependencies of the input fused substreams by the Actor network Z t Generate action vector A t That is, the cwnd quota action, which will move the action vector A. t The action interface is used to write data to the MPTCP protocol stack, along with the current congestion window value in the running data of each substream. Jointly determine the effective congestion window for each sub-stream When sending data, the effective congestion window of each substream is used. As a constraint on the available sending window; S5. The Critic network represents the dependencies between the input fused substreams, Z. tWith action vector A t Generate action value Q; S6, Execute action vector A t Then obtain the new state The reward R is calculated by the reward calculation module based on the substream execution data. t , to use empirical data Store in the experience replay pool; S7. The self-attention deep deterministic policy gradient agent, namely SA-DDPG agent, is composed of self-attention layer, Actor network, Critic network, reward calculation module and experience replay pool. The state monitoring module and SA-DDPG agent together constitute a cooperative congestion control model. After training, the trained cooperative congestion control model is obtained. S8. The sending end deploys the trained collaborative congestion control model in user space to achieve online adaptive control.

[0010] Furthermore, in step S2, the sub-stream operation data includes the current congestion window value, unacknowledged data volume, smooth round-trip delay, jitter index, acknowledged data volume in the current statistical period, and transmission anomaly index for each sub-stream. The transmission anomaly index is determined by at least one of the following: number of packet losses, number of retransmissions, or number of timeouts.

[0011] Furthermore, in step S2, the state matrix S is constructed. t After normalization, the normalized state matrix is ​​obtained. Specifically, S21. At each decision time t, the control module constructs a state matrix S according to a pre-defined definition. t : , Where N is the number of sub-streams of the current MPTCP connection. Let be the fixed six-dimensional feature vector of the i-th sub-stream at time t. The fixed six-dimensional feature vector is composed of the sub-stream running data. S22, Regarding the state matrix S t A normalized state matrix is ​​obtained by performing a fixed-method normalization process. Normalized state matrix The normalization result of the k-th feature of the i-th subflow at time t for: , in, Let k represent the k-th feature of the i-th sub-stream at time t, where i = 1, 2, ..., N, and N is the number of sub-streams in the current MPTCP connection; k is the feature dimension index, k = 1, 2, ..., N. This represents the preset mean value corresponding to the k-th feature. ε represents the preset standard deviation corresponding to the k-th feature; ε represents a preset positive number used to avoid the denominator being zero.

[0012] Furthermore, in step S3, a self-attention layer is used to normalize the input state matrix. The features of each sub-flow are weighted and fused to obtain the representation Z of the dependencies between the fused sub-flows. t Specifically, S31. Normalize the state matrix Each feature vector of each substream is mapped to a query vector Q, a key vector K, and a value vector V: , in, These are the query mapping parameter matrix, the key mapping parameter matrix, and the value mapping parameter matrix, all of which are trainable parameter matrices. S32. Based on the query vector Q and key vector K, calculate the correlation between substreams, and obtain the attention weight matrix A by applying the normalized exponential function softmax: , Where d is the attention dimension, used for scale normalization, and T represents transpose; S33. Apply the attention weight matrix A to the value vector V to obtain the representation Z of the inter-stream dependencies of the fused substreams. t : .

[0013] Furthermore, in step S4, the Actor network represents the dependencies between the input fused substreams, Z. t Generate action vector A t That is, the cwnd quota action, which will move the action vector A. t The action interface is used to write data to the MPTCP protocol stack, along with the current congestion window value in the running data of each substream. Jointly determine the effective congestion window for each sub-stream Specifically, S41. Represent Z of the inter-stream dependencies. t The Actor network outputs action vector A after inputting it into the network. t : , in, Let represent the congestion window quota for sub-flow i, which is a directly executable cwnd constraint and is not normalized. Without any fixed constraints, R + Represents positive real numbers; S42, Transfer motion vector A tThe action interface is written to the kernel protocol stack, so that the congestion window of the i-th subflow is subject to the corresponding congestion window quota in the current control cycle. Constraints; S43, Action Vector A t After being written to the MPTCP protocol stack via the action interface, it is compared with the current congestion window value in the running data of each sub-stream. Jointly determine the effective congestion window for this sub-stream : , Here, clip() is the clipping function. The upper and lower bounds of the preset quota are defined, and min is the minimum value function.

[0014] Furthermore, in step S6, the reward calculation module calculates the reward R based on the sub-stream running data. t Specifically, S61. At the end of each control cycle, calculate the effective throughput TH based on the byte count of the received end. t : , Among them, TH t This represents the effective throughput within the control period corresponding to time t, where T is the length of the control period. This indicates that only confirmation events occurring within the interval (t−T,t] are counted; This represents the number of data bytes acknowledged within the interval (t−T,t]. S62. The time delay index D is characterized by the average value of the time delay estimate after each sub-slip processing, i.e., the smoothed RTT. t : , in, This is the smoothed RTT estimate of the i-th subflow at time t; S63. Jock index is measured by the change in smoothed RTT between adjacent time intervals. t : , Among them, J t This represents the jitter index within the control period corresponding to time t; N is the number of sub-streams of the current MPTCP connection. are the smoothed RTT estimates of the i-th subflow at times t and t-1, respectively; This represents the absolute value of the change in the smoothed RTT estimate between two adjacent times of the i-th substream; S64. Calculate the reward at time t based on the reward function weighted by throughput and stability indices. : , in, This is a packet loss cost metric used to characterize the packet loss, retransmission, or timeout situation of each sub-stream within the current control cycle; To avoid the undefined positive number for the logarithmic function when the effective throughput is zero, and These are preset weighting coefficients.

[0015] Furthermore, in step S7, the trained collaborative congestion control model is obtained by training using the deep deterministic policy gradient algorithm.

[0016] The beneficial effects of this invention are: I. This collaborative congestion control method based on attention-driven multi-path TCP sub-stream dependency modeling can achieve unified decision-making and collaborative control of multiple sub-streams, achieve consistency between congestion control effect and scheduling result, and improve the stability and overall performance of multi-path transmission.

[0017] II. This invention employs an SA-DDPG intelligent agent, which uses a normalized state matrix in each control cycle. As input, the coupling relationship between sub-streams is modeled through a self-attention layer, and then the Actor network outputs the action vector A. t This invention enables unified decision-making and collaborative control of multiple sub-streams. It introduces a self-attention layer before the Actor network and models sub-stream relationships through Q / K / V weighted fusion. This allows the policy to simultaneously consider the associated states of multiple sub-streams during decision-making, thereby improving perception and adaptation capabilities in heterogeneous and time-varying scenarios. It can explicitly characterize the impact of multi-sub-stream coupling, providing a more accurate global information representation for subsequent action decisions.

[0018] III. This collaborative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling, action vector A t Each sub-stream is directly assigned a congestion limit (cwnd) quota, and its components are not required to be normalized. The quota is written into the protocol stack through the action interface to form a window quota constraint. The available window space creates back pressure on the transmission, thereby indirectly guiding the transmission distribution without replacing the kernel scheduler, achieving synergistic consistency between congestion control effect and scheduling result.

[0019] Fourth, when calculating rewards during training, this invention incorporates throughput gains and latency / jitter costs into the optimization process. This allows the strategy to adaptively balance throughput and stability in dynamic environments, thereby reducing the estimated latency time (RTT) and jitter amplification, and improving overall transmission performance. This method can be applied to terminals or servers to achieve adaptive adjustment and performance optimization of substream transmission behavior under conditions of dynamic changes in heterogeneous networks or link states. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the collaborative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling according to an embodiment of the present invention. Figure 2 This is an explanatory block diagram of the cooperative congestion control method based on attention-driven multi-path TCP sub-stream dependency modeling in the embodiment. Detailed Implementation

[0021] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] The embodiment provides a cooperative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling, such as... Figure 1 This includes the following steps: S1. The sending end and the receiving end establish an MPTCP connection and generate N sub-streams for data transmission.

[0023] S2. The state monitoring module reads the sub-stream execution data from the MPTCP protocol stack and constructs the state matrix S. t After normalization, the normalized state matrix is ​​obtained. .

[0024] In step S2, the sub-stream operation data includes the current congestion window value, unacknowledged data volume, smooth round-trip delay, jitter index, acknowledged data volume in the current statistical period, and transmission anomaly index for each sub-stream. The transmission anomaly index is determined by at least one of the following: number of packet losses, number of retransmissions, or number of timeouts.

[0025] In step S2, the state matrix S is constructed. t After normalization, the normalized state matrix is ​​obtained. Specifically, S21. At each decision time t, the control module constructs a state matrix S according to a pre-defined definition. t : , Where N is the number of sub-streams of the current MPTCP connection. Let be the fixed six-dimensional feature vector of the i-th sub-stream at time t. The fixed six-dimensional feature vector is composed of the sub-stream running data. S22, Regarding the state matrix S t A normalized state matrix is ​​obtained by performing a fixed-method normalization process. Normalized state matrix The normalization result of the k-th feature of the i-th subflow at time t for: , in, Let k represent the k-th feature of the i-th sub-stream at time t, where i = 1, 2, ..., N, and N is the number of sub-streams in the current MPTCP connection; k is the feature dimension index, k = 1, 2, ..., N. This represents the preset mean value corresponding to the k-th feature. ε represents the preset standard deviation corresponding to the k-th feature; ε represents a preset positive number used to avoid the denominator being zero.

[0026] In step S2, the state matrix S t This is used to uniformly characterize the current multi-substream link quality and congestion evolution state. To eliminate the influence of differences in feature dimensions on the learning process, the state matrix is ​​normalized in a fixed manner before being input into the network.

[0027] S3. Use a self-attention layer to normalize the input state matrix. The features of each sub-flow are weighted and fused to obtain the representation Z of the dependencies between the fused sub-flows. t The Actor network and Critic network are output respectively.

[0028] In step S3, a self-attention layer is used to normalize the input state matrix. The features of each sub-flow are weighted and fused to obtain the representation Z of the dependencies between the fused sub-flows. t Specifically, S31. Normalize the state matrix Each feature vector of each substream is mapped to a query vector Q, a key vector K, and a value vector V: , in, These are the query mapping parameter matrix, the key mapping parameter matrix, and the value mapping parameter matrix, all of which are trainable parameter matrices. S32. Based on the query vector Q and key vector K, calculate the correlation between substreams, and obtain the attention weight matrix A by applying the normalized exponential function softmax: , Where d is the attention dimension, used for scale normalization, and T represents transpose; S33. Apply the attention weight matrix A to the value vector V to obtain the representation Z of the inter-stream dependencies of the fused substreams. t : .

[0029] In step S3, sub-flow relationship modeling is performed through a self-attention layer. This process enables the agent to explicitly characterize the impact of multi-sub-flow coupling, such as latency differences caused by link heterogeneity, transmission instability caused by congestion backlog, and the resulting out-of-order and jitter risks, thereby providing a more accurate global information representation for subsequent action decisions.

[0030] S4. Representation of the inter-stream dependencies of the input fused substreams by the Actor network Z t Generate action vector A t That is, the cwnd quota action, which will move the action vector A. t The action interface is used to write data to the MPTCP protocol stack, along with the current congestion window value in the running data of each substream. Jointly determine the effective congestion window for each sub-stream When sending data, the effective congestion window of each substream is used. As a constraint on the available sending window.

[0031] In step S4, the Actor network represents the dependencies between the input fused substreams, Z. t Generate action vector A t That is, the cwnd quota action, which will move the action vector A. t The action interface is used to write data to the MPTCP protocol stack, along with the current congestion window value in the running data of each substream. Jointly determine the effective congestion window for each sub-stream Specifically, S41. Represent Z of the inter-stream dependencies. t The Actor network outputs action vector A after inputting it into the network. t : , in, Let represent the congestion window quota for sub-flow i, which is a directly executable cwnd constraint and is not normalized. Without any fixed constraints, R + Represents positive real numbers; S42, Transfer motion vector A t The action interface is written to the kernel protocol stack, so that the congestion window of the i-th subflow is subject to the corresponding congestion window quota in the current control cycle. Constraints; S43, Action Vector A t After being written to the MPTCP protocol stack via the action interface, it is compared with the current congestion window value in the running data of each sub-stream. Jointly determine the effective congestion window for this sub-stream : , Here, clip() is the clipping function. The upper and lower bounds of the preset quota are defined, and min is the minimum value function.

[0032] In step S4, the action vector A t Each component in the diagram represents the congestion window quota for the corresponding sub-stream within the current control cycle. This invention indirectly controls the data transmission tendency among sub-streams by constraining the sub-stream's congestion window (cwnd) space, thereby achieving joint optimization without replacing the kernel scheduler rules. By creating backpressure through window quotas, it indirectly guides data packets to form a more reasonable transmission distribution among multiple sub-streams, achieving a synergistic effect between how much data is sent and where it is sent, without altering the basic rules of the kernel scheduler.

[0033] S5. The Critic network represents the dependencies between the input fused substreams, Z. t With action vector A t Generate action value Q.

[0034] In step S5, the generated action value Q is used for parameter optimization of the Actor network and Critic network during training.

[0035] S6, Execute action vector A t Then obtain the new state The reward R is calculated by the reward calculation module based on the substream execution data. t , to use empirical data Store in the experience replay pool.

[0036] In step S6, the reward calculation module calculates the reward R based on the sub-stream running data. t Specifically, S61. At the end of each control cycle, calculate the effective throughput TH based on the byte count of the received end. t : , Among them, TH t This represents the effective throughput within the control period corresponding to time t, where T is the length of the control period. This indicates that only confirmation events occurring within the interval (t−T,t] are counted; This represents the number of data bytes acknowledged within the interval (t−T,t]. S62. The time delay index D is characterized by the average value of the time delay estimate after each sub-slip processing, i.e., the smoothed RTT. t : , in, This is the smoothed RTT estimate of the i-th subflow at time t; S63. Jock index is measured by the change in smoothed RTT between adjacent time intervals. t : , Among them, J t This represents the jitter index within the control period corresponding to time t; N is the number of sub-streams of the current MPTCP connection. are the smoothed RTT estimates of the i-th subflow at times t and t-1, respectively; This represents the absolute value of the change in the smoothed RTT estimate between two adjacent times of the i-th substream; S64. Calculate the reward at time t based on the reward function weighted by throughput and stability indices. : , in, This is a packet loss cost metric used to characterize the packet loss, retransmission, or timeout situation of each sub-stream within the current control cycle; To avoid the undefined positive number for the logarithmic function when the effective throughput is zero, and These are preset weighting coefficients.

[0037] S7. A self-attention deep deterministic policy gradient agent, or SA-DDPG agent, is composed of a self-attention layer, an Actor network, a Critic network, a reward calculation module, and an experience replay pool. The state monitoring module and the SA-DDPG agent together constitute a cooperative congestion control model. Figure 2 After training, a trained collaborative congestion control model is obtained.

[0038] In step S7, the cooperative congestion control model is trained using the deep deterministic policy gradient algorithm.

[0039] S8. The sending end deploys the trained collaborative congestion control model in user space to achieve online adaptive control.

[0040] In step S8, the trained collaborative congestion control model generates action vector A in the Actor network. t After the action interface is written to the MPTCP protocol stack, the action vector A is... t The cwnd quota action adopts preset upper and lower bound constraints and change rate limits to make the cwnd quota change smoothly within a reasonable range; and when a sub-stream failure, continuous timeout or abnormal packet loss is detected based on the sub-stream operation data, a preset backoff mechanism is triggered to maintain the reliability of the connection and the continuity of transmission.

[0041] This collaborative congestion control method, based on attention-driven multi-path TCP sub-stream dependency modeling, can achieve unified decision-making and collaborative control of multiple sub-streams, ensure consistency between congestion control effects and scheduling results, and improve the stability and overall performance of multi-path transmission.

[0042] This collaborative congestion control method, based on attention-driven multi-path TCP sub-stream dependency modeling, runs on a sender supporting MPTCP. It establishes a closed-loop control process of observation-decision-execution-feedback for multiple sub-streams under the same MPTCP connection. The basic idea is as follows: within each decision cycle, multi-dimensional state variables reflecting the link quality, congestion level, and transmission status of each sub-stream are collected and input into a deep reinforcement learning decision model. The model outputs control actions for each sub-stream and maps these actions to constraints and proportional allocations of the congestion window (cwnd) or transmission quota for each sub-stream. This indirectly affects the data transmission distribution and congestion evolution among sub-streams, achieving collaborative optimization of congestion control and scheduling effects.

[0043] This invention employs an SA-DDPG intelligent agent, which uses a normalized state matrix in each control cycle. As input, the coupling relationship between sub-streams is modeled through a self-attention layer, and then the Actor network outputs the action vector A. t This invention enables unified decision-making and collaborative control of multiple sub-streams. It introduces a self-attention layer before the Actor network and models sub-stream relationships through Q / K / V weighted fusion. This allows the policy to simultaneously consider the associated states of multiple sub-streams during decision-making, thereby improving perception and adaptation capabilities in heterogeneous and time-varying scenarios. It can explicitly characterize the impact of multi-sub-stream coupling, providing a more accurate global information representation for subsequent action decisions.

[0044] This invention constructs a state matrix at each decision moment, including the current congestion window value, unacknowledged data volume, smoothed round-trip delay, jitter index, acknowledged data volume within the current statistical period, and transmission anomaly index, enabling real-time perception of link time-varying and jitter. By using a self-attention layer to weighted model the dependencies between sub-streams, it comprehensively considers the relative state changes between multiple sub-streams. The Actor network outputs the congestion limit (cwnd) of each sub-stream, and together with the congestion window calculated by the kernel congestion control algorithm, determines the effective congestion window, thereby constraining the transmission window for sub-streams with increased latency or exacerbated jitter. During the training phase, latency and jitter indices are incorporated into the reward function, enabling the collaborative congestion control model to balance throughput and stability under dynamic link conditions. The trained collaborative congestion control model is deployed to achieve adaptive adaptation to link time-varying and jitter.

[0045] This collaborative congestion control method, based on attention-driven multi-path TCP sub-stream dependency modeling, constructs a state matrix using the current congestion window value, unacknowledged data volume, smooth round-trip delay, jitter index, acknowledged data volume within the current statistical period, and transmission anomaly index of each sub-stream. This allows for a unified representation of latency differences, congestion backlog, and transmission anomalies among sub-streams. By modeling the dependencies between sub-streams through a self-attention layer, the control module can perceive the relative state differences between multiple sub-streams. The Actor network outputs the congestion window limit (cwnd) for each sub-stream, and together with the congestion window calculated by the kernel congestion control algorithm, determines the effective congestion window. This imposes sending window constraints on sub-streams with high latency, high jitter, or high retransmission risk. The window quota creates backpressure, indirectly guiding data distribution towards sub-streams with better states, reducing out-of-order accumulation, waiting for reordering, and unnecessary retransmissions caused by mismatches between window control and scheduling results. By incorporating effective throughput, latency, jitter, and packet loss costs into the reward function, the network parameter update process can learn control strategies to improve effective throughput and reduce retransmission risk, thereby suppressing out-of-order and retransmission overhead.

[0046] This collaborative congestion control method, based on attention-driven multi-path TCP sub-flow dependency modeling, has action vector A. t Each sub-stream is directly assigned a congestion limit (cwnd) quota, and its components are not required to be normalized. The quota is written into the protocol stack through the action interface to form a window quota constraint. The available window space creates back pressure on the transmission, thereby indirectly guiding the transmission distribution without replacing the kernel scheduler, achieving synergistic consistency between congestion control effect and scheduling result.

[0047] This collaborative congestion control method, based on attention-driven multi-path TCP sub-stream dependency modeling, incorporates sub-stream congestion control and sub-stream transmission resource allocation / scheduling tendency into a unified decision-making process, thereby achieving adaptive adjustment of the transmission behavior of each sub-stream. This approach balances performance metrics such as throughput, latency, and jitter under heterogeneous links and time-varying network conditions.

[0048] This invention incorporates throughput gains and latency / jitter costs into the reward calculation during the training phase, enabling the strategy to adaptively balance throughput and stability in dynamic environments. This reduces the estimated latency time (RTT) and jitter amplification, thereby improving overall transmission performance. The method can be applied to terminals or servers to achieve adaptive adjustment and performance optimization of substream transmission behavior under conditions of dynamic changes in heterogeneous networks or link states.

[0049] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A cooperative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling, characterized in that: Includes the following steps, S1. The sending end and the receiving end establish an MPTCP connection and generate N sub-streams for data transmission; S2. The state monitoring module reads the sub-stream execution data from the MPTCP protocol stack and constructs the state matrix S. t After normalization, the normalized state matrix is ​​obtained. ; S3. Use a self-attention layer to normalize the input state matrix. The features of each sub-flow are weighted and fused to obtain the representation Z of the dependencies between the fused sub-flows. t The Actor network and Critic network are output respectively. S4. Representation of the inter-stream dependencies of the input fused substreams by the Actor network Z t Generate action vector A t That is, the cwnd quota action, which will move the action vector A. t The action interface is used to write data to the MPTCP protocol stack, along with the current congestion window value in the running data of each substream. Jointly determine the effective congestion window for each sub-stream When sending data, the effective congestion window of each substream is used. As a constraint on the available sending window; S5. The Critic network represents the dependencies between the input fused substreams, Z. t With action vector A t Generate action value Q; S6, Execute action vector A t Then obtain the new state The reward R is calculated by the reward calculation module based on the substream execution data. t , to use empirical data Store in the experience replay pool; S7. The self-attention deep deterministic policy gradient agent, namely SA-DDPG agent, is composed of self-attention layer, Actor network, Critic network, reward calculation module and experience replay pool. The state monitoring module and SA-DDPG agent together constitute a cooperative congestion control model. After training, the trained cooperative congestion control model is obtained. S8. The sending end deploys the trained collaborative congestion control model in user space to achieve online adaptive control.

2. The cooperative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling as described in claim 1, characterized in that: In step S2, the sub-stream operation data includes the current congestion window value, unacknowledged data volume, smooth round-trip delay, jitter index, acknowledged data volume in the current statistical period, and transmission anomaly index for each sub-stream. The transmission anomaly index is determined by at least one of the following: number of packet losses, number of retransmissions, or number of timeouts.

3. The cooperative congestion control method based on attention-driven multi-path TCP sub-stream dependency modeling as described in claim 2, characterized in that: In step S2, the state matrix S is constructed. t After normalization, the normalized state matrix is ​​obtained. Specifically, S21. At each decision time t, the control module constructs a state matrix S according to a pre-defined definition. t : ， Where N is the number of sub-streams of the current MPTCP connection. Let be the fixed six-dimensional feature vector of the i-th sub-stream at time t. The fixed six-dimensional feature vector is composed of the sub-stream running data. S22, Regarding the state matrix S t A normalized state matrix is ​​obtained by performing a fixed-method normalization process. Normalized state matrix The normalization result of the k-th feature of the i-th subflow at time t for: ， in, Let k represent the k-th feature of the i-th sub-stream at time t, where i = 1, 2, ..., N, and N is the number of sub-streams in the current MPTCP connection; k is the feature dimension index, k = 1, 2, ..., N. This represents the preset mean value corresponding to the k-th feature. ε represents the preset standard deviation corresponding to the k-th feature; ε represents a preset positive number used to avoid the denominator being zero.

4. The cooperative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling as described in any one of claims 1-3, characterized in that: In step S3, a self-attention layer is used to normalize the input state matrix. The features of each sub-flow are weighted and fused to obtain the representation Z of the dependencies between the fused sub-flows. t Specifically, S31. Normalize the state matrix Each feature vector of each substream is mapped to a query vector Q, a key vector K, and a value vector V: ， in, These are the query mapping parameter matrix, the key mapping parameter matrix, and the value mapping parameter matrix, all of which are trainable parameter matrices. S32. Based on the query vector Q and key vector K, calculate the correlation between substreams, and obtain the attention weight matrix A by applying the normalized exponential function softmax: ， Where d is the attention dimension, used for scale normalization, and T represents transpose; S33. Apply the attention weight matrix A to the value vector V to obtain the representation Z of the inter-stream dependencies of the fused substreams. t : 。 5. The cooperative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling as described in any one of claims 1-3, characterized in that: In step S4, the Actor network represents the dependencies between the input fused substreams, Z. t Generate action vector A t That is, the cwnd quota action, which will move the action vector A. t The action interface is used to write data to the MPTCP protocol stack, along with the current congestion window value in the running data of each substream. Jointly determine the effective congestion window for each sub-stream Specifically, S41. Represent Z of the inter-stream dependencies. t The Actor network outputs action vector A after inputting it into the network. t : ， in, Let represent the congestion window quota for sub-flow i, which is a directly executable cwnd constraint and is not normalized. Without any fixed constraints, R + Represents positive real numbers; S42, Transfer motion vector A t The action interface is written to the kernel protocol stack, so that the congestion window of the i-th subflow is subject to the corresponding congestion window quota in the current control cycle. Constraints; S43, Action Vector A t After being written to the MPTCP protocol stack via the action interface, it is compared with the current congestion window value in the running data of each sub-stream. Jointly determine the effective congestion window for this sub-stream : ， Here, clip() is the clipping function. The upper and lower bounds of the preset quota are defined, and min is the minimum value function.

6. The cooperative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling as described in claim 2 or 3, characterized in that: In step S6, the reward calculation module calculates the reward R based on the sub-stream running data. t Specifically, S61. At the end of each control cycle, calculate the effective throughput TH based on the byte count of the received end. t : ， Among them, TH t This represents the effective throughput within the control period corresponding to time t, where T is the length of the control period. This indicates that only confirmation events occurring within the interval (t−T,t] are counted; This represents the number of data bytes acknowledged within the interval (t−T,t]. S62. The time delay index D is characterized by the average value of the time delay estimate after each sub-slip processing, i.e., the smoothed RTT. t : ， in, This is the smoothed RTT estimate of the i-th subflow at time t; S63. Jock index is measured by the change in smoothed RTT between adjacent time intervals. t : ， Among them, J t This represents the jitter index within the control period corresponding to time t; N is the number of sub-streams of the current MPTCP connection. are the smoothed RTT estimates of the i-th subflow at times t and t-1, respectively; This represents the absolute value of the change in the smoothed RTT estimate between two adjacent times of the i-th substream; S64. Calculate the reward at time t based on the reward function weighted by throughput and stability indices. : ， in, This is a packet loss cost metric used to characterize the packet loss, retransmission, or timeout situation of each sub-stream within the current control cycle; To avoid the undefined positive number for the logarithmic function when the effective throughput is zero, and These are preset weighting coefficients.

7. The cooperative congestion control method based on attention-driven multi-path TCP sub-flow dependency modeling as described in any one of claims 1-4, characterized in that: In step S7, the cooperative congestion control model is trained using the deep deterministic policy gradient algorithm.

Images