An Adaptive Congestion Control Method for Wireless Networks Based on Model and Data Fusion

Abstract

A wireless network adaptive congestion control method based on model and data fusion, which solves the problems that the traditional congestion control method has too large adjustment granularity of the congestion window and is difficult to flexibly adjust according to different service requirements. This method is divided into two modules, the traditional congestion control module and the learning module. The algorithm is divided into four stages, namely the exploration period, the transition period, the evaluation period and the decision period. By default, the traditional congestion control method is adopted in the initial stage of the algorithm. At the same time, the congestion window sizes obtained by the two algorithms are continuously compared, and the comparison is carried out during the evaluation, and the algorithm with better performance is selected and applied in the decision period. By combining the traditional congestion control and the learning-based congestion control, the transmission efficiency of the system and the adaptability to complex network environments are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of communication networks, and particularly relates to a wireless network adaptive congestion control method based on model and data fusion. Background Art

[0002] With the rapid development of the mobile Internet and the miniaturization of terminal access modules, more and more terminals have multiple network access modes. Users can use network information services such as electronic invoices, mobile communications, and video conferencing through different access methods according to different requirements in different network scenarios. However, currently, the Transmission Control Protocol (TCP) mode can only use one of the network interfaces for data transmission, which reduces the utilization rate of network resources and cannot meet the high-efficiency transmission requirements of massive electronic invoice data, audio and video, etc. Therefore, in order to make full use of network resources, multi-path transmission technology has emerged. The Internet Engineering Task Force (IETF) proposed the Multipath Transmission Control Protocol (MPTCP) in 2009. Based on the inheritance of TCP, this protocol uses multi-interface technology to establish multiple links, improves the utilization rate of network bandwidth, and reduces the risk of service interruption.

[0003] Same as traditional TCP, congestion control is one of the key technologies of MPTCP and has an important impact on the performance of MPTCP. Therefore, it is necessary to analyze the application scenarios of MPTCP and optimize the congestion control of MPTCP to achieve efficient and reliable multi-path transmission. Designing an MPTCP congestion control method faces two challenges: in a multi-path transmission environment, each sub-flow maintains a congestion window separately, resulting in multiple MPTCP flows over-occupying bandwidth, and in severe cases, TCP flows cannot work properly; in a heterogeneous network environment with large differences in link quality, the stability of multi-path transmission deteriorates and the bandwidth utilization rate decreases. So far, many congestion control methods have been proposed, which can be mainly divided into two categories, namely traditional congestion control methods and learning-based congestion control methods. Traditional congestion control methods have been implemented in many kernels and have strong practicability. With the rapid development and application of machine learning, more and more learning-based congestion control methods have been proposed. The intelligent agent dynamically adjusts the size of the congestion window according to the current network conditions. Such algorithms show strong flexibility and applicability, but few have been actually deployed. Summary of the Invention

[0004] The present invention aims to provide a congestion control method that combines traditional methods and learning-based methods for device terminals with multiple network interfaces. During data transmission, the congestion control method is adaptively switched according to the network environment and system performance to improve the practicality, flexibility, and stability of the algorithm.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A wireless network adaptive congestion control method based on model and data fusion, including:

[0007] Traditional congestion control method MPTCP: It can be regarded as a multipath transmission protocol extended from TCP. The congestion control adopted by the subflows in MPTCP is similar to traditional TCP. At the sending end, a congestion window variable needs to be maintained. The size of the congestion window depends on the current congestion level of the network. The rule for adjusting the congestion window is as follows: As long as there is no congestion in the network, in order to make full use of the current network resources, the congestion window needs to be increased, so as to send more data packets into the network; when congestion occurs in the network, at this time, there are more data packets cached in the network, and the congestion window needs to be decreased to reduce the number of data packets sent into the network.

[0008] Learning-based congestion control method: By defining a utility function as the reward value, this utility function can be flexibly adjusted according to different service preferences, so as to make full use of network resources and has strong flexibility. At the same time, in the learning-based congestion control method, the agent continuously observes the network state, and each decision result is based on the network condition at that time. Therefore, this algorithm has strong applicability. At present, many learning-based congestion control methods have been proposed, and these algorithms perform more excellently than traditional congestion control methods.

[0009] Specific steps:

[0010] Step S1: First, the algorithm enters the exploration period. The traditional congestion control method is adopted.

[0011] Step S2: Continuously collect network feature information as the input of the learning-based congestion control method.

[0012] Step S3: Train the learning model and compare the congestion window sizes of the learning-based congestion control method and the traditional congestion control method. If the difference is greater than the given threshold, it will enter the transition period.

[0013] Step S4: During the transition period, it is still necessary to continuously compare the difference between the two. If the duration of the difference between the two is greater than the given time threshold, it is necessary to enter the evaluation period; otherwise, it means that the difference between the two is greater than the specified threshold only due to network fluctuations and will not last, so the traditional congestion control will continue to be used.

[0014] Step S5: Divide the evaluation period into two evaluation intervals, which are respectively used to attempt the learning-based congestion control method and the traditional congestion control method, and take the optimal strategy as x prev ; After the evaluation period ends, enter the decision-making period.

[0015] Step S6: In the decision-making period, adopt the optimal congestion control decision x prev of the previous control period, and at the same time calculate the utility function u(x prev ). In order to ensure the effectiveness of the optimal control decision result x prev , this value needs to be continuously updated in each control period. The formula used is as follows:

[0016]

[0017] Further, in the above-mentioned step S1, the congestion window adjustment rule is: as long as there is no congestion in the network, in order to make full use of the current network resources, the congestion window needs to be increased, so as to send more data packets into the network; when the network is congested, at this time there are more data packets cached in the network, and the congestion window needs to be decreased to reduce the number of data packets sent into the network. In the present invention, the traditional congestion control method adopts the default congestion control method LIA (Linked Increase Algorithm) of the MPTCP protocol. This congestion control mechanism can be expressed as:

[0018] 1) For the i-th sub-flow, when the sender receives the packet acknowledgment signal, the congestion window w i will be increased, and the increased amplitude Δw i is as shown in the formula:

[0019]

[0020] where n represents the total number of sub-flows, and RTT i represents the round-trip delay of path i.

[0021] 2) For the i-th sub-flow, when receiving the packet loss signal, the congestion window w i will be halved, and the decreased amplitude is

[0022] Furthermore, in the learning-based congestion control method in step S2, a utility function is defined as the reward value. This utility function can be flexibly adjusted according to different service preferences, so as to make full use of network resources and has strong flexibility. At the same time, in the learning-based congestion control method, the agent continuously observes the network state, and each decision result is based on the current network condition. Therefore, this algorithm has strong applicability. At present, many learning-based congestion control methods have been proposed, and these algorithms perform better than traditional congestion control methods. The learning-based congestion control method adopted in the present invention is designed as follows:

[0023] 1) State set: s t,i ={I t,i ,R t,i ,L t,i} represents the state of path i at time t, where I t,i =αI t-1,i +(1-α)(ACK t,i -ACK t-1,i ) represents the average round-trip delay of path i, represents the average sending window of path i, L t,i represents the packet loss rate of path i, and α represents the average round-trip delay adjustment factor.

[0024] 2) Action: a t =(μ t ,υ t ), where μ t , υ t represent the adjustment parameters of congestion control. The adjustment formula for the congestion control window is: w t+1 ←μ t w t +υ t , and the values of μ t and υ t are selected from a finite set, where w t represents the window size at time t.

[0025] 3) Reward: where Tput t,i represents the throughput of path i at time t, RTT t,i represents the round-trip delay of path i at time t, L t,i represents the packet loss rate of path i at time t, and β and γ represent the delay penalty factor and the packet loss rate penalty factor respectively.

[0026] The beneficial effects achieved by the present invention are:

[0027] A wireless network adaptive congestion control method based on model and data fusion is proposed. The algorithm is mainly divided into two parts: traditional congestion control and learning-based congestion control algorithm. The algorithm divides each control cycle into four periods and automatically selects a more suitable congestion control method according to the network environment and system performance, improving the adaptability of the system. According to the simulation results, the algorithm can achieve a lower packet loss rate and a lower one-way transmission delay. At the same time, the algorithm also ensures the fairness of multiple data streams when sharing a bottleneck link and realizes a lower computational overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is an architecture diagram of a multipath transmission control based on the MPTCP protocol in an embodiment of the present invention.

[0029] Figure 2 It is a schematic flowchart of a wireless network adaptive congestion control method based on model and data fusion in an embodiment of the present invention.

[0030] Figure 3 It is a block diagram of a wireless network adaptive congestion control method based on model and data fusion in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings of the specification.

[0032] Based on the communication application scenario of the MPTCP protocol, the present invention proposes a wireless network adaptive congestion control method based on model and data fusion, as Figure 2 shown, including the following steps:

[0033] (1) Probing period. The traditional congestion control is adopted in the probing period. At the same time, it is necessary to continuously collect network feature information as the input of the learning-based congestion control method. In the exploration period, the learning-based congestion control method is used as a backup algorithm, and the congestion control window obtained by learning is used as a comparison value. If the difference between the congestion window size obtained by the traditional congestion control method and the congestion window size obtained by learning is greater than a given threshold, the transition period will be entered.

[0034] (2) Transition period. At this time, it is still necessary to continuously compare the difference between the two. If the difference between the two persists for a given time threshold, the evaluation period needs to be entered; otherwise, if the duration of the difference is less than the specified threshold, it means that the difference between the two is greater than the specified threshold only due to network fluctuations and will not persist. Therefore, the traditional congestion control method will continue to be used. When a certain time has passed in the transition period, the evaluation period will be entered. Whether the duration of the difference reaches the specified threshold or not, the evaluation period will be entered from the transition period.

[0035] (3) Evaluation period. The evaluation period is further divided into two evaluation intervals, which are respectively used to attempt the learning-based congestion control method and the traditional congestion control method, and calculate the utility function values u(x rl ) and u(x cl ). Which algorithm to use for the first evaluation interval depends on the smaller value of x rl and x cl . x rl and x cl respectively represent the window size obtained by the learning-based congestion control method and the window obtained by the traditional congestion control. The design purpose of the evaluation period is to make full use of the two algorithms to obtain the corresponding utility function values and provide a reference for the decision-making in the decision period.

[0036] (4) Decision period. In the decision period, the optimal congestion control decision x prev of the previous control cycle is adopted, and the utility function value u(x prev ) is calculated. To ensure the effectiveness of the optimal control decision result x prev , this value needs to be continuously updated in each control cycle.

[0037] The traditional congestion control described in the present invention adopts LIA (Linked Increase Algorithm). This congestion control mechanism can be expressed as:

[0038] 1) For the i-th sub-flow, when the sender receives the packet acknowledgment signal, the congestion window w i will be increased, and the increased amplitude Δw i is as shown in the formula:

[0039]

[0040] where n represents the total number of sub-flows, and RTT i represents the round-trip delay of path i.

[0041] 2) For the i-th sub-flow, when the packet loss signal is received, the congestion window w i will be halved, and the reduced amplitude is

[0042] The specific steps for obtaining the congestion window size by learning described in the present invention are as follows:

[0043] Step A1: The system obtains network environment information, including round-trip delay, throughput, packet loss rate, as the model input, and stores it in the state set S t,i . s t,i = {I t,i , R t,i , L t,i} represents the state of path i at time t, where It,i = αI t-1,i + (1 - α)(ACK t,i - ACK t-1,i represents the average round-trip delay of path i, represents the average sending window of path i, L t,i represents the packet loss rate of path i, and α represents the average round-trip delay adjustment factor.

[0044] Step A2: The learning model makes a response action a according to the obtained state t = (μ t , υ t ), where μ t , υ t represent the adjustment parameters of congestion control. The adjustment formula for the congestion control window is: w t+1 ← μ t w t + υ t , and the values of μ t and υ t are selected from a finite set, where w t represents the window size at time t.

[0045] Step A3: According to the action taken by the learning model, the system will generate a reward, where Tput t,i represents the throughput of path i at time t, RTT t,i represents the round-trip delay of path i at time t, L t,i represents the packet loss rate of path i at time t, and β and γ represent the delay penalty factor and the packet loss rate penalty factor respectively.

[0046] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. Any equivalent modification or change made by those of ordinary skill in the art according to the disclosure of the present invention shall be included in the protection scope recorded in the claims.

Claims

1. A method for adaptive congestion control in a wireless network based on model and data fusion, characterized in that: Combining traditional congestion control and learning-based congestion control methods, continuously comparing the transmission rates obtained by the two methods during the transmission process; in each control cycle, select the algorithm with better performance through comparison and apply it to the next control cycle; the specific steps are as follows: Step S1: First, enter the exploration period; adopt the traditional congestion control method. Step S2: Continuously collect network feature information as the input of the learning-based congestion control method. Step S3: Train the learning model, and compare the congestion window sizes of the learning-based congestion control method and the traditional congestion control method; if the difference is greater than a given threshold, enter the transition period; if the difference is less than the given threshold, continue to use the traditional congestion control method. Step S4: During the transition period, still continuously compare the difference between the two. If the duration of the difference between the two is greater than a given time threshold, enter the evaluation period; otherwise, it means that the difference between the two is greater than the specified threshold due to network fluctuations and will not last, so continue to use the traditional congestion control method. Step S5: Divide the evaluation period into two evaluation intervals, which are respectively used to attempt the learning-based congestion control method and the traditional congestion control method, and select the optimal strategy as x prev ; After the evaluation period ends, enter the decision-making period; Step S6. During the decision-making period, adopt the optimal congestion control decision x of the previous control cycle prev , and at the same time calculate the utility function u(x prev ). To ensure the effectiveness of the optimal control decision result x prev , update this value continuously in each control cycle; the formula adopted is as follows. In the formula, x rl and x cl respectively represent the window size obtained by the learning-based congestion control method and the window obtained by the traditional congestion control:

2. A method for adaptive congestion control in a wireless network based on model and data fusion according to claim 1, characterized in that: In step S1, traditional congestion control is adopted during the exploration period, and the congestion window adjustment rule is that when there is no congestion in the network, the congestion window is increased to send more data packets into the network; When congestion occurs in the network, the congestion window is reduced to reduce the number of data packets sent into the network.

3. A method for adaptive congestion control in a wireless network based on model and data fusion according to claim 2, characterized in that: The congestion control mechanism is expressed as: 1) For the i-th sub-flow, when the sender receives the packet acknowledgment signal, it will increase the congestion window w i , and the increased amplitude Δw i is as shown in the formula: where n represents the total number of sub - flows, and RTT i represents the round - trip delay of path i; 2) For the i-th sub-flow, when a packet loss signal is received, the congestion window w i is halved, and the reduction amplitude is 4. A method for adaptive congestion control in a wireless network based on model and data fusion according to claim 1, characterized in that: In step S2, the network features collected by the algorithm include the round-trip time RTT, throughput, and packet loss rate of each link.

5. A method for adaptive congestion control in a wireless network based on model and data fusion according to claim 1, characterized in that: In the step S3, the utility function values u(x rl ) and u(x cl ) of the two are calculated simultaneously; which algorithm is adopted for the first evaluation interval depends on the smaller value of x rl and x cl , where x rl and x cl respectively represent the window size obtained by the learning-based congestion control method and the window obtained by the traditional congestion control.

6. A method for adaptive congestion control in a wireless network based on model and data fusion according to claim 1, characterized in that: The learning-based congestion control method is as follows: 1) State set: s t,i = {I t,i , R t,i , L t,i} represents the state of path i at time t, where I t,i = αI t-1,i + (1 - α)(ACK t,i - ACK t-1,i ) represents the average round-trip delay of path i, R t,i represents the average sending window of path i, L t,i represents the packet loss rate of path i, and α represents the average round-trip delay adjustment factor; 2) Action: a t = (μ t , υ t ), where μ t , υ t represent the adjustment parameters of congestion control, and the adjustment formula for the congestion control window is: w t+1 ← μ t w t + υ t , μ t and υ t are selected from a finite set, where w t represents the window size at time t; 3) Reward: where Tput t,i represents the throughput of path i at time t, and RTT t,i represents the round-trip delay of path i at time t, and L t,i represents the packet loss rate of path i at time t. β and γ represent the delay penalty factor and the packet loss rate penalty factor respectively.

Images