Hybrid congestion control method and system based on dynamically adjusting monitoring time period

Abstract

The application provides a kind of hybrid congestion control method and system based on dynamic adjustment monitoring time period, it is related to communication technical field, method includes obtaining real-time network state, network state data is obtained by calculating round-trip delay variation according to network state;The network state data is normalized to standardize network state data, and the normalized network state data is compared with the preset threshold to classify the normalized network state data, and the monitoring time period value is calculated according to the classification result to dynamically adjust the monitoring time period value;Determine whether the current time reaches the monitoring time period value;If yes, then according to the monitoring time period value, the current congestion window is switched to the optimal action adjustment congestion window generated based on deep reinforcement learning.The application compared with traditional fixed monitoring time period scheme, not only has stronger environmental generalization ability, but also can quickly adapt to network condition change through real-time state classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a hybrid congestion control method and system based on dynamic adjustment of monitoring time period. BACKGROUND

[0002] Congestion control algorithm is the core mechanism in network protocol (e.g. Transmission Control Protocol, TCP) to avoid or alleviate network congestion, where TCP includes various heuristic congestion control algorithms. When data traffic in the network exceeds the processing capacity of links, routers or switches, it will lead to increased round-trip time, rising packet loss rate, and decreased throughput, etc. Congestion control algorithm balances network load and guarantees stability and efficiency of data transmission by dynamically adjusting the transmission rate or window size of the sender.

[0003] The core of congestion control algorithm is to dynamically adjust the congestion window (cwnd) or transmission rate of the sender by sensing network state (such as packet loss, round-trip time change, throughput). The "congestion window" represents the maximum number of data packets that the sender can continuously send before receiving an acknowledgment (ACK), and reflects the carrying capacity of the network, which is the most commonly used control index. Common network state signals include packet loss, RTT (Round-Trip Time, RTT, round-trip time) and throughput, where packet loss is usually considered as a direct sign of congestion (such as router buffer overflow leading to packet loss); RTT growth may indicate link or router queue accumulation, indicating potential congestion; throughput represents the difference between actual transmission rate and bottleneck link bandwidth, reflecting resource utilization efficiency.

[0004] Currently, traditional congestion control algorithms (such as TCP, CUBIC, BBR) rely on fixed rules to adjust the congestion window, which is difficult to adapt to dynamic network environment; while congestion control algorithms based on deep reinforcement learning (Deep Reinforcement Learning, DRL) (such as Aurora) have high adaptability, but have large computational overhead and insufficient stability. The existing hybrid method (such as Orca, Spine) combining heuristic-based congestion control algorithm and deep reinforcement learning-based congestion control algorithm uses a fixed monitoring time period to switch between heuristic-based congestion control algorithm and deep reinforcement learning-based congestion control algorithm, but has certain limitations, for example: when the monitoring time period is too large, the heuristic-based congestion control algorithm dominates, which may lead to increased latency and packet loss. On the contrary, when the monitoring time period is too small, the deep reinforcement learning-based congestion control algorithm plays a more important role, leading to instability in the initial stage and higher CPU utilization. SUMMARY

[0005] Based on this, the purpose of the present application is to provide a hybrid congestion control method and system based on dynamic adjustment of monitoring time period, to solve the limitation problem caused by the fixed monitoring time period of the prior art hybrid method.

[0006] In one aspect, the present application provides a hybrid congestion control method based on dynamic adjustment of monitoring time period, the method comprising:

[0007] Obtaining real-time network state, calculating round-trip time change according to the network state to obtain network state data, the network state data including round-trip time, round-trip time change rate and packet loss rate;

[0008] Normalizing the network state data to standardize the network state data, the standardized network state data including standardized round-trip time and standardized round-trip time change rate, comparing the standardized network state data with a preset threshold to classify the normalized network state data, and calculating a monitoring time period value according to the classification result to dynamically adjust the monitoring time period value;

[0009] Judging whether the current time reaches the monitoring time period value;

[0010] If yes, switching the current congestion window to the congestion window adjusted by the optimal action generated based on deep reinforcement learning according to the monitoring time period value.

[0011] The above hybrid congestion control method based on dynamic adjustment of monitoring time period can effectively identify congestion noise data in different network environments through the hybrid congestion control mechanism of dynamic adjustment of monitoring time period, the congestion noise data including random packet loss and delay jitter, significantly improving the accuracy of transmission control. Compared with the traditional fixed monitoring time period scheme, the dynamic adjustment method of combining the congestion control algorithm based on deep reinforcement learning with the heuristic congestion control algorithm not only has stronger environmental generalization ability, but also can quickly adapt to network condition changes through real-time state classification.

[0012] In addition, the hybrid congestion control method based on dynamic adjustment of monitoring time period according to the present application can have the following additional technical features:

[0013] Further, the calculation formula of the standardized round-trip time is:

[0014] ;

[0015] In the formula, RTT represents the standardized round-trip time; RTT t RTT represents the RTT value at the current t time, RTT minis the observed minimum RTT; wherein, RTT represents a round-trip delay;

[0016] The calculation formula of the normalized round-trip delay change rate is:

[0017] ;

[0018] In the formula, denotes the normalized round-trip delay change rate, RTT t-1 denotes the RTT value at t-1 time;

[0019] The calculation formula of the packet loss rate is:

[0020] ;

[0021] In the formula, Loss_pck t denotes the number of lost packets, Send_pck t denotes the number of sent packets.

[0022] Further, in the step of comparing the normalized network state data with the preset threshold to classify the normalized network state data, the preset threshold of the normalized round-trip delay is a first threshold, the preset threshold of the normalized round-trip delay change rate is a second threshold, and the preset threshold of the packet loss rate is a third threshold; the method further comprises:

[0023] When the normalized round-trip delay is not greater than the first threshold, the normalized round-trip delay change rate is less than the second threshold, and the packet loss rate is less than the third threshold, the network state of the bottleneck link is in a stable non-congestion state, and then the monitoring time period is increased to preferentially adopt the heuristic congestion control algorithm for transmission rate decision-making.

[0024] In the formula, T denotes the monitoring time period.

[0025] ;

[0026] In the formula, MTP n denotes the current monitoring time period. MTP n-1 denotes the last monitoring time period; S(x) denotes a Sigmoid function, wherein, x denotes a sample randomly selected from a normal distribution of the inverse of the round-trip delay.

[0027] Further, in the step of comparing the normalized network state data with the preset threshold to classify the normalized network state data, the preset threshold of the normalized round-trip delay is a first threshold, the preset threshold of the normalized round-trip delay change rate is a second threshold, and the preset threshold of the packet loss rate is a third threshold; the method further comprises:

[0028] when the standardized round-trip delay is not greater than the first threshold value, the standardized round-trip delay change rate is not less than the second threshold value, and the packet loss rate is less than the third threshold value, the network state of the bottleneck link is close to a congestion state, and then the monitoring time period is reduced to determine the sending rate by means of the congestion control algorithm based on deep reinforcement learning;

[0029] wherein the calculation formula of the monitoring time period is:

[0030]

[0031] wherein, MTP n represents the current monitoring time period; MTP n-1 represents the last monitoring time period; represents that the monitoring time period is dynamically reduced at an exponential rate according to the standardized round-trip delay increase degree.

[0032] Further, in the step of comparing the standardized network state data with the preset threshold value to classify the normalized network state data, the preset threshold value of the standardized round-trip delay is the first threshold value, the preset threshold value of the standardized round-trip delay change rate is the second threshold value, and the preset threshold value of the packet loss rate is the third threshold value; the method further comprises:

[0033] when the standardized round-trip delay is not greater than the first threshold value, and the packet loss rate is not less than the third threshold value, the network state of the bottleneck link is not congested, but packet loss will occur, and then the congestion window is switched to the window adjusted based on the deep reinforcement learning action;

[0034] wherein the conversion formula of converting the action into the congestion window is:

[0035]

[0036] wherein, cwnd represents the congestion window, represents the optimal action a t adjusted based on deep reinforcement learning at t time; represents the optimal action a t-1 adjusted based on deep reinforcement learning at t-1 time.

[0037] Further, in the step of comparing the standardized network state data with the preset threshold value to classify the normalized network state data, the preset threshold value of the standardized round-trip delay is the first threshold value, the preset threshold value of the standardized round-trip delay change rate is the second threshold value, and the preset threshold value of the packet loss rate is the third threshold value; the method further comprises: ​​

[0038] When the standardized round-trip delay is greater than the first threshold value, the network has been unable to bear additional data packets, then the monitoring time period is reduced.

[0039] The calculation formula of the monitoring time period is:

[0040] ;

[0041] In the formula, MTP n represents the current monitoring time period; MTP n-1 represents the last monitoring time period; represents that the monitoring time period is dynamically reduced at an exponential rate according to the standardized round-trip delay increase.

[0042] Another aspect of the present application provides a hybrid congestion control system based on dynamically adjusting the monitoring time period, the system comprising:

[0043] An acquisition module is configured to acquire real-time network status, calculate round-trip delay changes to obtain network status data according to the network status, and the network status data comprises round-trip delay, round-trip delay change rate and packet loss rate;

[0044] A classification module is configured to normalize the network status data to standardize the network status data, the standardized network status data comprises standardized round-trip delay and standardized round-trip delay change rate, compare the standardized network status data with a preset threshold to classify the normalized network status data, and calculate a monitoring time period value according to the classification result to dynamically adjust the monitoring time period value.

[0045] A judgment module is configured to judge whether the current time reaches the monitoring time period value.

[0046] A first execution module is configured to switch the current congestion window to the congestion window adjusted by the optimal action generated based on deep reinforcement learning according to the monitoring time period value if the current time reaches the monitoring time period value.

[0047] Another aspect of the present application provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the hybrid congestion control method based on dynamically adjusting the monitoring time period as described above.

[0048] Another aspect of the present application also provides a data processing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executes the program to implement the hybrid congestion control method based on dynamically adjusting the monitoring time period as described above. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 Flow chart of the mixed congestion control method based on dynamic adjustment of monitoring time period in the embodiment of the present application;

[0050] Figure 2 Principle diagram of the dynamic switching mechanism of monitoring time period in the embodiment of the present application;

[0051] The following detailed description will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0052] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The drawings show several embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description of the present application herein is only for the purpose of describing the specific embodiments and is not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0054] In order to solve the problem of fixed monitoring time period in the mixed congestion control algorithm based on heuristic congestion control algorithm and reinforcement learning-based congestion control algorithm in the prior art, the present application provides a mixed congestion control method and system based on dynamic adjustment of monitoring time period. Specifically, the technical solution adopted by the present application collects network state and further calculates delay variation; then the obtained network state data is normalized to obtain normalized network state data; then the state analyzer is used to analyze the normalized network state data and compare it with the preset threshold value, classify it into one of the four predefined situations through comparison, and calculate the appropriate monitoring time period value; finally, according to the dynamically adjusted monitoring time period value, it is judged whether the monitoring time period value has been reached or not, if not, the heuristic congestion control algorithm is used to calculate the congestion window; if the monitoring time period value is reached, switch to the deep reinforcement learning-based congestion control algorithm to generate the optimal action to adjust the congestion window.

[0055] By the hybrid congestion control mechanism of dynamically adjusting the monitoring time period, the congestion noise data in different network environments can be effectively identified, the congestion noise data including random packet loss and delay jitter, and the accuracy of the transmission control is significantly improved. Compared with the traditional fixed monitoring time period scheme, the dynamic adjustment method of the congestion control algorithm based on deep reinforcement learning and the heuristic congestion control algorithm fusion not only has stronger environment generalization ability, but also can quickly adapt to network condition changes through real-time state classification.

[0056] In order to make the present application more comprehensible, several embodiments of the present application will be given below. However, the present application can be realized in many different forms, and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0057] Embodiment one

[0058] Please refer to Figure 1 , which is a hybrid congestion control method based on dynamic adjustment of monitoring time period in the first embodiment of the present application, the method comprising steps S101 to S106:

[0059] S101, obtaining real-time network state, calculating round-trip time change according to network state to obtain network state data.

[0060] As a specific example, the real-time network state is obtained from the acknowledged data packet according to the round-trip time period, and the time change is calculated according to the network state to obtain the network state data. In this embodiment, the network state data includes round-trip time, round-trip time change rate and packet loss rate.

[0061] S102, normalizing the network state data to standardize the network state data.

[0062] Specifically, the standardized network state data includes standardized round-trip time and standardized round-trip time change rate. The calculation formula of the standardized round-trip time is:

[0063] ;

[0064] In the formula, denotes the standardized round-trip time; RTT t denotes the RTT value at the current t time, RTT min is the observed minimum RTT; wherein RTT denotes the round-trip time.

[0065] Further, the calculation formula of the standardized round-trip time change rate is:

[0066] ;

[0067] In the formula, denotes the normalized round-trip time variation rate, RTT t-1 denotes the RTT value at t-1 time;

[0068] Further, the packet loss rate calculation formula is:

[0069] ;

[0070] In the formula, Loss t denotes the packet loss rate; Loss_pck t denotes the number of lost packets, Send_pck t denotes the number of sent packets.

[0071] S103, compare the normalized network state data with the preset threshold to classify the normalized network state data, and calculate a monitoring time period value according to the classification result to dynamically adjust the monitoring time period value.

[0072] Specifically, the preset threshold of the normalized round-trip time is a first threshold, the preset threshold of the normalized round-trip time variation rate is a second threshold, and the preset threshold of the packet loss rate is a third threshold. In this embodiment, the first threshold is 1.5, the second threshold is 0, and the third threshold is 0.2. Further, the network state of the bottleneck link can be divided into four cases, specifically:

[0073] (1) When the normalized round-trip time ≤1.5, the normalized round-trip time variation rate <0, and the packet loss rate <0.2, it indicates that the network state of the bottleneck link is a stable non-congestion state, and more data packets can be transmitted. At this time, the monitoring time period needs to be increased to preferentially use the heuristic congestion control algorithm for sending rate decision-making. At this time, the calculation formula of the monitoring time period is:

[0074] ;

[0075] In the formula, MTP n denotes the current monitoring time period; MTP n-1 denotes the last monitoring time period; S(x) denotes a Sigmoid function, wherein x represents a sample randomly selected from a normal distribution of the inverse of the round-trip time.

[0076] (2) When the standardized round-trip delay is ≤1.5, the standardized round-trip delay variation rate is ≥0, and the packet loss rate is <0.2, it indicates that the network state of the bottleneck link is close to the congestion state, although it is not in the congestion state, but the number of data packets queued in the cache of the switch on the bottleneck link increases, at this time, the monitoring time period needs to be slowly reduced to maximize the transmission rate to determine the sending rate by means of the congestion control algorithm based on deep reinforcement learning. At this time, the calculation formula of the monitoring time period is:

[0077] ;

[0078] In the formula, indicates that the monitoring time period is dynamically reduced at an exponential rate according to the degree of standardized round-trip delay increase.

[0079] (3) When the standardized round-trip delay is ≤1.5, and the packet loss rate is ≥0.2, it indicates that the network state of the bottleneck link is not congested, but packet loss still occurs, which shows that the loss is mainly due to random factors, specifically, the random factors include channel errors or interference. When random packet loss occurs, the conservative deep reinforcement learning-based scheme needs to be switched to; at this time, the conversion formula of the action and the congestion window is:

[0080] ;

[0081] In the formula, cwnd indicates the congestion window, indicates the optimal action a t adjusted by deep reinforcement learning at t time; indicates the optimal action a t-1 adjusted by deep reinforcement learning at t-1 time.

[0082] (4) When the standardized round-trip delay is >1.5, that is, the normalized delay exceeds the threshold, it indicates that the network cannot bear additional data packets, in this case, the delay packet loss and the random packet loss are not distinguished, for two reasons: first, the probability of congestion packet loss is much higher than that of random packet loss at this time; second, a more conservative method is turned to, that is, the deep reinforcement learning-based method, that is, the sending rate control strategy output by the deep reinforcement learning control module, which is suitable for both types of packet loss. At this time, the monitoring time period needs to be reduced, specifically, the calculation formula of the monitoring time period is:

[0083] ;

[0084] MTP n indicates the current monitoring time period; MTP n-1 indicates the last monitoring time period; Indicates that the monitoring period is dynamically reduced at an exponential rate based on the degree of increase in normalized round-trip delay.

[0085] S104: Determine whether the current time reaches the monitoring time period value.

[0086] As a specific example, the current time is the time elapsed since the last monitoring time period, i.e., the time elapsed since the last selection of a Deep Reinforcement Learning (DRL) output. Specifically, if the current time has not reached the monitoring time period, step S105 is executed; if the current time has reached the monitoring time period, step S106 is executed.

[0087] S105: Calculate the congestion window using a heuristic congestion control algorithm.

[0088] S106. Switch the current congestion window to a congestion window adjusted based on the optimal action generated by deep reinforcement learning according to the monitoring time period value.

[0089] Specifically, the MTP controller chooses to use a heuristic congestion window or a congestion window based on deep reinforcement learning according to the current MTP value. What is switched is the calculation of the window, that is, switching to a heuristic window or a window adjusted by action based on deep reinforcement learning.

[0090] Monitoring time period dynamic switching mechanism such as Figure 2 As shown, intervals of different lengths represent continuous monitoring time periods, specifically, MTP n-1 Indicates the last monitoring time period, MTP n Indicates the current monitoring time period, MTP n+1 Indicates the next monitoring time period. During each monitoring time period, the monitoring time period value is dynamically adjusted according to the network status. If the monitoring time period threshold is not reached at this time, the heuristic congestion control algorithm represented by the orange arrow (such as TCP'scwnd, that is, the congestion window based on the heuristic congestion control algorithm) is used to control the sending rate; when the monitoring time period threshold is reached, the congestion window based on deep reinforcement learning (i.e., DRL's cwnd) represented by the purple arrow is enabled. at , a congestion window based on deep reinforcement learning. The algorithm demonstrates how it dynamically adapts heuristics and deep reinforcement learning methods over time, balancing performance and overhead to address varying network conditions.

[0091] The monitoring time period value is calculated according to the current network state to make a judgment, if the monitoring time period threshold is not reached at this time, the heuristic congestion control algorithm is continued to be used to calculate the congestion window, if the monitoring time period threshold is reached, the congestion control algorithm based on deep reinforcement learning is switched to, the network state S t The actor-critic algorithm is used to train the agent, wherein the agent refers to an entity that learns to make decisions by interacting with the environment, the policy network generates a policy decision to select an optimal action according to the current collected network state S t The action represents a sending rate ratio between consecutive time steps, and the action is converted into a congestion window through a conversion formula of the action and the congestion window. Meanwhile, a reward function is designed as a feedback signal to guide network performance optimization, wherein the network performance optimization goal is maximum throughput, minimum round-trip delay and packet loss rate. Finally, the value network evaluates the value function feedback action quality to guide policy optimization.

[0092] The reward function is:

[0093] ;

[0094] In the formula, R represents the reward of deep reinforcement learning, thr represents the current actual throughput, ζ is a coefficient, ζ = 5, loss represents the packet loss rate, thr max represents the maximum throughput, RTT represents the round-trip delay, and RTT min represents the minimum round-trip delay.

[0095] In summary, the hybrid congestion control method based on dynamic adjustment of the monitoring time period in the above-mentioned embodiments of the application can effectively identify congestion noise data in different network environments, including random packet loss and delay jitter, and significantly improve the accuracy of transmission control. Compared with the traditional fixed monitoring time period scheme, the dynamic adjustment method based on the fusion of the deep reinforcement learning-based congestion control algorithm and the heuristic congestion control algorithm not only has stronger environmental generalization ability, but also can quickly adapt to network condition changes through real-time state classification.

[0096] Embodiment two

[0097] The hybrid congestion control system based on dynamic adjustment of the monitoring time period in the second embodiment of the application comprises:

[0098] An acquisition module is configured to acquire a real-time network state, calculate a round-trip delay change to obtain network state data according to the network state, and the network state data comprises a round-trip delay, a round-trip delay change rate and a packet loss rate.

[0099] The classification module is configured to normalize the network state data to standardize the network state data, including standardizing the round-trip delay and the standardizing round-trip delay change rate, compare the standardized network state data with a preset threshold to classify the normalized network state data, and calculate a monitoring time period value according to the classification result to dynamically adjust the monitoring time period value.

[0100] The judgment module is configured to judge whether the current time reaches the monitoring time period value.

[0101] The first execution module is configured to, if the current time reaches the monitoring time period value, switch the current congestion window to the congestion window adjusted by the optimal action generated based on the deep reinforcement learning according to the monitoring time period value.

[0102] In summary, the hybrid congestion control system based on the dynamic adjustment of the monitoring time period in the above-mentioned embodiments can effectively identify congestion noise data in different network environments, including random packet loss and delay jitter, and significantly improve the accuracy of transmission control. Compared with the traditional fixed monitoring time period scheme, the dynamic adjustment method based on the fusion of the congestion control algorithm based on deep reinforcement learning and the heuristic congestion control algorithm not only has stronger environmental generalization ability, but also can quickly adapt to network condition changes through real-time state classification.

[0103] In addition, an embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps of the method in the above-mentioned embodiments.

[0104] In addition, an embodiment of the present application also provides a data processing device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the program to realize the steps of the method in the above-mentioned embodiments.

[0105] The logic and / or steps represented in the flowchart and / or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination of the above. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0106] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, using suitable methods, before being stored in a computer memory.

[0107] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the embodiments described above, various steps or methods can be implemented, for example, in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following techniques, which are well known in the art, can be used to implement the application: a hybrid of the techniques mentioned above; a combination of one or more of the techniques mentioned above; or one or more other techniques suitable for use in the computer-based systems described above.

[0108] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like is intended to mean that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In the specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0109] Although embodiments of the application have been shown and described, it would be recognized by those of ordinary skill in the art that various changes, modifications, substitutions, and alterations can be made hereto without departing from the spirit and scope of the application. The scope of the application is limited only by the claims and the equivalents thereof.

Claims

1. A hybrid congestion control method based on dynamic adjustment of monitoring time period, characterized in that: Methods include: Acquire real-time network status, and calculate round-trip delay change based on the network status to obtain network status data, the network status data including round-trip delay, round-trip delay change rate, and packet loss rate; performing normalization processing on the network status data to standardize the network status data, the standardized network status data including a standardized round-trip delay and a standardized round-trip delay change rate, comparing the standardized network status data with a preset threshold to classify the normalized network status data, and calculating a monitoring time period value based on the classification result to dynamically adjust the monitoring time period value; Determine whether the current time reaches the monitoring time period value; If so, the current congestion window is switched to the congestion window adjusted based on the optimal action generated by deep reinforcement learning according to the monitoring time period value; In the step of comparing the normalized network status data with preset thresholds to classify the normalized network status data, the preset threshold of the normalized round-trip delay is a first threshold, the preset threshold of the normalized round-trip delay variation rate is a second threshold, and the preset threshold of the packet loss rate is a third threshold; When the normalized round-trip delay is not greater than the first threshold, the normalized round-trip delay change rate is less than the second threshold, and the packet loss rate is less than the third threshold, the network state of the bottleneck link is stable and non-congested. In this case, the monitoring period is increased to prioritize the use of the heuristic congestion control algorithm for sending rate decisions. The calculation formula for the monitoring time period is: ； Where, MTP n Indicates the current monitoring time period; MTP n-1 Indicates the last monitoring time period; S(x) represents the Sigmoid function, , where x represents a sample randomly selected from the normal distribution of the inverse of the round-trip delay; When the normalized round-trip delay is no greater than a first threshold, the normalized round-trip delay change rate is no less than a second threshold, and the packet loss rate is less than a third threshold, the network state of the bottleneck link is close to congestion. The monitoring period is reduced to determine the sending rate using a congestion control algorithm based on deep reinforcement learning. The calculation formula for the monitoring time period is: ； Where, MTP n Indicates the current monitoring time period; MTP n-1 Indicates the last monitoring time period; Indicates that the monitoring period is dynamically reduced at an exponential rate based on the degree of increase in the normalized round-trip delay; Indicates the normalized round-trip delay.

2. The hybrid congestion control method based on dynamic adjustment of monitoring time period according to claim 1, characterized in that: The formula for calculating the normalized round-trip delay is: ； Where, RTT t Indicates the RTT value at the current time t, RTT min is the minimum observed RTT; where RTT represents the round-trip time; The formula for calculating the normalized round-trip delay variation rate is: ； Where, Indicates the normalized round-trip delay change rate, RTT t-1 Indicates the RTT value at time t-1; The packet loss rate calculation formula is: ； Where Loss_pck t Indicates the number of packet losses, Send_pck t Indicates the number of packets sent.

3. The hybrid congestion control method based on dynamic adjustment of monitoring time period according to claim 1, characterized in that: In the step of comparing the normalized network status data with preset thresholds to classify the normalized network status data, the preset threshold for the normalized round-trip delay is a first threshold, the preset threshold for the normalized round-trip delay variation rate is a second threshold, and the preset threshold for the packet loss rate is a third threshold; the method further includes: When the normalized round-trip delay is no greater than a first threshold and the packet loss rate is no less than a third threshold, the network status of the bottleneck link is not congested, but packet loss occurs. In this case, the congestion window is switched to a window adjusted by deep reinforcement learning. The conversion formula for converting actions into congestion windows is: ； Where, cwnd represents the congestion window, Represents the optimal action a generated based on deep reinforcement learning at time t t Adjusted congestion window; Represents the optimal action a generated based on deep reinforcement learning at time t-1 t-1 Adjusted congestion window.

4. The hybrid congestion control method based on dynamic adjustment of monitoring time period according to claim 1, characterized in that: In the step of comparing the normalized network status data with preset thresholds to classify the normalized network status data, the preset threshold for the normalized round-trip delay is a first threshold, the preset threshold for the normalized round-trip delay variation rate is a second threshold, and the preset threshold for the packet loss rate is a third threshold; the method further includes: When the normalized round-trip delay is greater than the first threshold, the network can no longer bear the additional data packets, and the monitoring time period is reduced; The calculation formula for the monitoring time period is: ； Where, MTP n Indicates the current monitoring time period; MTP n-1 Indicates the last monitoring time period; Indicates that the monitoring period is dynamically reduced at an exponential rate based on the degree of increase in normalized round-trip delay.

5. A hybrid congestion control system based on dynamic adjustment of monitoring time period, characterized in that: The system comprises: An acquisition module is used to obtain real-time network status and calculate the round-trip delay change based on the network status to obtain network status data, the network status data including round-trip delay, round-trip delay change rate and packet loss rate; a classification module, configured to perform normalization processing on the network status data to standardize the network status data, the standardized network status data including a standardized round-trip delay and a standardized round-trip delay change rate, compare the standardized network status data with a preset threshold to classify the normalized network status data, and calculate a monitoring time period value based on the classification result to dynamically adjust the monitoring time period value; A judgment module is used to judge whether the current time reaches the monitoring time period value; A first execution module is configured to switch the current congestion window to a congestion window adjusted by an optimal action generated based on deep reinforcement learning according to the monitoring time period value if the current time reaches the monitoring time period value; In the step of comparing the normalized network status data with preset thresholds to classify the normalized network status data, the preset threshold of the normalized round-trip delay is a first threshold, the preset threshold of the normalized round-trip delay variation rate is a second threshold, and the preset threshold of the packet loss rate is a third threshold; When the normalized round-trip delay is not greater than the first threshold, the normalized round-trip delay change rate is less than the second threshold, and the packet loss rate is less than the third threshold, the network state of the bottleneck link is stable and non-congested. In this case, the monitoring period is increased to prioritize the use of the heuristic congestion control algorithm for sending rate decisions. The calculation formula for the monitoring time period is: ； Where, MTP n Indicates the current monitoring time period; MTP n-1 Indicates the last monitoring time period; S(x) represents the Sigmoid function, , where x represents a sample randomly selected from the normal distribution of the inverse of the round-trip delay; When the normalized round-trip delay is no greater than a first threshold, the normalized round-trip delay change rate is no less than a second threshold, and the packet loss rate is less than a third threshold, the network state of the bottleneck link is close to congestion. The monitoring period is reduced to determine the sending rate using a congestion control algorithm based on deep reinforcement learning. The calculation formula for the monitoring time period is: ； Where, MTP n Indicates the current monitoring time period; MTP n-1 Indicates the last monitoring time period; Indicates that the monitoring period is dynamically reduced at an exponential rate based on the degree of increase in the normalized round-trip delay; Indicates the normalized round-trip delay.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the hybrid congestion control method based on dynamic adjustment of the monitoring time period as claimed in any one of claims 1 to 4 is implemented.

7. A data processing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the hybrid congestion control method based on dynamic adjustment of the monitoring time period according to any one of claims 1 to 4 is implemented.

Images