TCP congestion control method and device based on multi-agent deep reinforcement learning

Abstract

The invention discloses a TCP congestion control method and device based on multi-agent deep reinforcement learning. The method comprises the following steps of: in a transmission control protocol (TCP), modeling a congestion control problem of a multi-stream competition bottleneck link into a Markov game process by setting a continuous statistical interval, expressing a congestion control strategy of the TCP by utilizing deep reinforcement learning and a neural network, and training an optimal congestion control strategy in a simulation environment, the adaptive problem and the fairness problem of a traditional heuristic congestion control algorithm in a dynamic and changeable network environment are fundamentally solved. According to the method, an online point change detection technology is used for dividing a statistical interval, an Actor-Critic deep reinforcement learning framework is adopted, parallel joint training is carried out on a plurality of intelligent agents, modeling and learning are directly carried out on TCP congestion control, and therefore an optimal congestion control strategy is generated.

Description

technical field [0001] The invention relates to a network communication protocol, in particular to a TCP congestion control method and device based on multi-agent deep reinforcement learning. Background technique [0002] Congestion control is the basic mechanism of the transmission control protocol (Transmission Control Protocol, TCP) to adjust the transmission rate at the sender to effectively use the network bandwidth and alleviate network congestion. The optimization of the TCP protocol focuses on the design of congestion control. The difficulty in designing an excellent congestion control algorithm lies in: First, there is noise interference in a complex network environment, and it is necessary to effectively detect network congestion. For example, the reasons for packet loss include self-congestion packet loss, random packet loss, buffer limit, etc. , if you blindly regard packet loss as network congestion, you will not be able to effectively use bandwidth resources; s...

AI Technical Summary

Problems solved by technology

[0003] Purpose of the invention: Aiming at the defects of the prior art, the present invention proposes a TCP congestion control method based on multi-agent deep reinforcement learning, which can fundamentally solve the problem of existing heuristic TCP congestion control algorithms in complex, diverse and dynamically changing network environments. Adaptability and Fairness Issues in

Images