Dynamic beam scheduling method based on deep reinforcement learning

Abstract

The invention provides a dynamic beam scheduling method based on deep reinforcement learning, which belongs to the field of multi-beam satellite communication systems. The dynamic beam scheduling method comprises the steps of: firstly, modeling a dynamic beam scheduling problem into a Markov decision process, wherein states of each time slot comprise a data matrix, a delay matrix and a channel capacity matrix in a satellite buffer, actions represent a dynamic beam scheduling strategy, and a target is the long-term reduction of accumulated waiting delay of all data packets; and secondly, solving a best action strategy by utilizing a deep reinforcement learning algorithm, establishing a Q network of a CNN+DNN structure, training the Q network, using the trained Q network to make action decisions, and acquiring the best action strategy. According to the dynamic beam scheduling method, a satellite directly outputs a current beam scheduling result according to the environment state at the moment through a large amount of autonomous learning, maximizes the overall performance of the system in the long term, and greatly reduces the transmission waiting delay of the data packets while keeping the system throughput almost unchanged.

Description

technical field [0001] The invention belongs to the field of multi-beam satellite communication systems, and relates to a dynamic beam scheduling method based on deep enhanced learning. Background technique [0002] As a supplement to the basic structure of the ground communication network, the satellite communication system has been studied for its strong global coverage, long communication distance, high system capacity, strong ability to resist major natural disasters, and the ability to provide fixed and mobile communication services. The attention of personnel and the support of the country. With the increasing demand for its capacity and the continuous consumption of spectrum resources, a multi-beam satellite communication system has been proposed. It uses multiple high-gain narrow beams to cover a large area together, which can effectively improve system performance. However, the more spot beams a satellite provides, the more transmitters are needed, and the cost of ...

AI Technical Summary

Problems solved by technology

Therefore, the dynamic beam scheduling problem of a multi-beam satellite communication system is a sequential decision-making problem in a complex environment

Images