A low-orbit satellite communication resource allocation method, device, equipment and medium

Abstract

The present disclosure provides a low-orbit satellite communication resource allocation method, device, equipment and medium, the method comprising: receiving service request signaling sent by a ground user; obtaining satellite state information at the current time according to the service request signaling; inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time; wherein the multi-agent deep reinforcement learning model comprises a first decision area and a second decision area, the first decision area is used to determine a beam scheduling scheme for the next time based on the satellite state information, and the second decision area is used to determine a frequency band and power allocation scheme for the next time based on the beam scheduling scheme for the next time. According to the embodiments of the present disclosure, real-time perception and accurate response to uneven user distribution and time-varying traffic characteristics are realized, and the dynamic adaptation capability and overall operation efficiency of the low-orbit satellite communication system resource allocation are effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of satellite communication technology, and in particular to a method, apparatus, equipment and medium for allocating low-orbit satellite communication resources. Background Technology

[0002] Driven by the global demand for digitalization and universal communication, large-scale low-Earth orbit (LEO) satellite communication resource allocation systems have become a core development direction in the aerospace communication field. Compared to traditional high-Earth orbit (HEO) satellites, LEO satellites, due to their lower orbital altitude (typically 500-2000 kilometers), possess significant technological advantages: Firstly, their signal transmission latency is greatly reduced, providing users with faster and more stable communication services, meeting the demands of high-speed services such as high-definition video and real-time data interaction; secondly, LEO satellites can form a seamless global coverage network through networking, reaching remote mountainous areas, oceans, polar regions, and other areas that are difficult for terrestrial communication base stations to cover, effectively compensating for coverage blind spots in terrestrial communication networks.

[0003] However, the current geographical distribution of ground users is significantly uneven, with a large number of users in densely populated urban areas and sparse users in vast rural and uninhabited areas. At the same time, the distribution of traffic services is also highly unbalanced, with high-frequency traffic such as video and cloud services accounting for a very high proportion in urban core areas, while remote areas mainly have low-frequency basic communication needs. This uneven distribution characteristics bring fundamental challenges to the resource adaptation of low-Earth orbit satellite communications, requiring resource allocation schemes to dynamically adjust beam scheduling and corresponding frequency band and power allocation strategies according to real-time service request signaling in order to adapt to the differentiated needs of different regions.

[0004] Current mainstream low-Earth orbit (LEO) satellite communication resource allocation technologies have many limitations. On the one hand, the uneven distribution of ground users and the spatiotemporal distribution of traffic services have become key bottlenecks restricting system performance. Traditional algorithms struggle to quickly generate resource allocation schemes for the next moment based on the current satellite status information. On the other hand, traditional satellite resource allocation algorithms are not adapted to the highly dynamic environment of LEO satellites. Greedy algorithms, while computationally simple, only focus on local optima, and simulated annealing algorithms, while capable of global optimization, have slow convergence speeds and insufficient dynamic adaptability. They cannot adjust beam scheduling schemes and frequency band and power allocation schemes in a timely manner based on real-time service request signaling, making it difficult for the system to effectively cope with the dynamic challenges brought about by uneven user distribution and time-varying traffic. Summary of the Invention

[0005] This disclosure provides a method, apparatus, device, and medium for allocating low-Earth orbit (LEO) satellite communication resources, which addresses the problem that existing LEO satellite communication resource allocation methods are unable to effectively cope with the dynamic challenges brought about by uneven user distribution and time-varying traffic.

[0006] In a first aspect, this disclosure provides a method for allocating low-Earth orbit satellite communication resources, the method comprising:

[0007] Receive service request signaling sent by ground users;

[0008] Based on the service request signaling, obtain the satellite status information at the current moment;

[0009] The satellite state information is input into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step. The multi-agent deep reinforcement learning model includes a first decision region and a second decision region. The first decision region is used to determine the beam scheduling scheme for the next time step based on the satellite state information. The second decision region is used to determine the frequency band and power allocation scheme for the next time step based on the beam scheduling scheme for the next time step. The resource allocation scheme includes the beam scheduling scheme and the frequency band and power allocation scheme for the next time step.

[0010] Furthermore, before inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step, the method further includes:

[0011] The decision-making process of the multi-agent deep reinforcement learning model is modeled as a Markov decision process, which is defined by a state space, an action space, and a reward space.

[0012] The state space includes the satellite state information, the action space is a set of resource allocation schemes for the next moment, and the reward space includes a comprehensive weighted value calculated based on the average data waiting latency and network throughput obtained after executing the resource allocation scheme for the next moment.

[0013] Furthermore, the comprehensive weighted value of the reward space is calculated using the following formula:

[0014]

[0015] In the formula, express The overall weighted value at each moment, Indicates the trade-off coefficient. express Time Satellite Storage Cell Average data wait time This indicates the number of beams equipped on the satellite. express Time Satellite beam The corresponding network throughput.

[0016] Furthermore, the step of inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step specifically includes:

[0017] The satellite state information is input into the first decision region of the multi-agent deep reinforcement learning model, and the beam scheduling scheme for the next moment is determined by the dual deep Q network (DDQN) model.

[0018] The satellite status information and the beam scheduling scheme for the next moment are input into the second decision region of the multi-agent deep reinforcement learning model, and the frequency band and power allocation scheme for the next moment is determined by the deep deterministic policy gradient (DDPG) model.

[0019] The resource allocation scheme is obtained by combining the beam scheduling scheme for the next time step and the frequency band and power allocation scheme for the next time step.

[0020] The satellite status information includes the current beam scheduling scheme, frequency band allocation scheme, power allocation scheme, and the corresponding average data latency and network throughput.

[0021] Furthermore, the determination of the beam scheduling scheme for the next time step using the dual-depth Q-network (DDQN) model specifically includes:

[0022] Using the target Q-network of the DDQN model, an ε-greedy strategy is used to select one of the discrete beam scheduling actions as a candidate beam scheduling scheme.

[0023] The Q-values ​​of the candidate beam scheduling schemes are evaluated using the target network of the DDQN model to determine the beam scheduling scheme for the next time step.

[0024] Furthermore, the determination of the frequency band and power allocation scheme for the next time step using the Deep Deterministic Policy Gradient (DDPG) model specifically includes:

[0025] Using the actor network in the DDPG model, an initial allocation strategy for frequency bands and power is generated based on the satellite state information and the beam scheduling scheme at the next moment. White noise following a Gaussian distribution is introduced into the initial allocation strategy to balance the exploration and utilization of the strategy.

[0026] The satellite state information, the beam scheduling scheme for the next time step, and the initial allocation strategy are input into the critic network of the DDPG model. The critic network evaluates the state and action pairs corresponding to the initial allocation strategy to generate an evaluation value. The initial allocation strategy is then adjusted based on the evaluation value to obtain the frequency band and power allocation scheme for the next time step.

[0027] Furthermore, after inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step, the method further includes:

[0028] The resource allocation scheme for the next time moment is stored in the onboard buffer to form a satellite data table;

[0029] During the establishment of a link with the gateway station, the satellite data table stored in the onboard buffer is sent to the ground data center so that the ground data center can update the parameters of the multi-agent deep reinforcement learning model based on the satellite data table;

[0030] Receive and load the updated model parameters from the ground data center.

[0031] Secondly, this disclosure provides a low-Earth orbit satellite communication resource allocation device, the device comprising:

[0032] The service request signaling receiving module is used to receive service request signaling sent by ground users;

[0033] The satellite status information acquisition module is connected to the service request signaling receiving module and is used to acquire the satellite status information at the current time according to the service request signaling.

[0034] A resource allocation scheme generation module, connected to the satellite state information acquisition module, is used to input the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step. The multi-agent deep reinforcement learning model includes a first decision region and a second decision region. The first decision region is used to determine the beam scheduling scheme for the next time step based on the satellite state information. The second decision region is used to determine the frequency band and power allocation scheme for the next time step based on the beam scheduling scheme for the next time step. The resource allocation scheme includes both the beam scheduling scheme and the frequency band and power allocation scheme for the next time step.

[0035] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the low-Earth orbit satellite communication resource allocation method described in the first aspect above.

[0036] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the low-Earth orbit satellite communication resource allocation method described in the first aspect.

[0037] The low-Earth orbit (LEO) satellite communication resource allocation method, apparatus, equipment, and medium disclosed herein receive service request signaling from ground users and obtain satellite status information at the current moment. Utilizing a multi-agent deep reinforcement learning model with dual decision-making regions, it dynamically generates beam scheduling and frequency band and power allocation schemes for the next moment. This achieves real-time perception and accurate response to uneven user distribution and time-varying traffic characteristics, effectively improving the dynamic adaptability and overall operational efficiency of LEO satellite communication system resource allocation. It allows limited on-board resources to be more efficiently allocated to high-demand areas, significantly improving network throughput and user service quality. This solves the problem that existing LEO satellite communication resource allocation methods are unable to effectively address the dynamic challenges posed by uneven user distribution and time-varying traffic. Attached Figure Description

[0038] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings:

[0039] Figure 1 A flowchart illustrating a low-Earth orbit satellite communication resource allocation method provided in this embodiment of the disclosure;

[0040] Figure 2 An architecture diagram of a low-Earth orbit satellite communication resource allocation system provided in this embodiment of the disclosure;

[0041] Figure 3 A schematic diagram of the resource allocation algorithm based on a multi-agent deep reinforcement learning model provided in this embodiment of the disclosure;

[0042] Figure 4 A diagram of a resource allocation algorithm based on deep reinforcement learning provided in an embodiment of this disclosure;

[0043] Figure 5 A system flowchart of a low-Earth orbit satellite communication resource allocation method provided in this embodiment of the disclosure;

[0044] Figure 6 A block diagram of a low-Earth orbit satellite communication resource allocation device provided in this disclosure embodiment;

[0045] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0046] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0047] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0048] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0049] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0050] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0051] Figure 1 A flowchart illustrating a low-Earth orbit satellite communication resource allocation method provided in this embodiment of the disclosure. (Refer to...) Figure 1 The method includes:

[0052] Step S101: Receive service request signaling sent by ground users.

[0053] Specifically, the service request signaling is sent by the ground user to the low-Earth orbit satellite via the uplink. The information it contains is used to enable the satellite to identify the user's communication needs. For example, it may include the cell identifier where the user is located to determine the user's geographical location, as well as the data type of the service requested by the user, such as high-definition video, real-time data interaction, etc., thereby providing a basis for subsequent resource allocation.

[0054] It should be noted that the low-Earth orbit satellite communication resource allocation method provided in this disclosure is applied to low-Earth orbit satellites, which are based on beam-hopping technology, equipped with phased array antennas, and generate multiple beams through beamforming technology.

[0055] Step S102: Obtain the satellite status information at the current moment according to the service request signaling.

[0056] Specifically, the satellite status information reflects the current operational status and resource usage of the satellite, and is an important basis for resource allocation decisions. The satellite status information preferably includes the current beam scheduling scheme, frequency band allocation scheme, power allocation scheme, and the corresponding average data latency and network throughput.

[0057] Step S103: Input the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step; wherein, the multi-agent deep reinforcement learning model includes a first decision region and a second decision region, the first decision region is used to determine the beam scheduling scheme for the next time step based on the satellite state information, and the second decision region is used to determine the frequency band and power allocation scheme for the next time step based on the beam scheduling scheme for the next time step, and the resource allocation scheme includes the beam scheduling scheme for the next time step and the frequency band and power allocation scheme for the next time step.

[0058] Specifically, each low-Earth orbit (LEO) satellite in space is considered an intelligent agent, and a multi-agent deep reinforcement learning model is deployed on each LEO satellite to determine the satellite-to-ground beam scheduling scheme. This multi-agent deep reinforcement learning model can autonomously learn and optimize resource allocation strategies based on input satellite state information to adapt to the dynamically changing communication environment. The first decision region focuses on beam scheduling decisions, processing current satellite state information to determine how to adjust beam direction and coverage to better match the distribution and needs of ground users. The second decision region, based on the beam scheduling scheme determined by the first decision region, further optimizes frequency band and power allocation to ensure efficient utilization of spectrum and power resources while meeting beam coverage requirements. The final generated resource allocation scheme provides comprehensive resource scheduling guidance for satellite communication in the next moment.

[0059] In some embodiments, before inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step, the method further includes:

[0060] The decision-making process of the multi-agent deep reinforcement learning model is modeled as a Markov decision process, which is defined by a state space, an action space, and a reward space.

[0061] The state space includes the satellite state information, the action space is a set of resource allocation schemes for the next moment, and the reward space includes a comprehensive weighted value calculated based on the average data waiting latency and network throughput obtained after executing the resource allocation scheme for the next moment.

[0062] Specifically, the state space comprehensively covers the environmental information required for low-Earth orbit satellites to make decisions. Besides the current beam, frequency band, and power allocation schemes, it also integrates latency and throughput metrics reflecting network service quality. The action space contains all possible combinations of resource allocation strategies that low-Earth orbit satellites can adopt in the future. The reward space, by comprehensively weighting latency and network throughput (e.g., introducing a tradeoff coefficient α to balance their importance), forms a quantitative evaluation index to guide the model to learn the optimal resource allocation strategy. For example, the comprehensive weighted value of the reward space can be calculated using the following formula to achieve overall optimization of network performance:

[0063]

[0064] In the formula, express The overall weighted value at each moment, Indicates the trade-off coefficient. express Time Satellite Storage Cell Average data wait time This indicates the number of beams equipped on the satellite. express Time Satellite beam The corresponding network throughput.

[0065] In some embodiments, the step of inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step specifically includes:

[0066] The satellite state information is input into the first decision region of the multi-agent deep reinforcement learning model, and the beam scheduling scheme for the next moment is determined by the DDQN (Double Deep Q-Network) model.

[0067] The satellite state information and the beam scheduling scheme for the next moment are input into the second decision region of the multi-agent deep reinforcement learning model, and the frequency band and power allocation scheme for the next moment is determined by the DDPG (Deep Deterministic Policy Gradient) model.

[0068] The resource allocation scheme is obtained by combining the beam scheduling scheme for the next time step and the frequency band and power allocation scheme for the next time step.

[0069] Specifically, the multi-agent deep reinforcement learning model employs a phased decision-making approach. It first utilizes the DDQN model to handle discrete beam scheduling decisions, and then uses the DDPG model to handle continuous frequency band and power allocation decisions, achieving accurate solutions to different types of resource allocation problems. The DDQN model excels at handling optimization problems in discrete action spaces and can effectively learn the optimal beam scheduling strategy; the DDPG model is suitable for optimization in continuous action spaces, further optimizing frequency band and power allocation based on a determined beam scheduling scheme. Finally, by combining the two decision results, a complete resource allocation scheme is formed to adapt to the dynamic needs of low-Earth orbit satellite communication systems.

[0070] In some embodiments, determining the beam scheduling scheme for the next time step using a dual-depth Q-network (DDQN) model specifically includes:

[0071] Using the target Q-network of the DDQN model, an ε-greedy strategy is used to select one candidate beam scheduling scheme from discrete beam scheduling actions.

[0072] The Q-values ​​of the candidate beam scheduling schemes are evaluated using the target network of the DDQN model to determine the beam scheduling scheme for the next time step.

[0073] Specifically, the DDQN model adopts a dual-network structure, where the target Q-network is responsible for generating all possible beam scheduling actions based on the current satellite state information, and selecting candidate beam scheduling schemes according to an ε-greedy strategy. This strategy randomly explores new actions with probability ε, and selects the action with the largest current Q value with probability 1-ε. The probabilistic model of its action selection can be expressed as:

[0074]

[0075] In the formula, Indicates a given observation state With action Obtain the output value of the target Q-network. These are the Q-network parameters.

[0076] Specifically, the target network is used to evaluate the Q-value of candidate beam scheduling schemes, which is the cumulative reward that the scheme may obtain in the future. By comparing the Q-values ​​of different candidate schemes, the optimal beam scheduling scheme for the next time step is finally determined, thereby improving the accuracy and stability of the decision.

[0077] In some embodiments, determining the frequency band and power allocation scheme for the next time step using the Deep Deterministic Policy Gradient (DDPG) model specifically includes:

[0078] Using the actor network in the DDPG model, an initial allocation strategy for frequency bands and power is generated based on the satellite state information and the beam scheduling scheme at the next moment. White noise following a Gaussian distribution is introduced into the initial allocation strategy to balance the exploration and utilization of the strategy.

[0079] The satellite state information, the beam scheduling scheme for the next time step, and the initial allocation strategy are input into the critic network of the DDPG model. The critic network evaluates the state and action pairs corresponding to the initial allocation strategy to generate an evaluation value. The initial allocation strategy is then adjusted based on the evaluation value to obtain the frequency band and power allocation scheme for the next time step.

[0080] Specifically, the DDPG model adopts an actor-critic framework, where the actor network acts as the policy network, responsible for generating continuous frequency band and power allocation policies based on the current satellite state information and the determined beam scheduling scheme for the next time step. Gaussian noise (e.g., the noise has a mean of 0 and a preset variance) is introduced to increase the diversity of policies and avoid getting trapped in local optima. The critic network acts as the value network, responsible for evaluating the value (i.e., Q-value) of the state-action pairs corresponding to the policies generated by the actor network, i.e., the cumulative reward that may be obtained after executing the policy, and feeding the evaluation results back to the actor network. The actor network adjusts the policy parameters based on this feedback, continuously optimizes the allocation policy, and finally obtains the frequency band and power allocation scheme for the next time step that maximizes network performance.

[0081] In some embodiments, after inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step, the method further includes:

[0082] The resource allocation scheme for the next time moment is stored in the onboard buffer to form a satellite data table;

[0083] During the establishment of a link with the gateway station, the satellite data table stored in the onboard buffer is sent to the ground data center so that the ground data center can update the parameters of the multi-agent deep reinforcement learning model based on the satellite data table;

[0084] Receive and load the updated model parameters from the ground data center.

[0085] Specifically, the onboard buffer serves as a local data storage unit for the low-Earth orbit (LEO) satellite, temporarily storing generated resource allocation schemes. The resulting satellite data table can contain resource allocation records at multiple times, providing data support for subsequent analysis and model updates. Link establishment between the LEO satellite and the gateway station typically occurs during satellite transit. At this time, using a satellite-to-ground link to transmit the satellite data table to the ground data center avoids resource consumption caused by frequent communication. After receiving the satellite data table, the ground data center updates the parameters of the multi-agent deep reinforcement learning model based on the resource allocation scheme and corresponding network performance data (such as latency and network throughput) using optimization algorithms such as gradient descent, thereby improving the model's decision-making accuracy. Upon receiving the updated model parameters, the LEO satellite loads them into its local model, enabling iterative optimization of the model and allowing subsequent resource allocation decisions to better adapt to changes in the satellite communication environment.

[0086] In one specific embodiment, the low-Earth orbit (LEO) satellite communication resource allocation method is applied to a LEO satellite communication resource allocation system, the architecture of which is shown in the figure below. Figure 2As shown, the system consists of four parts: low-Earth orbit (LEO) satellites, gateway stations, data centers, and ground users. Each LEO satellite in the LEO satellite communication resource allocation system is equipped with a phased array antenna and uses beamforming technology to generate multiple beams to cover ground users. In 5G NR, ground users are divided into cells (hexagons in the diagram). Ground users send service request signaling to the satellites in space via the uplink, which includes the user's cell ID and the data type of the requested service. When an LEO satellite receives a service request signaling from a ground user at any time during its operation, it generates a corresponding resource allocation scheme based on the user's service request. This involves scheduling idle beams and allocating available frequencies and power to cover the user's cell, while simultaneously storing the generated resource allocation scheme in the onboard buffer and forming a satellite data table. During the establishment of a link between the satellite and the gateway station, the satellite data table is forwarded to the data center via the gateway station through the satellite-to-ground downlink. The data center updates the satellite resource allocation algorithm model parameters based on the satellite data table and sends the updated model parameters back to the satellite via the satellite-to-ground uplink during the next link establishment period. The satellite updates and corrects its algorithm model based on the received parameter data to improve the overall performance of the system.

[0087] Figure 3 This is a schematic diagram illustrating the resource allocation algorithm based on a multi-agent deep reinforcement learning model in this embodiment. Figure 3 As shown, a pre-trained multi-agent deep reinforcement learning model (represented by a model parameter table) serves as the decision-making core, sensing the differentiated traffic demands of different ground cells (cell 1, cell 2, ..., cell N) in real time. Based on this environmental perception, the model outputs corresponding beam scheduling decisions, allocating more beam resources to cells with high demand, thereby achieving dynamic on-demand allocation of on-board resources.

[0088] Specifically, the output scheme of this low-orbit satellite communication resource allocation method is divided into three parts: beam scheduling, frequency band and power allocation, and algorithm update. (1) Beam scheduling: In order to meet the traffic demand of different regions, this scheme adopts beam skipping technology to schedule the satellite physical beams, and deploys a multi-agent deep reinforcement learning model on each satellite to determine the satellite-ground beam scheduling scheme. (2) Frequency band and power allocation: Based on the determination of the beam scheduling scheme, this technical scheme aims to maximize network throughput and minimize network latency, and uses multi-agent deep reinforcement learning technology to determine the frequency band and power allocation scheme of the satellite-ground beams. (3) Algorithm update: During the operation of the low-orbit satellite in orbit, the generated resource allocation scheme is stored in the onboard memory to form a satellite data table. During the establishment of a link with the ground gateway station, the data table in the onboard memory is forwarded to the data center through the gateway station. The data center updates the parameters of the onboard resource allocation algorithm model and forwards the trained model parameters to the satellite, thereby completing the update of the onboard resource allocation algorithm. This low-orbit satellite communication resource allocation method includes the following parts:

[0089] 1. System Model

[0090] The low-Earth orbit satellite communication resource allocation system has a total of One low-orbit satellite for Provides services to each community. (Used) Indicates satellite The coverage area is a set of cells, and each satellite has the same coverage area size. Each satellite can cover multiple cells simultaneously, and each cell can also see multiple satellites simultaneously. Indicates to the community A visible collection of satellites. Furthermore, each satellite is equipped with... One beam. As mentioned earlier, due to the uneven geographical distribution, the distribution of traffic demand among cells is also uneven. Therefore, it is necessary to design a reasonable beam scheduling scheme to adapt to the traffic demand distribution of different cells. Using Indicates in Time Satellite The beam scheduling scheme is expressed as follows:

[0091]

[0092] In the above formula, Indicates in Time Satellite Beam on The served cell. This technology employs frequency division multiple access (FDMA) to avoid inter-beam interference by allocating different frequency bands to different cells. Therefore, the spectrum of the low-Earth orbit satellite communication resource allocation system is evenly divided into... Segments, each with a bandwidth of .use Indicates satellite exist Beam frequency allocation at any given time, if , indicating satellite exist Frequency band Assigned to beam Conversely, the opposite is true. Use Indicates satellite Can be assigned to beam The power available for each satellite varies because the payload of each satellite is different.

[0093] In a satellite-to-ground communication system, at any given moment, the satellite receives service request signaling from multiple cells, generates traffic data based on the received signaling, and stores it in the onboard buffer for transmission. Assume each data item is [size missing]. Due to the limited capacity of the onboard buffer, data stored in the satellite buffer is discarded after a certain time limit. The buffer time limit is used... Indicates, using Indicates satellite exist Real-time storage cell The data to be transmitted is expressed as follows:

[0094]

[0095] In the formula, Indicates satellite exist Real-time storage cell The Batch of pending data (i.e., data already waiting) Number of data packets per time unit).

[0096] As can be seen from the above, to ensure the stability of the low-Earth orbit satellite communication resource allocation system, cell data should be prevented from being stored in the onboard buffer for too long and thus being discarded. This technical solution introduces an average waiting time. To represent satellite exist Real-time storage cell The average waiting time for data. Its expression is as follows:

[0097]

[0098] Because each satellite's beam coverage area includes multiple cells, each cell will simultaneously receive different beams from different satellites, thus introducing beam interference. Furthermore, since each satellite is equipped with multiple beams, different beams within the same satellite can also interfere with each other. Indicates satellite The frequency band used beam The total interference received.

[0099]

[0100] In the above formula, the first term represents inter-satellite interference, and the second term represents intra-satellite interference. Indicates the total number of satellites. This indicates the number of beams equipped on each satellite. Indicates satellite At any moment frequency band Assigned to beam The indicator variable (1 indicates allocation, 0 indicates no allocation). Indicates satellite At any moment frequency band Assigned to beam Indicator variables, Indicates satellite upper beam Transmission channel gain Indicates satellite upper beam Transmission channel gain, Indicates satellite Assigned to beam The power of the satellite Frequency band beam The transmission signal and signal power are allocated as follows Signal-to-noise ratio Represented as:

[0101]

[0102] In the above formula, Indicates satellite At any moment frequency band Assigned to beam Indicator variables, Indicates satellite upper beam Transmission channel gain This indicates the bandwidth of each frequency band. This represents the power spectral density of thermal noise, a fixed value. According to Shannon's theorem, the achievable transmission rate of a signal depends on the frequency band allocated to the beam and the signal-to-noise ratio. Therefore, according to the above formula, the satellite... upper beam Average achievable data rate Represented as:

[0103]

[0104] In the formula, This indicates the total number of frequency bands into which the system spectrum is divided.

[0105] Once the beam scheduling mode is determined, according to the First Come First Served (FCFS) protocol, data packets with longer waiting times will be transmitted first. Therefore, the throughput of the entire network is expressed as:

[0106]

[0107] In the formula, express Time Satellite upper beam network throughput, Indicates the transmission time interval. This indicates the fixed size of each data packet.

[0108] Therefore, we can obtain the algorithm evaluation metric (average latency). With network throughput The mathematical expression of ). As can be seen from formula (6) and formula (3), the average waiting time and network throughput are heavily dependent on three factors: beam scheduling mode, frequency and power allocation.

[0109] This technical solution treats each low-Earth orbit satellite as an intelligent agent and employs multi-agent deep reinforcement learning technology to determine the optimal beam scheduling, frequency, and power allocation scheme with the optimization indicators of minimizing average latency and maximizing network throughput. Considering the complexity of this optimization problem and its high demands on satellite hardware during actual implementation, this technical solution breaks down the optimization problem into three steps. Step 1: Satellite-to-Ground Beam Scheduling Scheme. Step 1 first fixes the beam frequency band and power. After receiving service request signaling from ground users, the satellite uses DDQN technology to determine the optimal beam scheduling scheme in real time and stores the generated beam scheduling scheme in the onboard buffer. Step 2: After determining the optimal beam scheduling scheme in Step 1, DDPG technology is used to determine the optimal beam frequency band and power allocation scheme in real time, and the determined frequency band and power allocation scheme is stored in the satellite buffer. Step 3: When the satellite establishes a link with the ground gateway station, the scheme stored in the onboard buffer is sent to the ground data center via the ground gateway station, and the data center updates the satellite model parameters accordingly.

[0110] 2. Resource allocation algorithm based on deep reinforcement learning

[0111] This technical solution proposes a partitioned approach based on a deep reinforcement learning framework, namely, the framework includes two decision computation regions, such as... Figure 4 As shown. In the proposed solution framework, the final decision includes two types: discrete actions and continuous actions. To achieve joint optimization, this technical solution adopts a resource allocation algorithm based on joint DDQN and DDPG. Based on the characteristics of DDQN and DDPG, DDQN is used to learn the optimal beam scheduling strategy, and DDPG is used to learn beam frequency band and power allocation. In this framework, each low-Earth orbit satellite acts as an agent, and each agent contains two decision regions. Each satellite stores the resource allocation scheme in its onboard buffer and distributes the stored content to the ground data center during link establishment with the ground gateway station. The data center updates the model parameters of each satellite based on the distributed data.

[0112] In this technical solution, after each satellite receives the user's service request signaling, it generates a decision scheme through the DDQN and DDPG models. Therefore, the decision at each moment is independent of the decision at the previous moment. Thus, the decision-making process of this technical solution can be viewed as a Markov decision process, with its state space, action space, and reward space functions as shown below:

[0113] 1) State Space The state space represents the set of all possible states and defines the boundary of an agent's perception of the environment. Each state needs to contain all relevant information required to make a decision; the selection of this information directly affects the agent's decision quality and learning efficiency. At any given moment, each agent observes a state from the environment. Represented as .in, , , They represent Satellite beam allocation scheme, satellite frequency band allocation scheme, and satellite power allocation scheme. and They represent The latency and network throughput at each moment are determined under the conditions of beam, frequency band, and power allocation scheme. At each moment, after the satellite allocation scheme is generated, the state space is... It is stored in the onboard cache for subsequent model training and for updating model parameters with the ground data center.

[0114] 2) Action Space The action space defines the set of all possible actions an agent can take in each state. The design of the action space directly determines the boundaries of the agent's behavioral capabilities and is a key element connecting state transitions and reward acquisition. The action space at any given time... It is a resource allocation strategy based on observed environmental state information, and it is represented as: .

[0115] 3) Reward Space The purpose of the reward space is to evaluate the performance of a given action. Always in the zone After the action is performed, the environment state changes. And return the corresponding reward. The reward function depends not only on the objective function but also on the corresponding constraints. A larger reward is obtained if the current action reduces system task overhead while satisfying all constraints. Therefore, the following definition is provided. Instant rewards for moments Therefore, since the optimization objectives of this technical solution are to minimize average latency and maximize network throughput, and the optimization goals of these two metrics are inconsistent, this technical solution will allocate a reward space. Defined as a weighted average of the two indicators, it is expressed as:

[0116]

[0117] In the above formula, express The overall weighted value at each moment, Indicates the trade-off coefficient. express Time Satellite Storage Cell Average data wait time This indicates the number of beams equipped on the satellite. express Time Satellite beam The corresponding network throughput.

[0118] (2.1) Beam scheduling scheme based on DDQN technology

[0119] Since beam scheduling involves determining the connectivity between different cells and different satellites, this decision-making process is discrete. DDQN (Target Q-Network) performs well in handling discrete decisions, therefore, it is adopted for beam scheduling. DDQN is a reinforcement learning algorithm based on an evaluation model, primarily composed of a target network and a target Q-network. Both networks have identical structures. The target Q-network improves the algorithm's stability. In action space decision-making, assuming fixed frequency bands and power, DDQN is used only to... Making decisions, that is At the beginning of each decision-making process, the agent will use a greedy algorithm and a state space to... To calculate the action:

[0120]

[0121] In the above formula, Indicates a given observation state With action Obtain the output value of the target Q-network. These are the Q-network parameters.

[0122] (2.2) Frequency band and power allocation scheme based on DDPG technology

[0123] After determining the beam scheduling scheme in (2.1), this technical solution uses DDPG to allocate beam frequency bands and power. DDPG is a policy-dependent reinforcement learning algorithm based on an actor-critic framework, combining the advantages of deterministic policy gradient algorithms and DQN algorithms. In the DDPG algorithm, the actor network is used to generate the policy, and the critic network is used to evaluate the reward of the state-action pair. After obtaining the observed state, the agent generates beam frequency band and power decisions through the DDPG decision region. In addition, the actor network is used to calculate the numerical magnitude of the frequency band and power allocation, using... It means that among them Let represent the actor network parameters. Since the DDPG learning policy is deterministic, random noise is introduced to balance exploration and exploitation. Therefore, the agent's frequency band and power allocation decisions are expressed as:

[0124]

[0125] In the above formula, This represents white noise that follows a Gaussian distribution.

[0126] (2.3) Algorithm model parameter update

[0127] As can be seen from (2.1) and (2.2), to update the algorithm model, the main focus should be on the parameters. With parameters Updates are performed to enable the Q-network in DDQN and the actor network in DDPG to make better decisions based on the actual system environment. Therefore, this technical solution utilizes a ground data center to update the satellite model parameters, and employs gradient descent to update the model parameters based on the resource allocation scheme table collected from the satellite.

[0128] 3. Specific implementation details

[0129] The system flowchart of the low-Earth orbit satellite communication resource allocation method is as follows: Figure 5As shown, ground users send service request data to the ground station. The ground station uploads the user request data to the low-Earth orbit (LEO) satellite communication resource allocation system via the uplink. The satellite, based on the principles of minimum latency and maximum network throughput, generates an optimal beam, frequency, and power allocation scheme using a resource allocation algorithm based on a multi-agent deep reinforcement learning model. The LEO satellite schedules idle beams and allocates optimal communication resources (frequency and power) to the ground user according to the optimal beam resource allocation scheme. The ground station collects the beam resource schemes allocated by the LEO satellite to the ground user in real time and updates the model parameters based on the collected data. The updated model parameters are then uploaded to the LEO satellite via the uplink to update the algorithm model.

[0130] It should be noted that the low-Earth orbit satellite communication resource allocation method provided in this disclosure has the following characteristics:

[0131] a) Architecture innovation and global optimization: This disclosure proposes a low-orbit satellite communication resource allocation system architecture based on beam hopping technology. By decomposing the overall resource allocation optimization problem into three sub-problems—beam scheduling, frequency band allocation, and power allocation—it achieves coordinated optimization of network communication latency and throughput, effectively alleviating the contradiction between resource overload in hot spots and resource idleness in cold spots.

[0132] b) Dynamic adaptation and intelligent decision-making: In response to the spatiotemporal unevenness and dynamic changes in the traffic demand of ground users, this disclosure adopts multi-agent deep reinforcement learning technology, enabling each satellite to make autonomous decisions as an independent intelligent agent and generate the optimal resource allocation scheme in real time, which significantly improves the system's adaptability to dynamic environments.

[0133] c) Precise solution in stages: To address the discreteness of the beam scheduling problem, this disclosure uses DDQN technology to model it as a Markov decision process, effectively avoiding the "bootstrapping" problem of evaluation values ​​in traditional Q networks and improving the accuracy and stability of the beam scheduling scheme. To address the continuity of the frequency band and power allocation problem, DDPG technology combined with the actor-critic framework is adopted, which reduces the variance of policy updates while adapting to continuous optimization problems, further ensuring the accuracy of resource allocation schemes and the stability of network operation.

[0134] d) Space-Ground Collaboration and Lightweight Design: Considering the computation and storage limitations of the onboard payload of low-Earth orbit satellites, this disclosure achieves algorithm optimization for space-ground collaboration by allocating storage resources through an onboard cache and uploading data to a ground data center for model parameter updates during the establishment of the link between the satellite and the gateway station. This reduces the complexity of the onboard algorithm while continuously improving the overall performance of the network.

[0135] The low-Earth orbit (LEO) satellite communication resource allocation method provided in this disclosure receives service request signaling from ground users and obtains the satellite status information at the current moment. Utilizing a multi-agent deep reinforcement learning model with dual decision-making regions, it dynamically generates beam scheduling and frequency band and power allocation schemes for the next moment. This achieves real-time perception and accurate response to uneven user distribution and time-varying traffic characteristics, effectively improving the dynamic adaptability and overall operational efficiency of LEO satellite communication system resource allocation. It allows limited on-board resources to be more efficiently allocated to high-demand areas, significantly improving network throughput and user service quality. This solves the problem that existing LEO satellite communication resource allocation methods are unable to effectively address the dynamic challenges brought about by uneven user distribution and time-varying traffic.

[0136] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0137] Figure 6 This is a block diagram of a low-Earth orbit satellite communication resource allocation device provided in an embodiment of the present disclosure.

[0138] Reference Figure 6 This disclosure provides a low-Earth orbit (LEO) satellite communication resource allocation apparatus for executing the aforementioned LEO satellite communication resource allocation method. The apparatus includes:

[0139] Service request signaling receiving module 11 is used to receive service request signaling sent by ground users;

[0140] The satellite status information acquisition module 12 is connected to the service request signaling receiving module 11 and is used to acquire the satellite status information at the current time according to the service request signaling.

[0141] The resource allocation scheme generation module 13, connected to the satellite state information acquisition module 12, is used to input the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step. The multi-agent deep reinforcement learning model includes a first decision region and a second decision region. The first decision region is used to determine the beam scheduling scheme for the next time step based on the satellite state information. The second decision region is used to determine the frequency band and power allocation scheme for the next time step based on the beam scheduling scheme for the next time step. The resource allocation scheme includes both the beam scheduling scheme and the frequency band and power allocation scheme for the next time step.

[0142] Optionally, the device further includes:

[0143] The decision modeling module is used to model the decision process of the multi-agent deep reinforcement learning model as a Markov decision process, wherein the Markov decision process is defined by a state space, an action space, and a reward space.

[0144] The state space includes the satellite state information, the action space is a set of resource allocation schemes for the next moment, and the reward space includes a comprehensive weighted value calculated based on the average data waiting latency and network throughput obtained after executing the resource allocation scheme for the next moment.

[0145] Optionally, the comprehensive weighted value of the reward space is calculated using the following formula:

[0146]

[0147] In the formula, express The overall weighted value at each moment, Indicates the trade-off coefficient. express Time Satellite Storage Cell Average data wait time This indicates the number of beams equipped on the satellite. express Time Satellite beam The corresponding network throughput.

[0148] Optionally, the resource allocation scheme generation module 13 includes:

[0149] The beam scheduling scheme determination unit is used to input the satellite state information into the first decision region of the multi-agent deep reinforcement learning model and determine the beam scheduling scheme for the next moment through the dual deep Q network (DDQN) model.

[0150] The frequency band and power allocation scheme determination unit is used to input the satellite state information and the beam scheduling scheme for the next moment into the second decision region of the multi-agent deep reinforcement learning model, and determine the frequency band and power allocation scheme for the next moment through the deep deterministic policy gradient (DDPG) model.

[0151] The scheme combination unit is used to combine the beam scheduling scheme of the next time moment and the frequency band and power allocation scheme of the next time moment to obtain the resource allocation scheme;

[0152] The satellite status information includes the current beam scheduling scheme, frequency band allocation scheme, power allocation scheme, and the corresponding average data latency and network throughput.

[0153] Optionally, the beam scheduling scheme determination unit specifically includes:

[0154] The action selection unit is used to select one as a candidate beam scheduling scheme from discrete beam scheduling actions according to the ε-greedy strategy through the target Q network of the DDQN model.

[0155] The Q-value evaluation unit is used to evaluate the Q-value of the candidate beam scheduling scheme through the target network of the DDQN model, and determine the beam scheduling scheme at the next moment.

[0156] Optionally, the frequency band and power allocation scheme determination unit specifically includes:

[0157] The initial allocation strategy generation unit is used to generate an initial allocation strategy for frequency bands and power based on the satellite state information and the beam scheduling scheme at the next moment through the actor network in the DDPG model, and to introduce white noise that follows a Gaussian distribution into the initial allocation strategy to balance the exploration and utilization of the strategy.

[0158] The initial allocation strategy adjustment unit is used to input the satellite state information, the beam scheduling scheme for the next time moment, and the initial allocation strategy into the critic network of the DDPG model. The critic network performs reward evaluation on the state and action pairs corresponding to the initial allocation strategy, generates an evaluation value, and adjusts the initial allocation strategy according to the evaluation value to obtain the frequency band and power allocation scheme for the next time moment.

[0159] Optionally, the device further includes:

[0160] The resource allocation scheme storage module is used to store the resource allocation scheme for the next moment into the onboard buffer to form a satellite data table;

[0161] The resource allocation scheme uploading module is used to send the satellite data table stored in the onboard buffer to the ground data center during the establishment of a link with the gateway station, so that the ground data center can update the parameters of the multi-agent deep reinforcement learning model based on the satellite data table;

[0162] The model parameter receiving module is used to receive and load the model parameters updated by the ground data center.

[0163] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0164] Reference Figure 7 This disclosure provides an electronic device comprising: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs executable by the at least one processor 701, the one or more computer programs being executed by the at least one processor 701 to enable the at least one processor 701 to execute the aforementioned low-Earth orbit satellite communication resource allocation method.

[0165] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned low-Earth orbit satellite communication resource allocation method. The computer-readable storage medium may be volatile or non-volatile.

[0166] In summary, the low-Earth orbit (LEO) satellite communication resource allocation method, apparatus, device, and medium provided in this disclosure receive service request signaling from ground users and obtain satellite status information at the current moment. Utilizing a multi-agent deep reinforcement learning model with dual decision-making regions, it dynamically generates beam scheduling and frequency band and power allocation schemes for the next moment. This achieves real-time perception and accurate response to uneven user distribution and time-varying traffic characteristics, effectively improving the dynamic adaptability and overall operational efficiency of LEO satellite communication system resource allocation. It allows limited on-board resources to be more efficiently allocated to high-demand areas, significantly improving network throughput and user service quality. This solves the problem that existing LEO satellite communication resource allocation methods struggle to effectively address the dynamic challenges posed by uneven user distribution and time-varying traffic.

[0167] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0168] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0169] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0170] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0171] This disclosure has disclosed exemplary embodiments, and although specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for allocating low-Earth orbit satellite communication resources, characterized in that, The method includes: Receive service request signaling sent by ground users; Based on the service request signaling, obtain the satellite status information at the current moment; The satellite state information is input into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step. The multi-agent deep reinforcement learning model includes a first decision region and a second decision region. The first decision region is used to determine the beam scheduling scheme for the next time step based on the satellite state information. The second decision region is used to determine the frequency band and power allocation scheme for the next time step based on the beam scheduling scheme for the next time step. The resource allocation scheme includes the beam scheduling scheme and the frequency band and power allocation scheme for the next time step.

2. The method according to claim 1, characterized in that, Before inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step, the method further includes: The decision-making process of the multi-agent deep reinforcement learning model is modeled as a Markov decision process, which is defined by a state space, an action space, and a reward space. The state space includes the satellite state information, the action space is a set of resource allocation schemes for the next moment, and the reward space includes a comprehensive weighted value calculated based on the average data waiting latency and network throughput obtained after executing the resource allocation scheme for the next moment.

3. The method according to claim 2, characterized in that, The comprehensive weighted value of the reward space is calculated using the following formula: In the formula, express The overall weighted value at each moment, Indicates the trade-off coefficient. express Time Satellite Storage Cell Average data wait time This indicates the number of beams equipped on the satellite. express Time Satellite beam The corresponding network throughput.

4. The method according to claim 1, characterized in that, The step of inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step specifically includes: The satellite state information is input into the first decision region of the multi-agent deep reinforcement learning model, and the beam scheduling scheme for the next moment is determined by the dual deep Q network (DDQN) model. The satellite status information and the beam scheduling scheme for the next moment are input into the second decision region of the multi-agent deep reinforcement learning model, and the frequency band and power allocation scheme for the next moment is determined by the deep deterministic policy gradient (DDPG) model. The resource allocation scheme is obtained by combining the beam scheduling scheme for the next time step and the frequency band and power allocation scheme for the next time step. The satellite status information includes the current beam scheduling scheme, frequency band allocation scheme, power allocation scheme, and the corresponding average data latency and network throughput.

5. The method according to claim 4, characterized in that, The determination of the beam scheduling scheme for the next time step using the Dual Deep Q Network (DDQN) model specifically includes: Using the target Q-network of the DDQN model, an ε-greedy strategy is used to select one of the discrete beam scheduling actions as a candidate beam scheduling scheme. The Q-values ​​of the candidate beam scheduling schemes are evaluated using the target network of the DDQN model to determine the beam scheduling scheme for the next time step.

6. The method according to claim 4, characterized in that, The determination of the frequency band and power allocation scheme for the next time step using the Deep Deterministic Policy Gradient (DDPG) model specifically includes: Using the actor network in the DDPG model, an initial allocation strategy for frequency bands and power is generated based on the satellite state information and the beam scheduling scheme at the next moment. White noise following a Gaussian distribution is introduced into the initial allocation strategy to balance the exploration and utilization of the strategy. The satellite state information, the beam scheduling scheme for the next time step, and the initial allocation strategy are input into the critic network of the DDPG model. The critic network evaluates the state and action pairs corresponding to the initial allocation strategy to generate an evaluation value. The initial allocation strategy is then adjusted based on the evaluation value to obtain the frequency band and power allocation scheme for the next time step.

7. The method according to claim 1, characterized in that, After inputting the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step, the method further includes: The resource allocation scheme for the next time moment is stored in the onboard buffer to form a satellite data table; During the establishment of a link with the gateway station, the satellite data table stored in the onboard buffer is sent to the ground data center so that the ground data center can update the parameters of the multi-agent deep reinforcement learning model based on the satellite data table; Receive and load the updated model parameters from the ground data center.

8. A low-orbit satellite communication resource allocation device, characterized in that, The device includes: The service request signaling receiving module is used to receive service request signaling sent by ground users; The satellite status information acquisition module is connected to the service request signaling receiving module and is used to acquire the satellite status information at the current time according to the service request signaling. A resource allocation scheme generation module, connected to the satellite state information acquisition module, is used to input the satellite state information into a pre-deployed multi-agent deep reinforcement learning model to generate a resource allocation scheme for the next time step. The multi-agent deep reinforcement learning model includes a first decision region and a second decision region. The first decision region is used to determine the beam scheduling scheme for the next time step based on the satellite state information. The second decision region is used to determine the frequency band and power allocation scheme for the next time step based on the beam scheduling scheme for the next time step. The resource allocation scheme includes both the beam scheduling scheme and the frequency band and power allocation scheme for the next time step.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the low-orbit satellite communication resource allocation method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the low-orbit satellite communication resource allocation method as described in any one of claims 1-7.

Images