Comprehensive ground-satellite network dynamic resource allocation method based on EDT

Abstract

The invention discloses an EDT-based comprehensive ground-satellite network dynamic resource allocation method, which belongs to the technical field of communication networks and comprises the following steps: collecting real-time state information of a comprehensive ground-satellite network; modeling a resource allocation problem through a Markov decision process; the EDT is initialized, and decision optimization is carried out based on historical tracks of user association and power control; generating an association decision between the user and a ground base station or a low-orbit satellite, and outputting a power control strategy to maximize the energy efficiency of the system; executing a resource allocation operation; and continuously monitoring the network state, updating the Markov decision model in real time according to user demand changes or environmental factors, and dynamically adjusting a decision path through EDT. According to the method, by introducing a track splicing and dynamic historical length adjusting mechanism, the dynamic nature and isomerism of the integrated ground-satellite network are effectively dealt with, and meanwhile, the energy efficiency and the user experience are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of communication networks, and in particular to an EDT-based integrated ground-satellite network dynamic resource allocation method. Background Art

[0002] With the rapid growth of demand for global network connectivity, the application of integrated ground-satellite networks has become increasingly widespread. By combining the wide coverage capabilities of low-Earth orbit satellites with the high capacity and low latency of ground base stations, such networks have shown great application potential in rural areas, disaster recovery, and maritime communications. However, due to the dynamic and heterogeneous nature of the network environment, existing resource allocation methods have shown significant limitations in coping with rapidly changing user needs, channel conditions, and network status.

[0003] Traditional resource allocation methods usually rely on static or heuristic strategies, which often fail to find the global optimal solution in complex network environments. For example, static methods cannot adapt to the rapid changes in user needs and channel conditions in a timely manner, resulting in inefficient resource utilization. Although heuristic methods can provide certain optimization effects in certain specific scenarios, they usually lack a global perspective, are prone to falling into local optimal solutions, and have difficulty handling changing network environments.

[0004] In recent years, multi-agent deep reinforcement learning (DRL) methods have been gradually introduced into the resource allocation of integrated ground-satellite networks. These methods use the self-learning ability of agents to perform resource allocation and cache design in complex network environments. However, DRL methods have the problem of insufficient adaptability: the model often needs to be retrained in new scenarios, which takes a long time to train and consumes a lot of computing resources. In addition, these methods have limited generalization capabilities and perform poorly when dealing with multi-step predictions, making it difficult to achieve optimal resource allocation.

[0005] To overcome these challenges, the Transformer model was introduced into network resource allocation due to its outstanding performance in processing long sequence data. However, the existing Transformer model has limited adaptability in dynamic environments and has high computational complexity. Summary of the invention

[0006] The purpose of the present invention is to provide an EDT-based integrated ground-satellite network dynamic resource allocation method, which effectively copes with the dynamics and heterogeneity of the integrated ground-satellite network by introducing trajectory splicing and dynamic history length adjustment mechanism, solves the shortcomings of resource allocation in the prior art in a dynamic network environment, and improves the energy efficiency and user experience of the system.

[0007] To achieve the above object, the present invention provides an EDT-based integrated ground-satellite network dynamic resource allocation method, the steps comprising:

[0008] S1, collect status information from low earth orbit satellites and ground base stations in each time slot;

[0009] S2, model the resource allocation problem through the Markov decision process, define the state space, action space and energy efficiency maximization objective function of the integrated ground-satellite network system, and obtain the historical trajectory set;

[0010] S3, initialize EDT and optimize decision based on historical traces of user association and power control;

[0011] S4, generate association decisions between users and ground base stations or low-orbit satellites through EDT, and output power control strategies to maximize the energy efficiency of the integrated ground-satellite network system;

[0012] S5, transmitting the optimized user association and power control decision to the ground base station and the low-orbit satellite to perform resource allocation operations;

[0013] S6. Continuously monitor the status of the integrated ground-satellite network system, update the Markov decision model in real time based on changes in user demand or environmental factors, and dynamically adjust the decision path through EDT.

[0014] Preferably, the status information in step S1 includes channel status between the user equipment and the ground base station, the low-orbit satellite, network topology, user location, environmental conditions and user mobility pattern.

[0015] Preferably, the state space, action space and energy efficiency maximization objective function of the integrated ground-satellite network system defined in step S2 include:

[0016] The state space of the integrated ground-satellite network system is defined. The state space consists of the energy efficiency state at each time slot t, and its formula is:

[0017] S t ={SEE1(t), SEE2(t),...,SEE M (t)};

[0018]

[0019] In the formula, M represents the user set, u represents the random value in the user set, S represents the state space, and EE u (t) represents the energy efficiency of the u-th user in time slot t;

[0020] The action space of the integrated ground-satellite network system is defined. The action space includes the user association and power control decision at each time slot t, and its formula is:

[0021] At ={A1(t), A2(t)};

[0022] A1(t)={a1(t), a2(t),...,a M (t)};

[0023] A2(t)={p1(t), p2(t),...,p M (t)};

[0024] Where A1 represents the user association action, and A2 represents the power control action;

[0025]

[0026]

[0027] Where N is the number of base stations in the integrated ground-satellite network, L is the number of low-orbit satellites in the integrated ground-satellite network, and a bu (t) represents the user association variable between the u-th user and the b-th base station in time slot t, b su (t) represents the user association change between the u-th user and the s-th satellite in time slot t, P su (t) represents the transmission power from the sth satellite to the uth user in time slot t, p bu (t) represents the transmission power from the bth base station to the uth user in time slot t, a u (t) represents the total user association variable of the u-th user in time slot t, p u (t) represents the total transmission power of the u-th user in time slot t;

[0028] Rewards are defined. The reward function R defines the total energy efficiency of all users at the current time slot t. The energy efficiency of the integrated ground-satellite network system is maximized through user association and power control. The formula is:

[0029]

[0030] In the formula, R t Indicates reward.

[0031] Preferably, in step S3, EDT is initialized, and decision optimization is performed based on historical trajectories of user association and power control through trajectory splicing and dynamic history length adjustment.

[0032] Preferably, in step S4, generating an association decision between a user and a ground base station or a low-orbit satellite through EDT, and outputting a power control strategy to maximize the energy efficiency of the integrated ground-satellite network system includes:

[0033] S41, calculating the signal to interference and noise ratio through the channel state information between the ground base station and the user equipment, and between the low-orbit satellite and the user equipment;

[0034] S42. Based on the obtained signal-to-noise ratio data, EDT performs decision optimization based on user association and power control historical data, and dynamically adjusts the transmission power to maximize the energy efficiency of the integrated ground-satellite network system.

[0035] Preferably, the formula for calculating the signal-to-dryness ratio in step S41 is:

[0036]

[0037]

[0038] Where, SINR bu (t) represents the signal-to-noise-interference ratio of the u-th user connected to the b-th base station in time slot t, SINR su (t) represents the signal-to-noise-interference ratio of the u-th user connected to the s-th satellite in time slot t, h bu (t) represents the channel state information between the u-th user and the b-th base station in time slot t, g su (t) represents the channel state information between the u-th user and the s-th satellite in time slot t, I B (t) represents the interference generated by the base station user in time slot t, I SO (t) represents the interference generated by other satellite users in time slot t, N0 represents the additive white Gaussian noise power, I BO (t) represents the interference caused by other base station users in time slot t, I S (t) represents the interference caused by the satellite user in time slot t.

[0039] Preferably, the comprehensive ground-satellite network system energy efficiency formula in step S42 is:

[0040]

[0041] Therefore, the present invention adopts the above-mentioned EDT-based integrated ground-satellite network dynamic resource allocation method, which has the following beneficial effects:

[0042] (1) Through EDT’s trajectory splicing and dynamic history length adjustment, it achieves rapid adaptation and efficient convergence in dynamic and heterogeneous network environments;

[0043] (2) While optimizing user association and power control, the energy efficiency of the integrated ground-satellite network system is significantly improved.

[0044] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0046] Figure 2 A comparison chart of energy efficiency optimization experimental results of the present invention and other algorithms in different scenarios in an embodiment of the present invention;

[0047] Figure 3 This is a comparison chart of the optimized convergence speed of the present invention and the baseline algorithm in different scenarios in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0049] Example

[0050] Refer to Figure 1 The present invention provides a method for dynamic resource allocation of integrated ground-satellite networks based on EDT (Elastic Decision Transformer), the steps comprising:

[0051] S1. Collect status information from low earth orbit satellites and ground base stations in each time slot. The status information includes channel status between user equipment and ground base stations, low earth orbit satellites, network topology, user location, environmental conditions, and user mobility patterns.

[0052] S2. Model the resource allocation problem through the Markov decision process, define the state space, action space and energy efficiency maximization objective function of the integrated ground-satellite network system, and obtain the historical trajectory set.

[0053] Specifically, the state space, action space and energy efficiency maximization objective function of the integrated ground-satellite network system are defined as follows:

[0054] The state space of the integrated ground-satellite network system is defined. The state space consists of the energy efficiency state at each time slot t, and its formula is:

[0055] S t ={SEE1(t), SEE2(t),...,SEE M (t)};

[0056]

[0057] In the formula, M represents the user set, u represents the random value in the user set, S represents the state space, and EE u (t) represents the energy efficiency of the u-th user in time slot t;

[0058] The action space of the integrated ground-satellite network system is defined. The action space includes the user association and power control decision at each time slot t, and its formula is:

[0059] A t ={A1(t), A2(t)};

[0060] A1(t)={a1(t), a2(t),...,a M (t)};

[0061] A2(t)={p1(t), p2(t),...,p M (t)};

[0062] Where A1 represents the user association action, and A2 represents the power control action;

[0063]

[0064]

[0065] Where N is the number of base stations in the integrated ground-satellite network, L is the number of low-orbit satellites in the integrated ground-satellite network, and a bu (t) represents the user association variable between the u-th user and the b-th base station in time slot t, b su (t) represents the user association change between the u-th user and the s-th satellite in time slot t, P su (t) represents the transmission power from the sth satellite to the uth user in time slot t, p bu (t) represents the transmission power from the bth base station to the uth user in time slot t, a u (t) represents the total user association variable of the u-th user in time slot t, p u (t) represents the total transmission power of the u-th user in time slot t;

[0066] Rewards are defined. The reward function R defines the total energy efficiency of all users at the current time slot t. The energy efficiency of the integrated ground-satellite network system is maximized through user association and power control. The formula is:

[0067]

[0068] In the formula, R t Indicates reward.

[0069] S3: Initialize EDT and optimize the decision based on the historical trajectory of user association and power control. The optimization process includes trajectory splicing and dynamic history length adjustment.

[0070] S4. Generate association decisions between users and ground base stations or low-orbit satellites through EDT, and output power control strategies to maximize the energy efficiency of the integrated ground-satellite network system.

[0071] The specific steps include:

[0072] S41. Calculate the signal to interference plus noise ratio based on the channel state information between the ground base station and the user equipment, and between the low-orbit satellite and the user equipment.

[0073] The formula for calculating the dryness ratio is:

[0074]

[0075]

[0076] Where, SINR bu (t) represents the signal-to-noise-interference ratio of the u-th user connected to the b-th base station in time slot t, SINR su (t) represents the signal-to-noise-interference ratio of the u-th user connected to the s-th satellite in time slot t, h bu (t) represents the channel state information between the u-th user and the b-th base station in time slot t, g su (t) represents the channel state information between the u-th user and the s-th satellite in time slot t, I B (t) represents the interference generated by the base station user in time slot t, I SO (t) represents the interference generated by other satellite users in time slot t, N0 represents the additive white Gaussian noise power, I BO (t) represents the interference caused by other base station users in time slot t, I S (t) represents the interference caused by the satellite user in time slot t.

[0077] S42. According to the obtained signal-to-noise ratio data, the EDT is used to optimize the decision based on user association and power control history data, and the transmission power is dynamically adjusted to maximize the energy efficiency of the integrated ground-satellite network system. The energy efficiency formula of the integrated ground-satellite network system is:

[0078]

[0079] S5. Transmit the optimized user association and power control decision to the ground base station and the low-orbit satellite to perform resource allocation operations.

[0080] S6. Continuously monitor the status of the integrated ground-satellite network system, update the Markov decision model in real time based on changes in user demand or environmental factors, and dynamically adjust the decision path through EDT.

[0081] In order to verify the resource allocation effect of the method of the present invention in the integrated ground-satellite network, a comparative experiment was conducted between the method of the present invention and other models, and the specific contents are as follows.

[0082] Model setup

[0083] The Decision Transformer (DT) method, the Proximal Policy Optimization (PPO) algorithm, and the random strategy were selected as baseline models. The DT method reframes the reinforcement learning problem as a sequence modeling problem and uses the Transformer architecture to directly generate the optimal behavior sequence. As a commonly used traditional reinforcement learning algorithm, the PPO algorithm has achieved a good balance between stability and efficiency and is suitable for general network resource allocation scenarios. However, its disadvantage is that it has poor adaptability and often requires retraining in new scenarios. The random strategy is used as a control benchmark to evaluate the relative effectiveness of other algorithms.

[0084] Evaluation Metrics

[0085] System energy efficiency is selected as the only evaluation metric to measure the network performance after optimizing user association and power control.

[0086] Dataset Description

[0087] Several typical scenario datasets of simulated integrated ground-satellite networks are used. These datasets include communication records of different users with low-orbit satellites and ground base stations in different time periods, as well as information such as user location, channel status, network topology, and traffic demand. The specific parameters are: the network consists of 2 low-earth orbit satellites and 4 ground base stations, with a total of 12 users. Each satellite and each base station can serve up to 2 users at the same time. The satellite uses S-band carrier frequency with a maximum transmission power of 30dBm. The base station channel adopts the Rayleigh fading model, and the maximum transmission power is set to 23dBm. In the hyperparameter setting of the EDT model, the optimizer uses the Adam optimizer, the activation function uses the ReLU function, the batch size is set to 10, the learning rate is 0.0001, and the discount factor is set to 0.95. The total number of experimental rounds is 1000, and the centralized controller completes 100 steps in each round of the experiment. The model training uses the data of 12 users and generalization tests are performed in new scenarios.

[0088] Results Analysis

[0089] Figure 2 and Figure 3 The experimental results in different network scenarios are presented, and the tests are conducted for the cases where the number of users is 18 (scenario 1) and 24 (scenario 2). The experimental results show that the EDT-based method has significant improvements in the energy efficiency and convergence speed of the integrated ground-satellite network system compared with the comparison model. In the scenario with 18 users, the energy efficiency of the EDT model is about 1.2 times higher than that of the comparison model, and in the scenario with 24 users, the energy efficiency improvement is also very significant. In addition, the convergence speed of EDT is significantly better than other algorithms. The experimental results show that its convergence speed is 2-3 times faster than the PPO algorithm.

[0090] Therefore, the present invention adopts the above-mentioned EDT-based integrated ground-satellite network dynamic resource allocation method, which performs resource allocation using the EDT model. It has strong adaptability and generalization, can effectively optimize resource allocation under different network load conditions, and improves energy efficiency and convergence speed.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for dynamic resource allocation of integrated ground-satellite network based on EDT, characterized in that the steps include: S1, collect status information from low earth orbit satellites and ground base stations in each time slot; S2, model the resource allocation problem through the Markov decision process, define the state space, action space and energy efficiency maximization objective function of the integrated ground-satellite network system, and obtain the historical trajectory set; S3, initialize EDT and optimize decision based on historical traces of user association and power control; S4, generate association decisions between users and ground base stations or low-orbit satellites through EDT, and output power control strategies to maximize the energy efficiency of the integrated ground-satellite network system; S5, transmitting the optimized user association and power control decision to the ground base station and the low-orbit satellite to perform resource allocation operations; S6. Continuously monitor the status of the integrated ground-satellite network system, update the Markov decision model in real time based on changes in user demand or environmental factors, and dynamically adjust the decision path through EDT.

2. The method for dynamic resource allocation of an integrated ground-satellite network based on EDT according to claim 1, characterized in that: The status information in step S1 includes the channel status between the user equipment and the ground base station, the low-orbit satellite, the network topology, the user location, the environmental conditions and the user mobility pattern.

3. The EDT-based integrated ground-satellite network dynamic resource allocation method according to claim 2, characterized in that: The state space, action space and energy efficiency maximization objective function of the integrated ground-satellite network system defined in step S2 include: The state space of the integrated ground-satellite network system is defined. The state space consists of the energy efficiency state at each time slot t, and its formula is: S t ={SEE1(t),SEE2(t),...,SEE M (t)}; In the formula, M represents the user set, u represents the random value in the user set, S represents the state space, and EE u (t) represents the energy efficiency of the u-th user in time slot t; The action space of the integrated ground-satellite network system is defined. The action space includes the user association and power control decision at each time slot t, and its formula is: A t ={A1(t),A2(t)}; A1(t)={a1(t),a2(t),...,a M (t)}; Stop 2(t) = {p1(t), p2(t),..., p M (t)}; In the formula, stop 1 represents the user association action, and stop 2 represents the power control action; Where N is the number of base stations in the integrated ground-satellite network, L is the number of low-orbit satellites in the integrated ground-satellite network, and a bu (t) represents the user association variable between the u-th user and the b-th base station in time slot t, b su (t) represents the user association change between the u-th user and the s-th satellite in time slot t, P su (t) represents the transmission power from the sth satellite to the uth user in time slot t, p bu (t) represents the transmission power from the bth base station to the uth user in time slot t, a u (t) represents the total user association variable of the u-th user in time slot t, p u (t) represents the total transmission power of the u-th user in time slot t; Rewards are defined. The reward function R defines the total energy efficiency of all users at the current time slot t. The energy efficiency of the integrated ground-satellite network system is maximized through user association and power control. The formula is: In the formula, R t Indicates reward.

4. The method for dynamic resource allocation of an integrated ground-satellite network based on EDT according to claim 3, characterized in that: In step S3, EDT is initialized, and decision optimization is performed based on historical trajectories of user association and power control through trajectory splicing and dynamic history length adjustment.

5. The EDT-based integrated ground-satellite network dynamic resource allocation method according to claim 4, characterized in that: In step S4, the association decision between the user and the ground base station or the low-orbit satellite is generated by EDT, and the power control strategy is output to maximize the energy efficiency of the integrated ground-satellite network system, including: S41, calculating the signal to interference and noise ratio through the channel state information between the ground base station and the user equipment, and between the low-orbit satellite and the user equipment; S42. Based on the obtained signal-to-noise ratio data, EDT performs decision optimization based on user association and power control historical data, and dynamically adjusts the transmission power to maximize the energy efficiency of the integrated ground-satellite network system.

6. The EDT-based integrated ground-satellite network dynamic resource allocation method according to claim 5, characterized in that: The formula for calculating the drying ratio in step S41 is: Where, SINR bu (t) represents the signal-to-noise-interference ratio of the u-th user connected to the b-th base station in time slot t, SINR su (t) represents the signal-to-noise-interference ratio of the u-th user connected to the s-th satellite in time slot t, h bu (t) represents the channel state information between the u-th user and the b-th base station in time slot t, g su (t) represents the channel state information between the u-th user and the s-th satellite in time slot t, I B (t) represents the interference generated by the base station user in time slot t, I SO (t) represents the interference generated by other satellite users in time slot t, N0 represents the additive white Gaussian noise power, I BO (t) represents the interference caused by other base station users in time slot t, I S (t) represents the interference caused by the satellite user in time slot t.

7. The EDT-based integrated ground-satellite network dynamic resource allocation method according to claim 6, characterized in that: The energy efficiency formula of the integrated ground-satellite network system in step S42 is:

Images