A satellite communication method and system based on rate division multiple access

Abstract

The application relates to the technical field of satellite communication, and particularly discloses a satellite communication method and system based on rate division multiple access, wherein the method comprises the following steps: constructing a satellite communication system model by using a geostationary satellite, a high-altitude communication platform and a ground user terminal; taking the maximization of the total reachable rate of the system as a target, taking the power budget of a transmitter and the minimum rate of energy distribution as constraint conditions, and determining an optimization problem according to the satellite communication system model; modeling the optimization problem into a Markov decision process through a state space, an action space and a reward function; solving the Markov decision process by using an improved DQN algorithm; and determining an optimized rate distribution coefficient based on the solution of the Markov decision process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of satellite communication, and particularly relates to a satellite communication method and system based on rate-splitting multiple access. BACKGROUND

[0002] The rate-splitting multiple access (RSMA) technology is introduced into the satellite communication system, mainly to solve the limitations of traditional multiple access technologies in terms of spectrum efficiency, dynamic resource allocation, and interference management. The following discusses the disadvantages of traditional technologies and the advantages of RSMA.

[0003] (1) Disadvantages of traditional multiple access technologies

[0004] With the demand for access of large-scale ground equipment, satellite communication systems, as a seamless infrastructure, have become the core of the next generation of wireless communication systems. Early satellite communication systems mainly used frequency division multiple access (FDMA), time division multiple access (TDMA), code division multiple access (CDMA), and other multiple access technologies. Subsequently, orthogonal frequency division multiple access (OFDMA) and non-orthogonal multiple access (NOMA) were introduced. However, these technologies have the following problems in the satellite scenario:

[0005] 1) The first problem is low spectrum utilization. For example, frequency division multiple access (FDMA) and time division multiple access (TDMA) allocate spectrum resources by dividing fixed frequency bands or time slots, and users exclusively occupy the allocated spectrum resource units, resulting in low spectrum utilization and difficulty in adapting to the dynamic changes in user demand in satellite communication. Code division multiple access (CDMA) access technology relies on spreading codes to distinguish users, and when the number of users increases, the code interference (MAI) significantly increases, requiring complex power control, which is difficult to control in a satellite-ground transmission time delay cross environment.

[0006] 2) The second problem is the lack of interference management capability. In a satellite communication system based on NOMA, NOMA superimposes user signals in the power domain and uses serial interference cancellation (SIC) to improve system capacity, but in a satellite channel, user channel differences are small (such as a stationary orbit satellite covering a wide area, with limited user path loss differences), resulting in poor power reuse effect and prominent SIC decoding error accumulation problems.

[0007] 3) Other problems include that traditional OFDMA technology is sensitive to frequency offset and phase noise, requiring further development of complex equalization algorithms, which greatly increases system overhead. Moreover, traditional technologies usually rely on static or semi-static resource allocation, which is difficult to adapt to scenarios where user distribution is uneven and business demand is bursty (such as intermittent transmission between Internet of Things devices), resulting in resource waste or congestion.

[0008] Therefore, based on the aforementioned multiple access technologies, how to utilize Rate Division Multiple Access (RSMA) technology to enhance the transmission performance of satellite communication systems is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0009] To achieve the objective of this invention, this application provides a satellite communication method based on rate segmentation multiple access, comprising:

[0010] Step S1: Construct a satellite communication system model using geostationary orbit satellites, high-altitude communication platforms, and ground user terminals;

[0011] Step S2: With the goal of maximizing the total achievable rate of the system, and with the transmitter power budget and the minimum rate of energy allocation as constraints, determine the optimization problem based on the satellite communication system model;

[0012] Step S3: Model the optimization problem as a Markov decision process using the state space, action space, and reward function;

[0013] Step S4: Solve the Markov decision process using the improved DQN algorithm;

[0014] Step S5: Determine the optimized rate allocation coefficients based on the solution of the Markov decision process.

[0015] In some specific embodiments, step S1 includes:

[0016] Step S11: The satellite transmits a data-carrying beam to a nearby space node via a coherent laser link;

[0017] Step S12: The high-altitude communication platform uses an adaptive optics receiving array to perform wavefront correction and energy coupling on the incident beam, converting the modulated optical signal into a radio frequency signal;

[0018] Step S13: The ground user terminal receives signals from the high-altitude communication platform via a direct transmission link using single-layer RSMA technology.

[0019] In some specific embodiments, step S2 includes: the optimization problem is determined according to the following formula:

[0020] ;

[0021] In the formula, Represents the total reachable rate of the system. Indicates the first i The received signal rate of the public information stream for each ground user. w represents the rate segmentation coefficient. k This represents the beamforming matrix for active transmission from a high-altitude communication platform. Indicates the first i The received signal rate of the private information stream of a ground user, c i Indicates the first i The data rate at which a ground user receives public information. and This indicates that the public information can be successfully decoded by every ground user. In Indicates the first Achievable rate requirements for individual ground users Indicates the first i Energy allocation requirements for individual ground users Greater than J i , Represented as the maximum transmittable power budget at the transmitter. Less than , This indicates the rate segmentation coefficient limit.

[0022] In some specific embodiments, step S3 includes:

[0023] The state space is determined based on the information set of all current ground user terminals, the action vectors selected by all current ground user terminals, and the instantaneous rewards of all current ground user terminals during the local training phase.

[0024] The action space is determined based on the active transmit beam vector, power division factor, and achievable rate of common information flow;

[0025] The reward function includes: an immediate reward under unconstrained conditions and a penalty under constrained conditions.

[0026] In some specific embodiments, step S4 includes:

[0027] Step S41: Define each direct link from the high-altitude communication platform to the ground user terminal as an autonomous decision-making unit, so that each autonomous decision-making unit has local state observation capability, independent action space and personalized reward function;

[0028] Step S42: Each autonomous decision-making unit shares a set of deep neural network parameters, learns a collaborative strategy through a global experience pool, and broadcasts the deep neural network parameters of the trained strategy to each agent;

[0029] Step S43: Based on the time-varying characteristics of the link between the ground user terminal and the high-altitude communication platform, establish a dual-mode parameter update mechanism and use mini-batch gradient descent to update the parameters of the deep neural network.

[0030] Step S44: Optimize the transmit beamforming matrix and rate splitting coefficients of the high-altitude communication platform using the improved DQN algorithm;

[0031] Step S45: Solve the Markov decision process based on the deep neural network parameters, the beamforming matrix transmitted by the high-altitude communication platform, and the rate segmentation coefficients.

[0032] To achieve the same inventive objective, this application also provides a satellite communication system based on rate division multiple access, comprising:

[0033] Model building module: used to build satellite communication system models using geostationary orbit satellites, high-altitude communication platforms, and ground user terminals;

[0034] Optimization problem determination module: used to determine the optimization problem based on the satellite communication system model with the objective of maximizing the total achievable rate of the system and the constraints of the transmitter power budget and the minimum rate of energy allocation;

[0035] Decision process establishment module: used to model the optimization problem as a Markov decision process through state space, action space and reward function;

[0036] Process Solver Module: Used to solve the Markov decision process using the improved DQN algorithm;

[0037] Scheme verification module: used to determine the optimized rate allocation coefficients based on the solution of the Markov decision process.

[0038] In some specific embodiments, the model building module is used to perform the following steps:

[0039] Step S11: The satellite transmits a data-carrying beam to a nearby space node via a coherent laser link;

[0040] Step S12: The high-altitude communication platform uses an adaptive optics receiving array to perform wavefront correction and energy coupling on the incident beam, converting the modulated optical signal into a radio frequency signal;

[0041] Step S13: The ground user terminal receives signals from the high-altitude communication platform via a direct transmission link using single-layer RSMA technology.

[0042] In some specific embodiments, in the model building module, the optimization problem is determined according to the following formula:

[0043] ;

[0044] In the formula, Represents the total reachable rate of the system. Indicates the first i The received signal rate of the public information stream for each ground user. denotes the rate splitting coefficient, w k denotes the beamforming matrix of the active transmission of the HAP, denotes the received signal rate of the private information stream of the i i denotes the data rate of the common information received by the i and denotes that the common information can be successfully decoded by each ground user, denotes the achievable rate requirement of the denotes the energy allocation requirement of the i is greater than J i , denotes that the maximum transmissible power budget at the transmitter is less than , denotes the rate splitting coefficient constraint.

[0045] In some embodiments, the decision process establishing module is configured to perform the following steps:

[0046] determining the state space according to the information set of all current ground user terminals, the action vectors selected by all current ground user terminals, and the immediate reward of the local training stage of all current ground user terminals;

[0047] determining the action space according to the active transmission beam vector, the power splitting coefficient, and the achievable rate of the common information stream;

[0048] the reward function comprises an immediate reward item in an unconstrained case and a penalty item in a case of meeting a constraint condition.

[0049] In some embodiments, the process solving module is configured to perform the following steps:

[0050] Step S41: defining each direct link from the high-altitude communication platform to the ground user terminal as an autonomous decision unit, and enabling each autonomous decision unit to have a local state observation capability, an independent action space, and an individualized reward function;

[0051] Step S42: each autonomous decision unit shares a set of deep neural network parameters, learns a cooperative strategy through a global experience pool, and broadcasts the deep neural network parameters of the trained strategy to each intelligent agent;

[0052] ​​​​​Step S43: According to the time-varying characteristics of the link between the ground user terminal and the high-altitude communication platform, a double-mode parameter updating mechanism is established, and the deep neural network parameters are updated using small batch gradient descent;

[0053] Step S44: The high-altitude communication platform transmit beamforming matrix and rate segmentation coefficient are optimized using the improved DQN algorithm.

[0054] Step S45: The Markov decision process is solved according to the deep neural network parameters, the high-altitude communication platform transmit beamforming matrix, and the rate segmentation coefficient.

[0055] The beneficial effects of the above technical solutions are:

[0056] The method provided by the application breaks through the idealized assumption of the channel model of the traditional beamforming algorithm, and establishes an end-to-end mapping relationship from the original channel state information to the beam weight value through the DQN algorithm. Unlike the fixed rate allocation mode in the traditional system, a rate segmentation multiple access strategy multi-agent cooperation mechanism is introduced, and each agent corresponds to the rate segmentation decision of a specific user or service flow. By defining a joint state space including user QoS requirements, service type priority, and network load state, the DQN agent can learn how to dynamically adjust the rate allocation coefficient between public information flow and private information flow under different network conditions.

[0057] The application also proposes to model the communication process of satellites, high-altitude communication platforms, and ground terminals as a heterogeneous Markov decision process, wherein the beamforming decision of the high-altitude platform and the power control strategy of the ground terminal influence each other. By establishing a cross-air-layer joint reward function, the system can evaluate the overall network performance of the air-ground collaborative strategy, such as total throughput and energy efficiency. In actual deployment, this collaborative framework can effectively cope with channel quality fluctuations caused by user mobility, terrain shielding, etc. Through intelligent cooperation of air-ground nodes, the robustness of network performance is improved. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0059] Figure 1 A flowchart of a satellite communication method based on rate segmentation multiple access provided by an embodiment of the application is shown.

[0060] Figure 2 A structural diagram of a satellite communication system based on rate segmentation multiple access provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application.

[0062] Examples of the embodiments are shown in the drawings, in which the same or similar symbols represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0063] Embodiment one

[0064] One embodiment of the present application provides a satellite communication method based on rate division multiple access, referring to Figure 1 as shown, comprising:

[0065] Step S1: constructing a satellite communication system model by using a geostationary satellite, a high-altitude communication platform and a ground user terminal;

[0066] Step S2: determining an optimization problem according to the satellite communication system model, with the maximum total reachable rate of the system as the target and the power budget of the transmitter and the minimum rate of energy allocation as the constraint condition;

[0067] Step S3: modeling the optimization problem as a Markov decision process through a state space, an action space and a reward function;

[0068] Step S4: solving the Markov decision process by using an improved DQN algorithm.

[0069] In one specific embodiment of the present application, step S1 comprises:

[0070] Step S11: the satellite transmits a light beam carrying data to a near space node through a coherent laser link;

[0071] Step S12: the high-altitude communication platform uses an adaptive optical receiving array to correct the wavefront and couple the energy of the incident light beam, and converts the modulated light signal into a radio frequency signal;

[0072] Step S13: the ground user terminal receives the high-altitude communication platform signal through a direct transmission link by using a single-layer RSMA technology.

[0073] The system model proposed in the present application is composed of a geostationary satellite, a high-altitude communication platform and a ground user terminal. Its signal transmission mechanism is divided into two core stages according to the characteristics of the transmission medium: in the initial transmission stage, the satellite transmits a light beam carrying data to a near space node through a coherent laser link The high-altitude communication platform uses an adaptive optical receiving array to correct the wavefront and couple the energy of the incident light beam, and converts the modulated light signal into a radio frequency signal through an optoelectronic conversion module. The conversion process follows the cascade transformation principle of optical-electric-radio frequency, and the output signal can be represented as:

[0074] ;

[0075] where, is the satellite transmit power, is the conversion coefficient from optical signal to radio frequency signal, is the environmental noise, is the scalar channel fading coefficient from satellite to high-altitude communication platform. In addition, in order to solve the potential error caused by atmospheric turbulence, an Equal Gain Combining (EGC) scheme is adopted at the high-altitude communication platform.

[0076] In the second transmission stage, the high-altitude communication platform applies single-layer RSMA technology, and the ground user receives the high-altitude communication platform signal through a direct transmission link.

[0077] Assuming that the associated signal sent by the high-altitude communication platform to the th ground user is divided into two components, i.e. the public information part and the private information part . Then, all the public information parts are combined into and encoded into a common information stream . The private part of the th ground user terminal is separately encoded into a private information stream , satisfying . That is, the high-altitude communication platform sends a common information stream and private information streams, a total of symbols, to the ground user equipment through a direct link. Then the transmission signal of the high-altitude communication platform can be represented as:

[0078] ;

[0079] where, is the active transmit beam vector of the th information stream . Therefore, the total transmit power at the high-altitude communication platform is represented as:

[0080] ;

[0081] where, and respectively, are the efficiency parameter of the power amplifier and the power consumption of each circuit module. The time interval in the system is analyzed in a discrete form, and the received signal of the first ground user in the first time slot is expressed as

[0082]

[0083] is the channel gain from the high-altitude communication platform to the first ground user.

[0084] In addition, the power splitting ratio of the first ground user is defined as This ratio is crucial for how the received signal is divided in terms of energy allocation and signal transmission. Notably, the common information stream will be broadcast to all users as an effective energy allocation transmission carrier and will not interfere with the decoding of the private information stream. Specifically, part of the signal power is used for user identification, and the remaining part is used for energy allocation.

[0085] Therefore, in the RSMA-based satellite communication system, ground users employ a hierarchical decoding strategy to achieve information separation. Specifically, each user terminal first treats the private data streams of other users as additive interference terms and preferentially decodes the common information stream. This processing mechanism is based on the difference in dual information attributes: the common information stream, as the basic service data shared by the entire network, needs to be accessible to all users; the private information stream carries personalized services and only has payload value for the target user. After completing the decoding of the common information stream, the receiver reconstructs the common signal component through the serial interference cancellation (SIC) technology and projects it out in the received signal space. At this point, the channel observation model is transformed into an enhanced observation containing only the target user's private information, and a second decoding operation is performed. Therefore, the received signal rates of the first ground user's common information stream and private information stream are expressed as

[0086]

[0087] and

[0088]

[0089] where is the group channel coefficient of the high-altitude communication platform-ground user terminal, and to ensure that the common information stream can be successfully decoded at all ground users, the rate of the common information stream can be set to Then, let be the received signal rate of the first ground user's private information stream.​​​​ The data rate at which a ground user receives public information must meet the following requirements:

[0090] ;

[0091] Then, the total reachable rate of the system under consideration can be expressed as:

[0092] ;

[0093] In the formula, This represents the system ergodicity and rate of the FSO link. Here, it's assumed that the FSO link has a higher system ergodicity and rate than the RF link. This means that the signal primarily experiences attenuation in free space during transmission, with less obstruction or reflection from other objects. Therefore, compared to RF transmission links, FSO links have lower signal loss and can more effectively maintain signal strength and quality. Secondly, RF transmission links are limited by the physical characteristics of RF signals. RF signals are affected by various factors during transmission, such as multipath effects, signal attenuation, and interference. These factors all reduce the modulation efficiency and spectral efficiency of the RF transmission link.

[0094] In one specific embodiment of the invention, the core optimization problem is to maximize the total reachable rate of the system, aiming to ensure that the system achieves optimal energy utilization while satisfying various operational constraints. These constraints mainly include the transmitter's power budget and the minimum rate requirement for energy allocation. To solve this optimization problem, a sophisticated algorithm is designed to optimize several key parameters. These parameters include the beamforming matrix of the active transmission of the high-altitude communication platform. and rate segmentation coefficient The optimization problem is determined according to the following formula:

[0095] ;

[0096] In the formula, Represents the total reachable rate of the system. Indicates the first i The received signal rate of the public information stream for each ground user. w represents the rate segmentation coefficient. k This represents the beamforming matrix for active transmission from a high-altitude communication platform. Indicates the first i The received signal rate of the private information stream of a ground user, c i Indicates the first i The data rate at which a ground user receives public information. and This indicates that the public information can be successfully decoded by every ground user. In represents the achievable rate requirement of the th ground user, represents the energy allocation requirement of the i th ground user is greater than J i , represents the maximum transmissible power budget at the transmitter is less than , represents the rate partitioning coefficient limit.

[0097] In one specific embodiment of the present application, step S3 comprises:

[0098] determining the state space according to the information set of all ground user terminals at present, the action vector selected by all ground user terminals at present and the immediate reward of the local training stage of all ground user terminals at present;

[0099] determining the action space according to the active transmit beam vector, the power partitioning coefficient and the achievable rate of the common information flow;

[0100] the reward function comprises: an immediate reward item in the unconstrained case and a penalty item in the case of meeting the constraint condition.

[0101] Specifically, the above optimization problem can be modeled as a Markov Decision Process (MDP), which specifically includes:

[0102] 1) State space: according to the design principle of the state space, it should include as much environment information related to the optimization problem as possible. In the system model considered, the state space should be composed of three parts. The information set of all ground user terminals at present , the selected action vector and the immediate reward of the local training stage . The information set of all users at present is defined as:

[0103] (4-24)

[0104] In the formula, and respectively represent the received signal rate of the common information flow and the private information flow of the th ground user at the last time. The selected action vector represents the action selected from the last time by using the improved DQN algorithm, and the immediate reward is calculated from the current state and the current action. This index can directly reflect the ability of the Agent to solve the optimization problem under different conditions. Therefore, the state space of the training node is represented as:

[0105] ;

[0106] 2) Action space: The action space in the local training phase mainly includes the active transmit beam vector , the power splitting coefficient , and the achievable rate of the common information stream , etc. When solving the problem by using the improved DQN algorithm, the input information needs to be input into the Q network, so in the neural network input or output, the complex form needs to be split into the real part and the imaginary part as input respectively. Therefore, the active transmit beam vector at the high-altitude communication platform is split as follows:

[0107] ;

[0108] where and represent the modulus of the corresponding transmit power and the unit length representing the beam direction respectively. By such decomposition, the system can control the transmit signal strength and the propagation direction respectively. Here, the hyperbolic tangent function is used to smooth the change of the power output within a predetermined range. This is because the hyperbolic tangent function is a nonlinear function, and its output value is between -1 and 1, and at the same time, the function also has the characteristics of smooth transition, that is, when the input value increases from negative infinity to positive infinity, the output value smoothly transitions from -1 to 1, avoiding the occurrence of sudden changes. Therefore, it is expressed as:

[0109] ;

[0110] Pmax is the maximum transmittable power at the transmitter, and the main purpose of this formula is to limit the output of the transmit power within the specified range after satisfying the corresponding constraints. And represents the output response of the activation function to the selected transmit power.

[0111] In the satellite communication system applying RSMA, the common information stream will be transmitted as the main energy carrier. Under this scheme, the decoding process of the common information stream will not be disturbed by the continuous serial interference cancellation technology for receiving the private information stream. Therefore, the common information stream beam forming direction can adopt the maximum ratio combining (MRT) scheme, which can maximize the power of the signal at the receiving end, thereby improving the reliability of transmission. For the private information stream, considering the need to reduce the interference between users, the zero-forcing transmission scheme can be used. The zero-forcing transmission scheme precisely controls the direction of the transmitted signal to ensure that the signal interference in the direction of non-target users is forced to zero, thereby effectively reducing the interference between users and improving the quality of the signal and the overall performance of the system. Therefore, the active transmit power direction can be specifically expressed as:

[0112] (4-28)

[0113] where, is denoted as the combined channel coefficient from the high altitude platform - ground user terminal. Define , and is denoted as the beam direction under the combined channel, where . On the other hand, the power splitting coefficient and the achievable rate of the common information stream can also be calculated by using the hyperbolic tangent function, which is denoted as:

[0114] ;

[0115] and:

[0116] ;

[0117] where, and are the achievable rate of the common information stream and the power splitting coefficient under the activation function, respectively. Therefore, the action space is denoted as:

[0118] ;

[0119] 3) Reward function: In the process of solving the optimization problem using the DQN algorithm, the reward function is the key content that needs to be considered in the design process. Not only does it need to consider the optimization objective, but it must also take into account the constraints in the optimization problem P0. Generally, the reward function is mainly composed of two parts: the first part is the immediate reward item reflecting the unconstrained case, and the other part is the penalty item to ensure that the constraint condition is met. In order to balance the relationship between the two, a demand-aware reward function needs to be carefully designed. In the subsection, the reward function can be denoted as:

[0120] ;

[0121] where, , and respectively represent the action penalty results under the conditions of not meeting the transmit power requirement, the common information decoding requirement, and the quality of service requirement in the corresponding optimization problem. They can be modeled as:

[0122] ;

[0123] ;

[0124] .

[0125] In constrained optimization problems, the design of the penalty function plays a key role in the convergence of the algorithm. By constructing a boundary barrier function, a dynamic weight penalty is imposed on the iteration points that exceed the feasible region, establishing an effective constraint coupling mechanism, making the optimization trajectory naturally tend to the feasible solution space in the iteration process. This strategy significantly improves the global search ability of the algorithm under strict constraint conditions, ensuring that physically feasible Pareto optimal solutions can still be stably obtained near complex constraint boundaries. In the satellite communication system optimization framework based on reinforcement learning, the construction of the reward function embodies the multi-objective collaborative design concept. Only when all the quality of service (QoS) constraints and energy consumption limits are met, the system gives a non-zero reward value. This threshold feedback mechanism strengthens the priority of constraint satisfaction. For effective actions that improve system achievable rate, the reward function adopts the principle of diminishing marginal returns, giving differential positive incentives according to the magnitude of rate improvement, guiding the agent to form an adaptive balance between energy utilization efficiency and communication performance.

[0126] The design of this reward and punishment mechanism follows the principle of double-track regulation: the penalty term forms a repulsive force field through the nonlinear mapping of constraint violation, preventing policy exploration from entering infeasible regions; the reward term constructs an attractive force field based on the performance improvement, driving the agent to evolve towards the Pareto frontier. This two-way regulation mechanism ensures the robustness of the system while achieving the organic unity of constraint satisfaction and performance optimization, providing a theoretically feasible optimization framework for autonomous decision-making of complex communication systems.

[0127] In one specific embodiment of the present application, step S4 comprises:

[0128] Step S41: defining each direct link from the high-altitude communication platform to the ground user terminal as an autonomous decision unit, so that each autonomous decision unit has local state observation capability, independent action space and individualized reward function;

[0129] Step S42: each autonomous decision unit shares a set of deep neural network parameters, learns a collaborative strategy through a global experience pool, and broadcasts the deep neural network parameters of the trained strategy to each agent;

[0130] Step S43: according to the time-varying characteristics of the link between the ground user terminal and the high-altitude communication platform, a double-mode parameter updating mechanism is established, and the deep neural network parameters are updated using small batch gradient descent;

[0131] Step S44: using the improved DQN algorithm to optimize the high-altitude communication platform transmit beamforming matrix and rate segmentation coefficient;

[0132] Step S45: solving the Markov decision process according to the deep neural network parameters, the high-altitude communication platform transmit beamforming matrix and the rate segmentation coefficient.

[0133] Benefiting from the function fitting ability of the deep neural network in DQN, the proposed DQN framework can handle problems with huge state space and action space. However, DQN may have unstable convergence speed because it only trains one neural network. Therefore, in order to accelerate the convergence effect of the algorithm and the stability of the DQN algorithm, considering the large-scale ground user access to satellite communication scenarios in the future, an improved DQN algorithm is proposed. The core idea is:

[0134] (1) First, define each high-altitude communication platform-ground user terminal direct link as an autonomous decision unit, and each autonomous decision unit needs to have local state observation capability (including link quality, transmission state, etc.), independent action space (power allocation decision) and individualized reward function (based on local QoS satisfaction);

[0135] (2) Then, all parameters use the hybrid paradigm of "centralized training-distributed execution". Mainly refers to the training phase: all agents share the same set of deep neural network parameters , learn the collaborative strategy through global experience pool. In particular, when the discount factor , the network degenerates into a first-order Markov decision process, which only needs to maintain a single Q-training network, significantly reducing the non-stationary problem in multi-agent systems. In the execution phase: the trained strategy parameters are broadcast to each agent, realizing fully distributed decision-making, and each node only relies on local observation to complete real-time resource scheduling;

[0136] (3) Finally, for the time-varying characteristics of the link between the ground user terminal and the high-altitude communication platform, a dual-mode parameter update mechanism is established, and the deep neural network parameters are fine-tuned using small batch gradient descent to trigger the exploration mode to enhance the robustness of the strategy.

[0137] The complete process of using the improved DQN algorithm to optimize the high-altitude communication platform transmit beamforming matrix and rate splitting coefficient can be divided into the following steps:

[0138] ① Input stage: initialize the experience pool, maximize the number of training rounds, and maximize the number of communication time slots ;

[0139] ② Training phase:

[0140]

[0141] ③ Output stage: optimized .

[0142] Specifically, for the proposed improved DQN optimization framework, the state space is represented by the joint representation of refined channel state vectors and environmental parameters, including instantaneous channel gain, interference power distribution, user mobility patterns, and network topology time-varying characteristics, etc. The state representation is formed by dynamically adjusting the tensor splicing. The network architecture adopts a deep residual design, the input layer dimension is adaptively adjusted according to the state variable size, the hidden layer neurons are expanded according to the exponential rule, and the complexity of single time step forward propagation is the product function of state variable square and network layer number. The global training complexity further considers the trajectory length and training period. The reward function design adopts a double-factor mechanism, the main item is based on the secrecy rate logarithmic measurement of signal-to-interference ratio, and the constraint item is punished by the exponential function for power out-of-bounds, and the two are dynamically balanced by the Lagrange multiplier method. When retraining, the historical strategy experience is retained through knowledge distillation, the gradual forgetting mechanism makes the historical experience influence decay over time, and a dual-mode mechanism is used to pre-train the hyperparameters to achieve rapid strategy adaptation in a new environment. Simulation experiments show that this mechanism can approach the performance level of traditional retraining with a small amount of parameter updates.

[0143] Step S5: determining the optimized rate allocation coefficient based on the solution of the Markov decision process.

[0144] At the level of dynamic beam optimization driven by reinforcement learning, the present application breaks through the idealized assumption of channel model in traditional beamforming algorithms, and establishes an end-to-end mapping relationship from the original channel state information to the beam weight through the DQN algorithm. The core of the algorithm is to design a hybrid network architecture that includes spatial feature extraction and decision fusion, in which the convolutional neural network is responsible for processing the spatial correlation of the multi-dimensional channel matrix, and the fully connected layer generates the beamforming vector according to the extracted features. This data-driven optimization method not only avoids the computational overhead caused by complex matrix operations, but also can adaptively capture non-ideal factors in the actual communication environment, such as atmospheric attenuation, multipath effect, etc., thereby realizing online evolution of beam patterns in high-altitude communication scenarios with dynamic topology.

[0145] Unlike the fixed rate allocation mode in traditional systems, the present application introduces a rate division multiple access strategy multi-agent cooperation mechanism, each agent corresponds to the rate division decision of a specific user or service flow. By defining a joint state space containing user QoS requirements, service type priority, and network load state, the DQN agent can learn how to dynamically adjust the rate allocation coefficient between public information flow and private information flow under different network conditions.

[0146] The construction of the satellite communication system cooperative optimization framework marks the leap of the patent from single technology optimization to system-level solution. The application proposes to model the communication process of satellites, high-altitude communication platforms and ground terminals as a heterogeneous Markov decision process, in which the beamforming decisions of high-altitude platforms and the power control strategies of ground terminals interact with each other. By establishing a joint reward function across the air layer, the system can evaluate the overall network performance of the air-ground cooperative strategy, such as total throughput, energy efficiency, etc. In actual deployment, this cooperative framework can effectively cope with the fluctuations in channel quality caused by user mobility, terrain obstruction, etc., and through the intelligent cooperation of air-ground nodes, the robustness of network performance is improved.

[0147] In view of the strong interference environment specific to high-altitude communication, the application innovatively proposes an anti-interference robustness enhancement design scheme. On the one hand, the generalization ability of DQN is used to integrate the interference avoidance mechanism into the beamforming optimization process, by adjusting parameters such as beam direction and null depth, the influence of interference signals on system performance is minimized while ensuring the communication quality of target users. In addition, for the problem of signal fading under multipath channel conditions, the application proposes an exploration-exploitation strategy based on double-parameter update, which intelligently searches for the optimal parameter variation in the beamforming parameter space, significantly enhancing the anti-interference robustness of the system.

[0148] Embodiment two

[0149] One embodiment of the application provides a satellite communication system based on rate division multiple access, referring to FIG. 1, which comprises: Figure 2

[0150] Model construction module 10: for constructing a satellite communication system model using stationary orbit satellites, high-altitude communication platforms and ground user terminals;

[0151] Optimization problem determination module 20: for determining an optimization problem according to the satellite communication system model, with the goal of maximizing the total achievable rate of the system and the constraint conditions of the power budget of the transmitter and the minimum rate of energy allocation;

[0152] Decision process establishment module 30: for modeling the optimization problem as a Markov decision process through state space, action space and reward function;

[0153] Process solving module 40: for solving the Markov decision process using an improved DQN algorithm;

[0154] Scheme verification module 50: for determining the optimized rate allocation coefficient based on the solution of the Markov decision process.

[0155] In one specific embodiment of the application, the model construction module 10 is used to perform the following steps: ​

[0156] Step S11: The satellite transmits a data-carrying beam to a nearby space node via a coherent laser link;

[0157] Step S12: The high-altitude communication platform uses an adaptive optics receiving array to perform wavefront correction and energy coupling on the incident beam, converting the modulated optical signal into a radio frequency signal;

[0158] Step S13: The ground user terminal receives signals from the high-altitude communication platform via a direct transmission link using single-layer RSMA technology.

[0159] In a specific embodiment of the present invention, in the model building module 20, the optimization problem is determined according to the following formula:

[0160] ;

[0161] In the formula, Represents the total reachable rate of the system. Indicates the first i The received signal rate of the public information stream for each ground user. w represents the rate segmentation coefficient. k This represents the beamforming matrix for active transmission from a high-altitude communication platform. Indicates the first i The received signal rate of the private information stream of a ground user, c i Indicates the first i The data rate at which a ground user receives public information. and This indicates that the public information can be successfully decoded by every ground user. In Indicates the first Achievable rate requirements for individual ground users Indicates the first i Energy allocation requirements for individual ground users Greater than J i , Represented as the maximum transmittable power budget at the transmitter. Less than , This indicates the rate segmentation coefficient limit.

[0162] In one specific embodiment of the present invention, the decision process establishment module 30 is used to perform the following steps:

[0163] The state space is determined based on the information set of all current ground user terminals, the action vectors selected by all current ground user terminals, and the instantaneous rewards of all current ground user terminals during the local training phase.

[0164] The action space is determined according to an active emission beam vector, a power division coefficient and a reachable rate of a common information flow;

[0165] The reward function comprises an immediate reward item in an unconstrained case and a penalty item in a case of meeting a constraint condition.

[0166] In one specific embodiment of the application, the process solving module 40 is configured to perform the following steps:

[0167] Step S41: defining each direct link from the high-altitude communication platform to the ground user terminal as an autonomous decision unit, and enabling each autonomous decision unit to have a local state observation capability, an independent action space and an individualized reward function;

[0168] Step S42: each autonomous decision unit shares a set of deep neural network parameters, learns a collaborative strategy through a global experience pool, and broadcasts the deep neural network parameters of the trained strategy to each intelligent agent;

[0169] Step S43: according to the time-varying characteristics of the link between the ground user terminal and the high-altitude communication platform, a double-mode parameter updating mechanism is established, and the deep neural network parameters are updated by using a small batch gradient descent;

[0170] Step S44: the high-altitude communication platform transmission beamforming matrix and the rate division coefficient are optimized by using an improved DQN algorithm;

[0171] Step S45: the Markov decision process is solved according to the deep neural network parameters, the high-altitude communication platform transmission beamforming matrix and the rate division coefficient.

[0172] The above is only a specific embodiment of the application, but the protection scope of the application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

[0173] The various embodiments in the specification are described in progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other. The embodiments of the present application are described with reference to flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a device that implements the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one flow or multiple flows and / or blocks Figure 1 one flow or multiple flows and / or blocks Figure 1 one flow or multiple flows and / or blocks Figure 1 one flow or multiple flows and / or blocks Figure 1The steps of a method, process, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal. In a further embodiment, the above described functionality of a software module can be provided within a general purpose computer, a special purpose computer, or a computer that includes a processor that can be a general purpose processor, a special purpose processor, or a combination of a general purpose processor and a special purpose processor.

[0174] The above detailed description of the method and device provided by the present application has been described in detail, and the principle and implementation mode of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed; in summary, the content of the description should not be understood as a limitation of the present application.

[0175] In the description of the present specification, the description of the terms “one embodiment”, “some embodiments”, “example”, “specific example”, “one specific embodiment” or “some examples” means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0176] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A satellite communication method based on rate segmentation multiple access, characterized in that, include: Step S1: Construct a satellite communication system model using geostationary orbit satellites, high-altitude communication platforms, and ground user terminals; Step S2: With the goal of maximizing the total achievable rate of the system, and with the transmitter power budget and the minimum rate of energy allocation as constraints, determine the optimization problem based on the satellite communication system model; Step S3: Model the optimization problem as a Markov decision process using the state space, action space, and reward function; Step S4: Solve the Markov decision process using the improved DQN algorithm; Step S5: Determine the optimized rate allocation coefficients based on the solution of the Markov decision process; Step S2 includes: The optimization problem is determined according to the following formula: In the formula, Represents the total reachable rate of the system. This indicates the received signal rate of the private information stream. w represents the rate segmentation coefficient. k This represents the beamforming matrix for active transmission from a high-altitude communication platform. Indicates the first i The received signal rate of the public information stream and the private information stream for each ground user, c i Indicates the first i The data rate at which a ground user receives public information. and This indicates that the public information can be successfully decoded by every ground user. In Indicates the first Minimum achievable rate requirement for each ground user Indicates the first i Energy allocation requirements for individual ground users Greater than J i , Represented as the maximum transmittable power budget at the transmitter. Less than , Indicates the rate segmentation coefficient limit; J i Indicates the first i Minimum energy allocation requirements for each ground user P max This represents the maximum transmittable power estimate at the transmitter. This indicates the achievable rate of the received signal for a public information flow; Step S4 includes: Step S41: Define each direct link from the high-altitude communication platform to the ground user terminal as an autonomous decision-making unit, so that each autonomous decision-making unit has local state observation capability, independent action space and personalized reward function; Step S42: Each autonomous decision-making unit shares a set of deep neural network parameters, learns a collaborative strategy through a global experience pool, and broadcasts the trained deep neural network parameters to each agent; Step S43: Based on the time-varying characteristics of the link between the ground user terminal and the high-altitude communication platform, establish a dual-mode parameter update mechanism and use mini-batch gradient descent to update the parameters of the deep neural network. Step S44: Optimize the transmit beamforming matrix and rate splitting coefficients of the high-altitude communication platform using the improved DQN algorithm; Step S45: Solve the Markov decision process based on the deep neural network parameters, the high-altitude communication platform's transmit beamforming matrix, and the rate segmentation coefficients; Step S3 includes: The state space is determined based on the information set of all current ground user terminals, the action vectors selected by all current ground user terminals, and the instantaneous rewards of all current ground user terminals during the local training phase. The action space is determined based on the active transmit beam vector, power division factor, and achievable rate of common information flow; The reward function includes: an immediate reward under unconstrained conditions and a penalty under constrained conditions.

2. The satellite communication method based on rate segmentation multiple access according to claim 1, characterized in that, Step S1 includes: Step S11: The satellite transmits a data-carrying beam to a nearby space node via a coherent laser link; Step S12: The high-altitude communication platform uses an adaptive optics receiving array to perform wavefront correction and energy coupling on the incident beam, converting the modulated optical signal into a radio frequency signal; Step S13: The ground user terminal receives signals from the high-altitude communication platform via a direct transmission link using single-layer RSMA technology.

3. A satellite communication system based on rate segmentation multiple access, characterized in that, include: Model building module: used to build satellite communication system models using geostationary orbit satellites, high-altitude communication platforms, and ground user terminals; Optimization problem determination module: used to determine the optimization problem based on the satellite communication system model with the objective of maximizing the total achievable rate of the system and the constraints of the transmitter power budget and the minimum rate of energy allocation; Decision process establishment module: used to model the optimization problem as a Markov decision process through state space, action space and reward function; Process Solver Module: Used to solve the Markov decision process using the improved DQN algorithm; Scheme verification module: used to determine the optimized rate allocation coefficients based on the solution of the Markov decision process; In the model building module, the optimization problem is determined according to the following formula: In the formula, Represents the total reachable rate of the system. This indicates the received signal rate of the private information stream. w represents the rate segmentation coefficient. k This represents the beamforming matrix for active transmission from a high-altitude communication platform. Indicates the first i The received signal rate of the public information stream and the private information stream for each ground user, c i Indicates the first i The data rate at which a ground user receives public information. and This indicates that the public information can be successfully decoded by every ground user. In Indicates the first i Minimum achievable rate requirement for each ground user Indicates the first i Energy allocation requirements for individual ground users Greater than J i , Represented as the maximum transmittable power budget at the transmitter. Less than , Indicates the rate segmentation coefficient limit; J i Indicates the first i Minimum energy allocation requirements for each ground user P max This represents the maximum transmittable power estimate at the transmitter. This indicates the achievable rate of the received signal for a public information flow; The process solver module is used to perform the following steps: Step S41: Define each direct link from the high-altitude communication platform to the ground user terminal as an autonomous decision-making unit, so that each autonomous decision-making unit has local state observation capability, independent action space and personalized reward function; Step S42: Each autonomous decision-making unit shares a set of deep neural network parameters, learns a collaborative strategy through a global experience pool, and broadcasts the deep neural network parameters of the trained strategy to each agent; Step S43: Based on the time-varying characteristics of the link between the ground user terminal and the high-altitude communication platform, establish a dual-mode parameter update mechanism and use mini-batch gradient descent to update the parameters of the deep neural network. Step S44: Optimize the transmit beamforming matrix and rate splitting coefficients of the high-altitude communication platform using the improved DQN algorithm; Step S45: Solve the Markov decision process based on the deep neural network parameters, the high-altitude communication platform's transmit beamforming matrix, and the rate segmentation coefficients; The decision-making process establishment module is used to perform the following steps: The state space is determined based on the information set of all current ground user terminals, the action vectors selected by all current ground user terminals, and the instantaneous rewards of all current ground user terminals during the local training phase. The action space is determined based on the active transmit beam vector, power division factor, and achievable rate of common information flow; The reward function includes: an immediate reward under unconstrained conditions and a penalty under constrained conditions.

4. The satellite communication system based on rate division multiple access according to claim 3, characterized in that, The model building module is used to perform the following steps: Step S11: The satellite transmits a data-carrying beam to a nearby space node via a coherent laser link; Step S12: The high-altitude communication platform uses an adaptive optics receiving array to perform wavefront correction and energy coupling on the incident beam, converting the modulated optical signal into a radio frequency signal; Step S13: The ground user terminal receives signals from the high-altitude communication platform via a direct transmission link using single-layer RSMA technology.

Images