A method for optimizing satellite-to-ground communication performance of FAS assisted by reinforcement learning

Abstract

This invention discloses a reinforcement learning-based method for optimizing the performance of FAS-assisted satellite-to-ground communication. In air-to-ground communication, when the direct path is blocked by obstacles, the signal is transmitted through a mixture of reflection from moving scatterers and the direct path. This invention describes the communication environment between the mobile transmitter and the ground user through geometric stochastic modeling, establishes a dynamic optimization framework based on FAS, couples satellite motion characteristics with adjustable FAS parameters to construct a three-dimensional channel model, designs an improved Memetic PPO reinforcement learning algorithm to dynamically optimize the FAS configuration set at the ground user in the discrete action space through a local search mechanism, and jointly optimizes channel capacity and modeling accuracy to generate the optimal strategy. This method reveals the nonlinear relationship between satellite motion and channel characteristics, significantly improves communication performance in complex obstruction environments, provides a new method for dynamic channel optimization of satellites, and has significant application value.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology, specifically relating to a reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization method. Background Technology

[0002] With the rapid development of sixth-generation (6G) mobile communication technology, fluid antenna systems (FAS), as a novel reconfigurable antenna technology, have attracted widespread attention due to their ability to dynamically adjust the antenna's position, shape, and radiation characteristics. By deploying multiple switchable ports within a limited space, FAS enables flexible utilization of spatial diversity gain, providing a new technical approach to improve the channel capacity and spectral efficiency of wireless communication systems.

[0003] Meanwhile, satellite-to-ground communication, as an important component of the integrated air-space-ground network, has broad application prospects in meteorological observation, emergency rescue, and logistics distribution. However, the high-speed mobility of satellites results in communication channels exhibiting significant time-varying and non-stationary characteristics. Coupled with factors such as building obstruction and multipath effects in urban environments, channel modeling and performance optimization face severe challenges.

[0004] Existing FAS optimization methods mainly have the following shortcomings:

[0005] (1) Channel model: Traditional FAS channel models mostly assume static scenarios and fail to fully consider the time-varying characteristics caused by satellite movement, making it difficult to accurately describe the dynamic changes of scattering clusters in the air-to-ground channel;

[0006] (2) In terms of optimization algorithms: FAS configuration parameters (such as antenna length and number of active ports) constitute a high-dimensional discrete action space. Traditional reinforcement learning algorithms (such as standard PPO) are inefficient in exploring the discrete space and are prone to getting trapped in local optima.

[0007] (3) Evaluation indicators: Existing studies focus on a single performance indicator (such as channel capacity), ignoring the modeling error between the FAS model and the benchmark ULA model, making it difficult to achieve a balance between performance and complexity.

[0008] Therefore, there is an urgent need for an intelligent optimization method that can simultaneously consider the dynamic characteristics of the channel and the complexity of FAS configuration in order to improve the performance of satellite-to-ground communication systems. Summary of the Invention

[0009] The purpose of this invention is to provide a reinforcement learning-based method for optimizing the performance of FAS-assisted satellite-to-ground communication, in order to solve the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a method for optimizing the performance of FAS-assisted satellite-to-ground communication based on reinforcement learning, comprising the following steps:

[0011] Step 1: Construct a reinforcement learning-based FAS-assisted satellite channel model and establish it in three-dimensional space. A Cartesian coordinate system is defined as a non-line-of-sight channel between the transmitting satellite and the ground user.

[0012] Step 2: When the signal emitted by the satellite reaches the ground user after being reflected by the scatterer, construct a functional expression for the transmission distance between the transmitter and the ground user and the scatterer;

[0013] Step 3: Construct the channel complex impulse response function expression for the transmission path of the signal reflected by the scatterer to the ground user, and the complex channel impulse response expression between the antenna and the FAS port;

[0014] Step 4: Construct the channel capacity and modeling error of the FAS-assisted satellite channel based on reinforcement learning, and derive the expressions for the channel capacity and modeling error;

[0015] Step 5: Define the state space, action space, and reward function in the FAS-assisted satellite channel based on reinforcement learning;

[0016] Step 6: Optimize the solution using the Memetic PPO algorithm in the FAS-assisted satellite channel based on reinforcement learning. This specifically includes:

[0017] Step 601: Initialize the policy network and value network parameters and And set an experience replay buffer;

[0018] Step 602: The agent follows the current policy. It interacts with the environment, collects trajectory data, and uses generalized advantage estimation to calculate the advantage function;

[0019] Step 603: Define the probability ratio to construct the pruning agent objective function, and update the policy network parameters by maximizing this objective function;

[0020] Step 604: Update the value network parameters by minimizing the mean square error of the value network. ;

[0021] Step 605: Output candidate actions in the policy network Then, a local search mechanism was introduced, defining... Centered on, with radius neighborhood ;

[0022] Evaluate the immediate rewards of all candidate actions within the neighborhood, and select the optimal action as the final action to be executed. The expression is:

[0023] ;

[0024] in, That is the optimal action obtained in the end. This represents the reward obtained by calculating the immediate reward function;

[0025] Step 606: Repeat steps 602 to 605 until convergence, and output the optimized FAS configuration parameters.

[0026] Preferably, the specific steps of step 3 are as follows:

[0027] The expression for the channel complex impulse response function of the transmission path from the scatterer to the ground user is constructed as follows:

[0028] ;

[0029] in, Indicates path delay. It is a time-varying path delay, caused by changes in propagation distance due to satellite motion. It is an absolute time variable, reflecting the time-varying characteristics of the channel. It is the impulse function. This indicates that in the proposed FAS channel model based on scattering clusters, the first... root transmitting antenna and the first The complex channel impulse response between FAS ports is expressed as follows:

[0030] ;

[0031] in, For wavelength, This represents a random phase, where it is assumed that each phase is independent and follows a certain order. Uniform distribution This indicates that after a signal is reflected and diffracted by a scatterer, its phase will undergo an unpredictable random shift, which is modeled as... Uniform distribution on For the satellite defined in step 201 root antenna to the first In the scattering cluster, the th Time-varying propagation distance of each scatterer Indicates the location index of the transmitting antenna. Indicates antenna The actual position offset relative to the array reference point Indicates the location index of the FAS port. This indicates the distance between adjacent ports in the FAS. Indicates port The actual position offset relative to the FAS reference point, and These are the time-varying departure angles of the emitted wave in the azimuth and vertical planes, respectively. and The angle of arrival of a wave is defined as its angle of arrival in the azimuth and vertical planes.

[0032] Preferably, the channel capacity expression in step 4 is:

[0033] ;

[0034] in, express An identity matrix of dimension 1 This represents the signal-to-noise ratio coefficient. This refers to the total number of antennas on the transmitting satellite. Represents the normalized channel matrix. This represents the conjugate transpose of the normalized channel matrix.

[0035] Preferably, the model error expression in step 4 is:

[0036] ;

[0037] in, and These represent the complex impulse response function of the FAS-assisted satellite channel model and the complex channel impulse response of the baseline ULA channel model, respectively. The number of ports activated for FAS.

[0038] Preferably, step 5 includes:

[0039] state space :

[0040] Includes FAS physical length Number of activated ports Port spacing state Belongs to the state space , represented as:

[0041] ;

[0042] in, The distance between adjacent ports of the FAS is calculated using the following formula:

[0043] ;

[0044] in For wavelength, This is the total number of FAS ports.

[0045] Action space :

[0046] Includes adjustment actions for the length of the FAS. and the action of adjusting the number of active ports. That is, action ,in It is a multidimensional discrete action space;

[0047] reward function :

[0048] Defined as a weighted combination of channel capacity and modeling error, its expression is:

[0049] ;

[0050] in, It is a weight value for channel capacity. It is the weight value of the model error of the FAS-assisted satellite channel.

[0051] Preferably, in step 602, the advantage function is calculated using generalized advantage estimation. The formula is:

[0052] ;

[0053] Among them, the time difference term ,in for Rewards at that time As a discount factor, For parameters The state-value function approximated by the neural network, Indicates from state To begin, follow the expected cumulative discount return that can be obtained by the current strategy. This represents the offset at a future time step. This represents the exponentially decaying weight, applied to the future... The timing difference error of the step, The generalized advantage estimation parameter determines the agent's emphasis on future rewards and directly affects how the reward is calculated; its value range is... When it is close to 1, the agent tends to consider long-term benefits, which is suitable for tasks that require long-term planning. When it is close to 0, the agent focuses more on immediate rewards, which helps to accelerate convergence, but may ignore long-term goals.

[0054] Preferably, in step 603, the expression for the pruning proxy objective function is: ;

[0055] in, The objective function representing the pruning surrogate is the core of the PPO algorithm. For the expectation operator, This indicates that for all time steps Calculate the average of the samples. Importance weight Limited to the range Inside, Represents the pruning factor; calculated by the ratio of the action probabilities of the new and old strategies. and restrict it to those by Within the specified range, it avoids policy collapse or drastic performance fluctuations due to excessive gradient updates, significantly reducing implementation complexity; Undefined probability ratio; ,in, This indicates the old strategy.

[0056] Preferably, the expression for the mean squared error loss function of the value network in step 604 is: ;

[0057] in, It is the loss function of the value network, which allows it to more accurately predict the value of a state. Accumulate rewards for discounts.

[0058] Preferably, in step 605, the neighborhood... The expression is:

[0059] ;in, As candidate actions, Indicates candidate actions With each dimension The absolute difference does not exceed .

[0060] The technical effects and advantages of this invention are as follows: This invention accurately describes the time-varying non-stationary characteristics in air-to-ground communication by constructing a FAS-assisted satellite channel model based on scattering clusters; it proposes a dual-objective optimization framework for channel capacity and modeling error, achieving a balance between communication performance and model accuracy; it designs the Memetic PPO algorithm, introducing a local search mechanism in the discrete action space, which significantly improves exploration efficiency and convergence stability; simulation results show that the method proposed in this invention can quickly converge to a better FAS configuration, effectively controlling modeling error while improving channel capacity, and providing an effective solution for intelligent optimization of FAS in dynamic air-to-ground communication scenarios. Attached Figure Description

[0061] Figure 1 : A schematic diagram of the channel model for FAS-assisted satellite channels based on reinforcement learning proposed in this invention;

[0062] Figure 2 A schematic diagram comparing channel capacity under different FAS configurations obtained by the model proposed in this invention;

[0063] Figure 3 : A schematic diagram comparing the modeling errors under different FAS configurations obtained by the model proposed in this invention;

[0064] Figure 4 A schematic diagram comparing the comprehensive scores obtained by the model proposed in this invention under different FAS configurations. Detailed Implementation

[0065] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0066] This invention proposes a reinforcement learning-based method for optimizing the performance of FAS-assisted satellite-to-ground communication, comprising the following steps:

[0067] Step 1: Construct a reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization method model, such as... Figure 1 As shown, establish in three-dimensional space A rectangular coordinate system is defined, where the line connecting the midpoint of the transmitter antenna array projection and the midpoint of the ground user antenna array is... Axis; defined as passing through the midpoint of the transmitter antenna array projection and perpendicular to it. The axis is Axis; defined as passing through the midpoint of the transmitter antenna array projection and perpendicular to it. The lines of the plane are Axis; in addition, the scattering objects of the transmission environment between the transmitter and the ground user are in a stationary state, while the transmitter satellite is in a moving state; there is no line-of-sight (LoS) channel between the transmitter and the ground user, only a non-line-of-sight (NLoS) channel.

[0068] Step 2: In the reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization method, when the signal emitted by the satellite reaches the ground user after being reflected by the scatterer, a functional expression for the transmission distance between the transmitter and the ground user and the scatterer is constructed. The specific steps are as follows:

[0069] Step 201: In the reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization method, when the signal emitted by the satellite reaches the ground user after being reflected by a scatterer, the scatterer is stationary, while the transmitting satellite is moving. The calculation of the satellite's... root Antenna to the indivual The first scattering cluster indivual The time-varying propagation distance of the scatterer is:

[0070] ;

[0071] in Indicates the first In the scattering cluster, the th The distance from each scatterer to the origin of the coordinate system This represents the L2 norm, used to calculate distances from vectors; It is the satellite number The time-varying transmission distance from the root antenna to the origin is expressed as:

[0072] ;

[0073] in, It is the satellite number The distance vector from the root antenna to the center of the antenna array. These are the time-varying position vectors of the satellite, and their specific expressions are as follows:

[0074] ;

[0075] ;

[0076] in This represents the distance between any two lines on the satellite's linear antenna. This refers to the satellite's azimuth and orientation angle. The satellite's pitch and orientation angle. The azimuth angle of the satellite's motion. The elevation angle of the satellite's motion. This refers to the satellite's velocity.

[0077] Step 202: In the reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization method, calculate the first... In the scattering cluster, the th The first scatterer to the ground user The transmission distance of each unit is:

[0078] ;

[0079] in, Indicating the first in the ground user fluid antenna system The position vector of each unit from the origin is expressed as follows:

[0080] ;

[0081] in Indicates the first The distance between each port and the center point of FAS and These are the azimuth and elevation angles of the FAS, respectively.

[0082] Step 3: Construct the channel complex impulse response function expression for the transmission path of the signal after reflection from the scatterer to the ground user:

[0083] ;

[0084] in, Indicates path delay. It is a time-varying path delay, caused by changes in propagation distance due to satellite motion. It is an absolute time variable, reflecting the time-varying characteristics of the channel. It is the impulse function. This indicates that in the proposed FAS channel model based on scattering clusters, the first... root transmitting antenna and the first The complex channel impulse response between FAS ports is expressed as follows:

[0085] ;

[0086] in, For wavelength, This represents a random phase, where it is assumed that each phase is independent and follows a certain order. Uniform distribution This indicates that after a signal is reflected and diffracted by a scatterer, its phase will undergo an unpredictable random shift, which is modeled as... Uniform distribution on Indicates the location index of the transmitting antenna. Indicates antenna The actual position offset relative to the array reference point Indicates the location index of the FAS port. This indicates the distance between adjacent ports in the FAS. Indicates port The actual position offset relative to the FAS reference point, and These are the time-varying departure angles of the emitted wave in the azimuth and vertical planes, respectively. and The angle of arrival of a wave is defined as its angle of arrival in the azimuth and vertical planes.

[0087] Step 4: Construct the channel capacity and modeling error of the FAS-assisted satellite channel based on reinforcement learning, and derive the expressions for the channel capacity and modeling error. The specific steps are as follows:

[0088] Step 401: In the reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization method, the channel capacity expression is:

[0089] ;

[0090] in, express An identity matrix of dimension 1 This represents the signal-to-noise ratio coefficient. This refers to the total number of antennas on the transmitting satellite. Represents the normalized channel matrix. This represents the conjugate transpose of the normalized channel matrix;

[0091] Step 402: In the reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization method, the model error expression is:

[0092] ;

[0093] in, and These represent the complex impulse response function of the FAS-assisted satellite channel model and the complex channel impulse response of the baseline ULA channel model, respectively. The number of ports activated for FAS.

[0094] Step 5: Define the state space, action space, and reward function in the reinforcement learning-based FAS-assisted satellite channel:

[0095] state space :

[0096] Includes FAS physical length Number of activated ports Port spacing state Belongs to the state space , is represented as:

[0097] ;

[0098] in, The distance between adjacent ports of the FAS is calculated using the following formula:

[0099] ;

[0100] in For wavelength, This is the total number of FAS ports.

[0101] Action space :

[0102] Includes adjustment actions for the length of the FAS. and the action of adjusting the number of active ports. That is, action ,in It is a multidimensional discrete action space;

[0103] reward function :

[0104] Defined as a weighted combination of channel capacity and modeling error, its expression is:

[0105] ;

[0106] in, It is a weight value for channel capacity. It is the weight value of the model error of the FAS-assisted satellite channel.

[0107] Step 6: Optimize the solution using the Memetic PPO algorithm in the FAS-assisted satellite channel based on reinforcement learning. The specific steps are as follows:

[0108] Step 601: When using the Memetic PPO algorithm for optimization in the FAS-assisted satellite channel based on reinforcement learning, initialize the policy network. and value network parameters and And set up an experience replay buffer to store trajectory data generated by the agent's interaction with the environment; initialize the state space and action space;

[0109] Step 602: Trajectory Acquisition and Advantage Estimation: The agent follows the current strategy Interact with the environment and collect trajectory data ,in, express The state at that time, In order to take action The reward received later The new state is represented, and then the advantage function is calculated using generalized advantage estimation. The calculation formula is as follows:

[0110] ;

[0111] Among them, the time difference term ,in for Rewards at that time As a discount factor, For parameters The state-value function approximated by the neural network, Indicates from state To begin, follow the expected cumulative discount return that can be obtained by the current strategy. This represents the offset at a future time step. This represents the exponentially decaying weight, applied to the future... The timing difference error of the step, The generalized advantage estimation parameter determines the agent's emphasis on future rewards and directly affects how the reward is calculated; its value range is... When it is close to 1, the agent tends to consider long-term benefits, which is suitable for tasks that require long-term planning. When it is close to 0, the agent focuses more on immediate rewards, which helps to accelerate convergence, but may ignore long-term goals.

[0112] Step 603: Optimization of agent pruning objectives in the policy network: Defining probability ratios ,in, This represents the old policy. Based on this probability ratio, a pruning agent objective function is constructed. The policy network parameters are updated by maximizing this objective function. The calculation formula is as follows:

[0113] ;

[0114] in, The objective function representing the pruning surrogate is the core of the PPO algorithm. For the expectation operator, This indicates that for all time steps Calculate the average of the samples. Importance weight Limited to the range Internally, to prevent the new strategy from deviating too far from the old strategy, This represents the pruning factor; its core function is to limit the magnitude of policy updates, thereby ensuring that the new policy does not deviate too far from the old policy and improving training stability; it is calculated by the ratio of action probabilities between the new and old policies. and restrict it to those by Within the specified range, it avoids policy collapse or drastic performance fluctuations due to excessive gradient updates, significantly reducing implementation complexity;

[0115] Step 604: Value Network Optimization: Update the value network parameters by minimizing the mean squared error of the value network. Its expression is:

[0116] ;

[0117] in, It is the loss function of the value network, which allows it to more accurately predict the value of a state. Accumulate rewards for discounts;

[0118] Step 605: Local Search Mechanism: Based on the characteristics of the discrete action space, candidate actions are output in the policy network. Subsequently, a local search mechanism is introduced for refined action selection; defining... Centered on, with radius neighborhood :

[0119] ;

[0120] in, As candidate actions, This means candidate actions With each dimension The absolute difference does not exceed ;

[0121] While keeping the environment state unchanged, evaluate the immediate rewards of all candidate actions in the neighborhood and select the optimal action as the final action to be executed, as given by the following expression:

[0122] ;

[0123] in, That is the optimal action obtained in the end. This represents the reward obtained by calculating the immediate reward function;

[0124] Step 606: Repeat steps 602 to 605 iterating until convergence, and output the optimized FAS configuration parameters. and .

[0125] like Figures 2 to 4 As shown, the simulation results of the reinforcement learning-based FAS-assisted satellite-to-ground communication performance optimization proposed in this invention are presented. Figure 2 As shown, in the scenario of FAS-assisted satellite-to-ground communication performance optimization based on reinforcement learning, different FAS configuration parameters (antenna length) are demonstrated. and number of activated ports The impact of the number of active ports on channel capacity. Simulation results indicate that channel capacity increases with the number of active ports. The channel capacity increases significantly with the increase in the number of active ports. This is because more active ports increase the rank of the channel matrix, thereby increasing the spatial degrees of freedom and allowing the system to make fuller use of the diversity gain brought by multipath propagation. For a fixed number of active ports, the channel capacity increases with the antenna length. The change in [value] shows a trend of first increasing and then decreasing, indicating that there exists an optimal antenna length that maximizes the channel capacity. When When the size is too small, the FAS ports are too concentrated, resulting in insufficient spatial sampling; when When the spacing is too large, the excessive port spacing leads to a decrease in the utilization of some spatial dimensions. Therefore, we can conclude that in FAS-assisted satellite communication systems, the appropriate selection of antenna length and the number of active ports is crucial for improving channel capacity, and both need to be optimized collaboratively to achieve optimal performance.

[0126] like Figure 3 As shown, in the scenario of FAS-assisted satellite-to-ground communication performance optimization based on reinforcement learning, different FAS configuration parameters (number of active ports) are demonstrated. and port spacing The impact of the modeling error on the proposed FAS channel model is investigated. The modeling error is defined as the normalized absolute error between the proposed FAS channel model and the baseline ULA channel model, used to measure the accuracy of the FAS model. Simulation results indicate that the modeling error increases with the number of active ports. The modeling error increases monotonically with the increase of the number of activated ports. This is because activating more ports within a fixed physical space leads to a denser port distribution, causing the deviation between the FAS channel characteristics and the ULA channel characteristics to gradually widen. For a fixed number of activated ports, the modeling error increases monotonically with the port spacing. The change exhibits a non-linear relationship, when When the values ​​are moderate, the FAS model can fully utilize spatial degrees of freedom while maintaining similarity to the ULA model. However, when the channel environment is more complex (e.g., with a large number of scattering clusters), the modeling error generally increases, reflecting the challenge of accurate modeling in complex multipath environments. Therefore, it can be concluded that when designing a FAS-assisted satellite communication system, a trade-off needs to be struck between the number of active ports and modeling accuracy to avoid model distortion due to an excessive pursuit of port density.

[0127] like Figure 4 As shown, in the scenario of FAS-assisted satellite-to-ground communication performance optimization based on reinforcement learning, the comprehensive score performance of the Memetic PPO algorithm proposed in this invention and the standard PPO algorithm are compared under different FAS configurations. The comprehensive score is defined as a weighted combination of channel capacity and modeling error, where the channel capacity weight is positive and the modeling error weight is negative. A higher comprehensive score indicates better overall system performance. Simulation results show that the Memetic PPO algorithm proposed in this invention can find FAS configurations with higher comprehensive scores, with the weights set to... , In practice, the optimal configuration found by Memetic PPO achieves a higher overall score than the configuration found by Standard PPO. Further analysis shows that the Memetic PPO algorithm, by introducing a local search mechanism on top of Standard PPO, can more effectively explore potential better actions within the neighborhood in the discrete action space, thus avoiding the problem of Standard PPO getting trapped in local optima due to the inefficiency of random exploration. During training, the Memetic PPO algorithm exhibits faster convergence speed and more stable policy updates, maintaining high returns over long-term training. Therefore, we can conclude that the Memetic PPO algorithm proposed in this invention has stronger exploration capabilities and better convergence performance in FAS configuration optimization tasks compared to the standard PPO algorithm, and can more effectively balance the contradiction between channel capacity and modeling error, achieving overall performance optimization of the satellite-to-ground communication system.

[0128] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0129] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

[0130] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for optimizing the performance of FAS-assisted satellite-to-ground communication based on reinforcement learning, characterized in that, Includes the following steps: Step 1: Construct a reinforcement learning-based FAS-assisted satellite channel model, establish a rectangular coordinate system in three-dimensional space, and define that only non-line-of-sight channels exist between the transmitting satellite and the ground user; Step 2: When the signal emitted by the satellite reaches the ground user after being reflected by the scatterer, construct a functional expression for the transmission distance between the transmitter and the ground user and the scatterer; Step 3: Construct the channel complex impulse response function expression for the transmission path of the signal reflected by the scatterer to the ground user, and the complex channel impulse response expression between the antenna and the FAS port; Step 4: Construct the channel capacity and modeling error of the FAS-assisted satellite channel based on reinforcement learning, and derive the expressions for the channel capacity and modeling error; The channel capacity expression is: ； in, express An identity matrix of dimension 1 This represents the signal-to-noise ratio coefficient. This refers to the total number of antennas on the transmitting satellite. Represents the normalized channel matrix. This represents the conjugate transpose of the normalized channel matrix; The model error expression is: ； in, and These represent the complex impulse response function of the FAS-assisted satellite channel model and the complex channel impulse response of the baseline ULA channel model, respectively. The number of ports activated for FAS; Step 5: Define the state space, action space, and reward function in the FAS-assisted satellite channel based on reinforcement learning; Step 6: Optimize the solution using the Memetic PPO algorithm in the FAS-assisted satellite channel based on reinforcement learning. This specifically includes: Step 601: Initialize the policy network and value network parameters and And set an experience replay buffer; Step 602: The agent follows the current policy. It interacts with the environment, collects trajectory data, and uses generalized advantage estimation to calculate the advantage function; Step 603: Define the probability ratio to construct the pruning agent objective function, and update the policy network parameters by maximizing this objective function; Step 604: Update the value network parameters by minimizing the mean square error of the value network. ; Step 605: Output candidate actions in the policy network Then, a local search mechanism was introduced, defining... Centered on, with radius neighborhood ; Evaluate the immediate rewards of all candidate actions within the neighborhood, and select the optimal action as the final action to be executed. The expression is: ； in, That is the optimal action obtained in the end. This represents the reward obtained by calculating the immediate reward function; Step 606: Repeat steps 602 to 605 until convergence, and output the optimized FAS configuration parameters.

2. The method for optimizing FAS-assisted satellite-to-ground communication performance based on reinforcement learning according to claim 1, characterized in that: The specific steps of step 3 are as follows: The expression for the channel complex impulse response function of the transmission path from the scatterer to the ground user is constructed as follows: ； in, Indicates path delay. It is a time-varying path delay, caused by changes in propagation distance due to satellite motion. It is an absolute time variable, reflecting the time-varying characteristics of the channel. It is the impulse function. This indicates that in the proposed FAS channel model based on scattering clusters, the first... root transmitting antenna and the first The complex channel impulse response between FAS ports is expressed as follows: ； in, For wavelength, This represents a random phase, where it is assumed that each phase is independent and follows a certain order. Uniform distribution This indicates that after a signal is reflected and diffracted by a scatterer, its phase will undergo an unpredictable random shift, which is modeled as... Uniform distribution on For the satellite defined in step 201 root antenna to the first In the scattering cluster, the th Time-varying propagation distance of each scatterer Indicates the location index of the transmitting antenna. Indicates antenna The actual position offset relative to the array reference point Indicates the location index of the FAS port. This indicates the distance between adjacent ports in the FAS. Indicates port The actual position offset relative to the FAS reference point, and These are the time-varying departure angles of the emitted wave in the azimuth and vertical planes, respectively. and The angle of arrival of a wave is defined as its angle of arrival in the azimuth and vertical planes.

3. The method for optimizing FAS-assisted satellite-to-ground communication performance based on reinforcement learning according to claim 1, characterized in that: Step 5 includes: state space : Includes FAS physical length Number of activated ports Port spacing state Belongs to the state space , represented as: ; in, The distance between adjacent ports of the FAS is calculated using the following formula: ； in For wavelength, This represents the total number of FAS ports; Action space : Includes adjustment actions for the length of the FAS. and the action of adjusting the number of active ports. That is, action ,in It is a multidimensional discrete action space; reward function : Defined as a weighted combination of channel capacity and modeling error, its expression is: ； in, It is a weight value for channel capacity. It is the weight value of the model error of the FAS-assisted satellite channel.

4. The FAS-assisted satellite-to-ground communication performance optimization method based on reinforcement learning according to claim 1, characterized in that: In step 602, the advantage function is calculated using generalized advantage estimation. The formula is: ； Among them, the time difference term ,in for Rewards at that time As a discount factor, For parameters The state-value function approximated by the neural network, Indicates from state Initially, the expected cumulative discount return obtained by following the current strategy is represented by the offset at future time steps. This represents the exponentially decaying weight, applied to the future... The timing difference error of the step, The generalized advantage estimation parameter determines the agent's emphasis on future rewards and directly affects how the reward is calculated; its value range is... When the value is close to 1, the agent tends to consider long-term benefits, which is suitable for tasks that require long-term planning. When the value is close to 0, the agent focuses more on immediate rewards, which helps to accelerate convergence, but may ignore long-term goals.

5. The method for optimizing FAS-assisted satellite-to-ground communication performance based on reinforcement learning according to claim 1, characterized in that: In step 603, the expression for the pruning proxy objective function is: ; in, The objective function representing the pruning surrogate is the core of the PPO algorithm. For the expectation operator, This indicates that for all time steps Calculate the average of the samples. Importance weight Limited to the range Inside, Represents the pruning factor; calculated by the ratio of the action probabilities of the new and old strategies. and restrict it to those by Within the specified range, it avoids policy collapse or drastic performance fluctuations due to excessive gradient updates, significantly reducing implementation complexity; Undefined probability ratio; ,in, This indicates the old strategy.

6. The method for optimizing FAS-assisted satellite-to-ground communication performance based on reinforcement learning according to claim 1, characterized in that: The expression for the mean squared error loss function of the value network in step 604 is as follows: ; in, It is the loss function of the value network, which allows it to more accurately predict the value of a state. Accumulate rewards for discounts.

7. The FAS-assisted satellite-to-ground communication performance optimization method based on reinforcement learning according to claim 1, characterized in that: The neighborhood in step 605 The expression is: ;in, As candidate actions, Indicates candidate actions With each dimension The absolute difference does not exceed .

Images