Communication network optimization method for air RIS attitude change

Abstract

The invention discloses a communication network optimization method for air RIS attitude change, which belongs to the field of wireless communication, and comprises an air RIS assisted communication network scene and an air RIS control method based on an Euler angle, so as to jointly optimize a UAV track and an air RIS attitude, and adopt a deep reinforcement learning algorithm to maximize a system communication rate sum. The coupling relation between the air RIS attitude and the UAV trajectory and the relation between RIS channel gain and orientation are considered, so that an application scene better conforms to the display condition, and a better effect is achieved when the method is applied to an actual scene; the proposed SAC algorithm not only pays attention to maximization of accumulated rewards, but also takes strategy entropy as one of key optimization targets; the weighted sum of the accumulated reward and the strategy entropy is maximized, and the strategy randomness is introduced to encourage the intelligent agent to have higher exploration ability during action selection.

Description

Technical Field

[0001] The present invention belongs to the field of wireless communications, and in particular relates to a method for optimizing a communication network assisted by a reconfigurable intelligent metasurface carried by an unmanned aerial vehicle. Background Art

[0002] As airspace resources become increasingly scarce, Unmanned Aerial Vehicle (UAV), as an important part of the low-altitude economy, has been widely used in emergency communications, signal coverage, and intelligent transportation due to its high mobility and rapid deployment capabilities. At the same time, Reconfigurable Intelligent Surface (RIS), as a key technology, can effectively improve spectrum efficiency and reduce energy consumption by dynamically controlling the phase of the reflected signal. However, the complexity of low-altitude airspace often has a serious impact on communication quality, and traditional UAV or RIS independent deployment is difficult to effectively cope with this challenge. By integrating UAV with RIS, the UAV-RIS assisted communication system can enhance the reliability and adaptability of transmission, thereby ensuring stable and efficient communication performance in the key tasks of the low-altitude economy.

[0003] However, in actual aerial RIS deployment, due to the inertial drag and aerodynamic effects of the UAV during acceleration and deceleration, the fuselage inevitably tilts, resulting in beam deviation and channel changes, thereby reducing the performance of aerial RIS-assisted communications. In addition, existing studies have shown that the actual gain of RIS is highly sensitive to the signal incident angle and reflection angle. Despite these physical limitations, most current studies ignore the impact of aerial RIS attitude changes, resulting in system performance failing to reach the theoretical upper limit of aerial RIS gain. This persistent modeling defect severely limits the effectiveness of aerial RIS in practical applications. Summary of the invention

[0004] The technical problem to be solved by the present invention is to propose a flight control paradigm based on Euler angles in response to the shortcomings of the background technology, thereby realizing the attitude optimization of RIS in the air; through the Euler angle control strategy, the framework can perform real-time phase offset compensation while optimizing attitude adjustment, thereby maintaining optimal beam alignment.

[0005] The present invention adopts the following technical solutions to solve the above technical problems: A communication network optimization method for aerial RIS attitude changes, including an aerial RIS-assisted communication network scenario, wherein the aerial RIS-assisted communication network scenario includes a communication environment between a base station and a ground user and a UAV-RIS control environment; The communication environment between the base station and the ground user includes a set of ground users and the base station; in this environment, the base station can provide communication services for multiple ground users at the same time; wherein, the communication links between the base station and the user are divided into two types: direct link and reflection link; the direct link is that the signal is directly transmitted to the ground user through free space propagation; the reflection link is that the signal is sent from the base station, reflected by the air RIS, and then transmitted to the ground user; The control environment of the UAV-RIS includes UAV and RIS arrays; in this control environment, the UAV platform, as a carrier, provides air signal reflection relay between the base station and the ground user through flexible flight capabilities; during the flight and hovering process, the UAV is affected by inertia and air resistance, and its fuselage posture including roll angle, pitch angle and yaw angle will change dynamically, which will lead to the deviation of RIS reflection angle and signal incident angle, thereby causing beam alignment error and channel gain fluctuation, which directly affects the communication performance of the reflection link; The specific steps include: Step 1: The drone is equipped with RIS and hovers or flies over the ground user. The base station sends a communication signal to the RIS in the air through a downlink. The RIS performs reflection enhancement and beam adjustment on the signal through real-time phase control to ensure that the reflected signal is accurately aligned with the target user. The optimized signal is transmitted to the ground user through the reflection link. Step 2: In order to cope with the changes in the UAV attitude, including the roll angle, pitch angle and yaw angle, and the angular impact on the received and reflected signals, that is, the impact on the system communication performance, the UAV attitude and RIS phase shift are jointly optimized based on the Soft Actor-Critic (SAC) deep reinforcement learning method, thereby maximizing the system communication rate and; Step 3: In each time slot l ,Through the state analysis of the RIS attitude and the trajectory of the UAV in the air, the optimal strategy is determined, and the UAV carries out the next action according to the obtained optimal strategy.

[0006] As a further preferred solution of the communication network optimization method for RIS attitude changes in the air of the present invention, in step 1, the environment adopts a three-dimensional Cartesian coordinate system, the UAV flies horizontally at a fixed altitude, and the ground users are randomly distributed.

[0007] As a further preferred solution of the communication network optimization method for the change of RIS posture in the air of the present invention, in step 2, the angles of the received and reflected signals will be represented by Euler angles, thereby representing the received and reflected beams, and the system communication rate and are calculated; the SAC-based deep reinforcement learning method is to improve the exploration efficiency through the maximum entropy strategy, so that the exploration and utilization can be balanced in the strategy evaluation and strategy improvement stages.

[0008] As a further preferred solution of the communication network optimization method for the change of the RIS attitude in the air of the present invention, in step 3, the optimal strategy includes: adjusting the flight trajectory of the UAV and the attitude of the RIS in the air according to the current environment, so as to ensure the optimal signal incident angle and reflection angle, thereby maximizing the channel gain of the RIS and improving the communication performance of the system.

[0009] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: 1. The present invention provides an aerial RIS control method based on Euler angles to jointly optimize the UAV trajectory and the aerial RIS attitude, and adopts a deep reinforcement learning algorithm to maximize the system communication rate and; the present invention considers the coupling relationship between the aerial RIS attitude and the UAV trajectory, as well as the relationship between the RIS channel gain and orientation, so that the application scenario is more in line with the display situation, and can achieve better results when applied to actual scenarios; 2. The SAC algorithm proposed in this invention not only focuses on maximizing the cumulative reward, but also takes the policy entropy as one of the key optimization objectives; the algorithm aims to maximize the weighted sum of the cumulative reward and the policy entropy, and by introducing policy randomness, it encourages the agent to have a higher exploration ability when selecting actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is an application scenario of the present invention, a UAV equipped with a RIS-assisted communication network environment; Figure 2 It is a flow chart of the SAC algorithm of the present invention; Figure 3 It is a comparison of the convergence performance of the SAC algorithm used in the present invention and other deep reinforcement learning algorithms (PPO, DDPG) under the system model of the present invention; Figure 4 It is a flow chart of the communication network optimization method based on the change of RIS posture in the air of the present invention. DETAILED DESCRIPTION

[0011] The present invention is described in detail below based on the accompanying drawings and preferred examples, and the purpose and effect of the present invention will be more clearly understood. The preferred examples described herein are only used to explain the present invention and are not used to limit the present invention: The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The present invention is described in detail below according to the drawings and preferred embodiments, and the purpose and effect of the present invention will become clearer. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0012] A communication network optimization method for aerial RIS posture changes. The aerial RIS-assisted communication network scenario includes the communication environment between the base station and the ground user and the control environment of the UAV-RIS.

[0013] The communication environment between the base station and the ground user includes the ground user set and the base station. In this environment, the base station can provide communication services to multiple ground users at the same time. The communication link between the base station and the user can be divided into two types: direct link and reflection link. The direct link is that the signal is directly transmitted to the ground user through free space propagation; the reflection link is that the signal is sent from the base station, reflected by the air RIS, and then transmitted to the ground user.

[0014] The control environment of the UAV-RIS includes UAV and RIS arrays. In this control environment, the UAV platform, as a carrier, provides air signal reflection relay between the base station and the ground user through flexible flight capabilities. During the flight and hovering process, the UAV is affected by inertia and air resistance, and its fuselage posture (including roll angle, pitch angle and yaw angle) will change dynamically, which will lead to the deviation of the RIS reflection angle and the signal incident angle, thereby causing beam alignment errors and channel gain fluctuations, which directly affect the communication performance of the reflection link.

[0015] The optimization of the RIS attitude change in the air is achieved by constructing a dynamic model based on Euler angles, establishing the relationship between the incident signal angle and the reflected signal angle in the reflection link and the Euler angle, and further deriving the relationship between the RIS gain and the Euler angle. Under this modeling, in order to maximize the RIS gain and thus improve the communication performance, the present invention proposes a SAC deep reinforcement learning method, which transforms the communication performance optimization problem into a sequential decision-making problem, and seeks a weighted, feasible and optimal strategy for adjusting the entropy of the strategy.

[0016] like Figure 4 As shown, the communication network optimization method based on the air RIS attitude change specifically includes the following steps: (1) The drone is equipped with RIS and hovers or flies over the ground user. The base station sends communication signals to the RIS in the air through the downlink. The RIS performs reflection enhancement and beam adjustment on the signal through real-time phase control to ensure that the reflected signal is accurately aimed at the target user. The optimized signal is transmitted to the ground user through the reflection link.

[0017] (2) In order to cope with the angular impact of UAV attitude changes, including roll angle, pitch angle and yaw angle, on the received and reflected signals, that is, the impact on the system communication performance, a SAC-based deep reinforcement learning method is proposed to jointly optimize the UAV attitude and RIS phase shift, thereby maximizing the system communication rate and.

[0018] (3) In each time slot l ,Through the state analysis of the RIS attitude and the trajectory of the UAV in the air, the optimal strategy is determined, and the UAV carries out the next action according to the obtained optimal strategy.

[0019] Furthermore, the environment system in step (1) includes the following features: the environment adopts a three-dimensional Cartesian coordinate system, the UAV flies horizontally at a fixed altitude, and the ground users are randomly distributed. Considering that each time slot interval is very short, it can be assumed that the terminal user is static in each time slot. At the same time, the base station position is fixed.

[0020] Furthermore, the angles of the received and reflected signals in step (2) are represented by Euler angles, thereby representing the received and reflected beams, and finally calculating the system communication rate and. In addition, the core idea of ​​the proposed SAC algorithm is to improve the exploration efficiency through the maximum entropy strategy, so that exploration and utilization can be balanced in the strategy evaluation and strategy improvement stages.

[0021] Furthermore, the optimal strategy of step (3) includes adjusting the flight trajectory of the UAV and the attitude of the RIS in the air according to the current environment, so as to ensure the optimal signal incident angle and reflection angle, thereby maximizing the channel gain of the RIS and ultimately improving the communication performance of the system. The optimization target is the system communication rate and.

[0022] In view of the shortcomings of existing research, the present invention proposes a flight control paradigm based on Euler angles to achieve attitude optimization of RIS in the air. Through the Euler angle control strategy, the framework can perform real-time phase offset compensation while optimizing attitude adjustment, thereby maintaining optimal beam alignment. In addition, the present invention uses the system communication rate and as performance indicators to reflect the effectiveness of attitude optimization of RIS in the air. Furthermore, the present invention reconstructs the communication rate maximization problem into a model based on Markov decision process, and proposes a deep reinforcement learning method based on SAC.

[0023] The present invention designs an aerial RIS control scheme based on Euler angles to jointly optimize the UAV trajectory and aerial RIS attitude, and adopts a deep reinforcement learning algorithm to maximize the system communication rate and.

[0024] Compared with other inventions, the present invention takes into account the coupling relationship between the RIS attitude in the air and the UAV trajectory, as well as the connection between the RIS channel gain and the orientation, so that the application scenario is more in line with the display situation and can achieve better results when applied to actual scenarios.

[0025] Some of the other inventions use traditional optimization methods, such as block coordinate descent, successive convex approximation and semidefinite relaxation, but these traditional methods usually have high computational complexity and often generate static solutions in complex environments. They cannot adapt to dynamically changing communication scenarios, resulting in the optimization results becoming suboptimal or outdated in actual deployment. The other part often uses DQN or DDPG or PPO algorithms, and these traditional deep reinforcement learning algorithms often exhibit problems such as slow convergence and low training efficiency, and are prone to falling into local optimal solutions, making it difficult to obtain the global optimal UAV autonomous maneuvering decision. The SAC algorithm proposed in the present invention not only focuses on maximizing the cumulative reward, but also takes policy entropy as one of the key optimization objectives. The algorithm aims to maximize the weighted sum of the cumulative reward and policy entropy, and encourages the agent to have higher exploration capabilities when selecting actions by introducing policy randomness.

[0026] Specific embodiment: System model: Figure 1 As shown in FIG. 1 , the aerial RIS-assisted wireless communication scenario includes K single-antenna user devices, a multi-antenna base station, and a UAV with RIS on the bottom. The entire flight cycle of the UAV is T For ease of processing, the total flight time T Divide into equal intervals L time slots, and the length of each time slot is The position of the UAV will change between adjacent time slots. In order to accurately describe the flight trajectory of the UAV, a three-dimensional Cartesian coordinate system is used to model the flight state of the UAV, assuming that the flight altitude of the UAV is a fixed value. H In the l In each time slot, the position coordinates of the UAV are expressed as RIS angle calculation and attitude transformation: In the local coordinate system, the initial unit normal vector of the RIS in the air is represented by e. In order to realize the spatial coordinate transformation from the global coordinate system to the local coordinate system, a two-stage coordinate transformation method is adopted. First, an initial translation transformation is performed to convert the coordinate origin to Pan to the instantaneous position of the UAV , thus ensuring the accurate spatial positioning of RIS in the global reference frame. Next, the attitude is adjusted with the help of rotation transformation, and the attitude correction is achieved through a series of rotation operations parameterized by Euler angles. The rotation transformation includes three consecutive operations: the roll angle around the x-axis, the pitch angle around the y-axis, and the yaw angle around the z-axis, which are denoted as The corresponding transformation matrices that control these rotations are expressed as: ;in, is the transformation matrix that controls the roll angle, is the transformation matrix that controls the pitch angle, is the transformation matrix that controls the yaw angle.

[0027] The complete attitude transformation matrix is Then, after applying this coordinate transformation, the unit normal vector of the RIS in the air can be derived in the global coordinate system as , where e represents the initial normal vector of the RIS in the air, and the multiplication results in is the transformed unit normal vector. In addition, the unit direction vector of the incident signal from the base station to the airborne RIS and the reflected signal from the airborne RIS to the ground user can be expressed as: in and Respectively represent the time slot l , the azimuth and elevation angles from the base station or user equipment to the RIS in the air. By using these direction vectors, the angle between the normal vector of the RIS plane in the air and the incident / reflected signal can be derived as: It can be seen that the change of the azimuth angle of the airborne RIS significantly affects the direction of the incident and reflected signals, thereby changing the equivalent receiving aperture. This change directly affects the airborne RIS gain and thus changes the performance of the entire system, because the change in signal reflection characteristics directly determines the link quality and communication efficiency.

[0028] Communication model: In any time slot, the channel gains between the base station and the airborne RIS and between the airborne RIS and the ground user are expressed as and .

[0029] Since the actual gain of the airborne RIS is significantly affected by the signal incident angle and reflection angle, the actual gain model introduces an expression that takes into account the azimuth angle and elevation angle. Specifically, the actual gain of the airborne RIS can be modeled as the product of the maximum directivity coefficient of the airborne RIS and the reflection angle characteristic function: ;in, and They represent the receiving gain from the base station to the airborne RIS and the transmitting gain from the airborne RIS to the user equipment respectively. and They are the normalized directional radiation functions from the base station to the airborne RIS and from the airborne RIS to the user equipment, respectively.

[0030] At the same time, in order to more accurately describe the gain characteristics of RIS, the maximum directivity coefficient is introduced. and the normalized directional radiation function This function reflects the directional radiation characteristics of the aerial RIS and is modeled as an exponential-Lambert radiation model, and its expression is: in and They represent the azimuth and elevation angles respectively, and are used to define the spatial relationship between the ground user (or base station) and the aerial RIS.

[0031] Based on these mathematical expressions, the RIS gain expression can be further deduced as: in is the RIS phase shift matrix. This expression reflects the angle selection characteristics of RIS for the reflected signal, that is, when the reflection angle meets certain conditions, RIS can effectively reflect the signal, otherwise the signal cannot be reflected.

[0032] Furthermore, in the airborne RIS-assisted wireless communication system, the ground user l The received signal can be expressed as: in It is a composite channel from the base station to the ground user, including the direct link and reflection link from the base station to the ground. is the beamforming vector; It is to send a signal; is additive white Gaussian noise and obeys complex Gaussian distribution, and the noise variance is Then, the ground user in the time slot l The communication rate within is: Finally, the total rate of the entire system over all time slots and all users can be expressed as: .

[0033] Markov decision process: In deep reinforcement learning applications, we first define the Markov decision process, which is the basic framework for solving sequential decision problems in a random environment. A Markov decision process usually consists of five parts: state space, action space, state transition probability function, reward function, and discount factor. l , the agent is based on the policy Choose an action. The policy defines the probability distribution of choosing an action given a state. ,in and For the time slot l The state space set and action space set under ; and Indicates the status and actions under the current policy.

[0034] After each action is executed, the system will transition to the next state according to the state transition probability function and give the agent an immediate reward. This state transition and reward feedback process guides policy optimization, and the agent adjusts its strategy through iterative interaction with the environment to gradually approach the optimal strategy.

[0035] SAC algorithm: Figure 2 As shown. Under the SAC algorithm framework, the strategy The distribution of the next state-action trajectory is expressed as Unlike traditional deep reinforcement learning algorithms, SAC introduces an entropy regularization term and integrates it into the objective function to improve the exploration efficiency of the strategy. Its optimization goal is to maximize the sum of the cumulative reward and the strategy entropy: in is the entropy of the strategy distribution; For all state-action pairs Distribution Sampling and expectation are performed. The entropy regularization term is introduced by introducing the temperature parameter To control the weight of entropy in the optimization process and balance the relationship between exploration and utilization. The SAC algorithm adopts a policy iteration framework, alternating between policy evaluation and policy improvement. In the policy evaluation phase, the soft state value function is calculated based on the Bellman expectation equation, and its expression is: in, state-action pair The immediate reward function under ; is the discount factor; is the state transition probability distribution of the environment; For the status Follow the strategy The expected cumulative benefit if the action continues.

[0036] In order to improve sampling efficiency, the SAC algorithm adopts a dual network structure, including an actor network and two critic networks. The critic network is used to estimate the Q value by minimizing the temporal difference loss. The loss function is expressed as: in is the target Q value, defined as: in is the immediate reward function; is the Q value of the next state estimated by the current Q network.

[0037] In the policy improvement phase, SAC uses the policy gradient method to optimize the policy, and the update goal is to minimize the policy network loss function: in, To sample states from experience replay ; For the previous strategy in state Take action The probability distribution of is the current estimated state-action value function.

[0038] In the SAC algorithm, each training iteration extracts batch samples from the experience replay buffer, optimizes the critic network to reduce the temporal difference error, and improves the policy performance by optimizing the actor network. In policy optimization, entropy regularization effectively enhances the randomness of action selection, improves exploration efficiency and algorithm stability.

[0039] Under the communication network model of the aerial RIS attitude change proposed in the present invention, the proposed SAC algorithm shows good performance. Compared with the traditional deep reinforcement learning algorithms PPO and DDPG, the SAC algorithm converges faster and has a larger reward value. In addition, due to its deterministic characteristics, DDPG usually converges faster and is more stable, but it lacks an inherent exploration mechanism, which limits its performance in complex environments and makes it easy to fall into local optimal solutions. The convergence performance of the SAC algorithm used in the present invention and other deep reinforcement learning algorithms (PPO, DDPG) under the system model of the present invention is compared. Figure 3 shown.

[0040] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention should be included in the protection scope of the invention. All technical features in this embodiment can be freely combined according to actual needs.

[0041] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A communication network optimization method for RIS attitude changes in the air, characterized by: The communication network scenario includes an aerial RIS-assisted communication network scenario, wherein the aerial RIS-assisted communication network scenario includes a communication environment between a base station and a ground user and a control environment of the UAV-RIS; The communication environment between the base station and the ground user includes a set of ground users and the base station; in this environment, the base station can provide communication services for multiple ground users at the same time; wherein, the communication links between the base station and the user are divided into two types: direct link and reflection link; the direct link is that the signal is directly transmitted to the ground user through free space propagation; the reflection link is that the signal is sent from the base station, reflected by the air RIS, and then transmitted to the ground user; The control environment of the UAV-RIS includes UAV and RIS arrays; in this control environment, the UAV platform, as a carrier, provides air signal reflection relay between the base station and the ground user through flexible flight capabilities; during the flight and hovering process, the UAV is affected by inertia and air resistance, and its fuselage posture including roll angle, pitch angle and yaw angle will change dynamically, which will lead to the deviation of RIS reflection angle and signal incident angle, thereby causing beam alignment error and channel gain fluctuation, which directly affects the communication performance of the reflection link; The specific steps include: Step 1: The drone is equipped with RIS and hovers or flies over the ground user. The base station sends a communication signal to the RIS in the air through a downlink. The RIS performs reflection enhancement and beam adjustment on the signal through real-time phase control to ensure that the reflected signal is accurately aligned with the target user. The optimized signal is transmitted to the ground user through the reflection link. Step 2: In order to cope with the changes in the UAV attitude, including the roll angle, pitch angle and yaw angle, and the angular impact on the received and reflected signals, that is, the impact on the system communication performance, the SAC deep reinforcement learning method is used to jointly optimize the UAV attitude and RIS phase shift, thereby maximizing the system communication rate and; Step 3: In each time slot l ,Through the state analysis of the RIS attitude and the trajectory of the UAV in the air, the optimal strategy is determined, and the UAV carries out the next action according to the obtained optimal strategy.

2. A communication network optimization method for RIS attitude changes in the air according to claim 1, characterized in that: In step 1, the environment adopts a three-dimensional Cartesian coordinate system, the UAV flies horizontally at a fixed altitude, and the ground users are randomly distributed.

3. A communication network optimization method for airborne RIS attitude changes according to claim 1, characterized in that: In step 2, the angles of the received and reflected signals will be represented by Euler angles, thereby representing the received and reflected beams, and calculating the system communication rate and; the SAC-based deep reinforcement learning method is to improve the exploration efficiency through the maximum entropy strategy, so that exploration and utilization can be balanced in the strategy evaluation and strategy improvement stages.

4. The communication network optimization method for RIS attitude changes in the air according to claim 1, characterized in that: In step 3, the optimal strategy includes: adjusting the flight trajectory of the UAV and the attitude of the RIS in the air according to the current environment to ensure the optimal signal incident angle and reflection angle, thereby maximizing the channel gain of the RIS and improving the communication performance of the system.

Images