Unmanned aerial vehicle emergency communication method, system and equipment assisted by intelligent reflecting surface, and medium

Through the H-PPO algorithm, the drone trajectory and intelligent reflective surface phase are optimized, and the energy efficiency and service quality problems of the drone communication system in extreme environments are solved, and effective coordination and efficient communication of multi-user resources are achieved.

CN119052830AActive Publication Date: 2024-11-29XIAN MOLI TECH CO LTD

Patent Information

Application Number
CN202411272370.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-11-29
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

The existing technology is difficult to adapt to the UAV communication system in extreme environments. Traditional optimization methods cannot effectively optimize the phase of the UAV trajectory and intelligent reflection surface, resulting in low energy efficiency and poor user service quality, and cannot solve the problem between multiple users. the issue of resource competition.

Method used

Using deep reinforcement learning algorithms, especially Hierarchical Proximal Policy Optimization (H-PPO) algorithm, optimize the flight trajectory and phase of intelligent reflection surfaces of the drone, and maximize system energy consumption and user services by building models and reward functions. Guarantee of quality.

Benefits of technology

In extreme environments, the energy efficiency of the drone communication system is improved, the service quality of all ground users is ensured, the resource competition problem between users is solved, and more flexible and lasting communication services are provided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052830A_ABST
    Figure CN119052830A_ABST
Patent Text Reader

Abstract

The invention relates to an intelligent reflecting surface assisted unmanned aerial vehicle emergency communication method, system and device, and a medium. The method comprises the following steps: constructing an unmanned aerial vehicle carried intelligent reflecting surface assisted energy-saving emergency communication scene; modeling is carried out on the communication process of UAV-RIS assisted multiple ground devices in an unmanned aerial vehicle carried intelligent reflection surface assisted energy-saving emergency communication scene; modeling propulsion energy consumption and trajectory constraint of the unmanned aerial vehicle; modeling the optimization target of the final energy consumption ratio, and constructing a target function when the model is solved; constructing a deep reinforcement learning algorithm model according to the optimization target; constructing a deep reinforcement learning training model, and setting a state space, an action space and a reward function; training the deep reinforcement learning training model to obtain an auxiliary energy-saving emergency communication decision model of the intelligent reflecting surface carried by the unmanned aerial vehicle, and obtaining an optimal solution of an optimization problem; systems, devices, and media for implementing the method; the method has the advantage of being capable of providing more flexible and more durable communication service in a severe emergency scene.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle communication, and in particular relates to an unmanned aerial vehicle emergency communication method, system, equipment and medium assisted by an intelligent reflective surface. Background Art

[0002] The increase in the frequency and intensity of natural disasters has brought significant challenges to effective communications in extreme and dynamic environments. Timely and reliable data transmission from ground devices (GDs) to central processing units is essential for effective disaster detection and response. Based on the flight flexibility of UAVs, UAV communication systems can generally provide line-of-sight channels in wide terrains, so UAVs can be used as aerial communication platforms to improve the performance of ground communication systems. Innovations in unmanned aerial vehicle (UAV) technology have been effectively utilized in disaster relief. Unmanned aerial vehicles (UAVs) can assist ground equipment in disaster perception, with advantages such as flexibility, wide coverage and line-of-sight communication.

[0003] The combination of reconfigurable smart surfaces (RIS) and unmanned aerial vehicles (UAVs) has recently attracted widespread attention. When a reconfigurable smart surface (RIS) is mounted on an unmanned aerial vehicle (UAV), it can act as a mobile relay. This innovative combination allows precise manipulation of electromagnetic wave propagation and provides a new dimension for improving the performance of communication systems. For example, there have been works that simultaneously adjusted the position of the UAV and the reflection coefficient of the smart reflector in the UAV-mounted smart reflector system, ultimately achieving the maximum downlink transmission capacity. In addition, in UAV communications, the propulsion energy consumption of the UAV is critical to maintaining a sustainable emergency communication system. Most of the existing works use traditional optimization methods to optimize the trajectory of the UAV. However, traditional optimization methods face huge challenges in real-time decision-making on the flight trajectory of the UAV and the phase of the reconfigurable smart surface (RIS). Deep reinforcement learning (DRL) provides a feasible solution to the above challenges. Most of the existing works using reinforcement learning for optimization use discrete DRL algorithms to optimize the trajectory of the UAV and the phase offset of the reconfigurable smart surface (RIS). However, the discrete action algorithms used to process the trajectory and phase limit the design space and have a negative impact on the full utilization of their potential performance.

[0004] In addition, in existing research, there is no research on drone-mounted smart reflective surfaces that simultaneously solve the problem of optimizing energy efficiency and ensuring the quality of service for individual users in scenarios that require long decision sequences. Solving the problem of competition between users for reconfigurable smart surface (RIS) resources is crucial to ensuring excellent service for individual users, which is an issue that many studies have overlooked. For scenarios where drone-mounted smart reflective surface systems assist multiple users in achieving emergency communications, how to meet the need for rapid decision-making. Considering the mobility of ground equipment, how to design the trajectory of the drone to maximize the energy efficiency of the system. Considering the competition among all ground equipment for reconfigurable smart surface (RIS) resources, how to design the phase of the smart reflective surface to ensure the quality of service (QoS) for all ground users.

[0005] Based on the above analysis, the shortcomings of the existing technology are:

[0006] (1) The existing technology only considers the communication problem of UAVs equipped with intelligent reflective surface networks. In fact, the propulsion energy consumption of UAVs is very important for UAVs. UAV systems that only consider communication performance are difficult to adapt to emergency communications in extreme environments.

[0007] (2) The existing technology does not take into account the competition among all ground devices for RIS resources and cannot guarantee the individual service quality of each ground user.

[0008] (3) Existing optimization methods are difficult to adapt to complex scenarios and cannot solve the proposed joint trajectory and phase design optimization problem.

[0009] The patent application document with publication number CN113873575B discloses an energy-saving optimization method for non-orthogonal multiple access UAV air-to-ground communication network assisted by an intelligent reflective surface. The energy-saving optimization method for non-orthogonal multiple access UAV air-to-ground communication network assisted by an intelligent reflective surface effectively reduces the transmission power of the UAV, improves the endurance of the UAV, and saves energy. However, the reconfigurable intelligent surface (RIS) is placed next to the NOMA remote user, which makes the reconfigurable intelligent surface (RIS) relay mobility very poor and cannot meet the needs of mobile scenarios.

[0010] The patent application document with publication number CN117596616A discloses a method for optimizing the energy efficiency of an uplink NOMA drone network assisted by an intelligent reflective surface. Through a method for optimizing the energy efficiency of an uplink NOMA drone network assisted by an intelligent reflective surface, the maximum energy efficiency of the uplink NOMA drone network assisted by an intelligent reflective surface is achieved. However, its optimization method is traditional optimization, which uses alternating optimization to optimize two sub-problems. However, due to the high complexity of the optimization problem and the dynamic nature of the scene, it is impossible to use mathematical methods to infer a low-complexity optimal solution to achieve dynamic and rapid service decisions for multiple users. Summary of the invention

[0011] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a UAV emergency communication method assisted by an intelligent reflective surface, which assists the uplink communication in emergency scenarios by carrying an intelligent reflective surface system (UAV-RIS) carried by a UAV, and proposes a hierarchical proximal policy optimization (H-PPO) algorithm to simultaneously optimize the flight trajectory of the UAV and the phase of the intelligent reflective surface, thereby solving the complex joint optimization problem of the UAV trajectory and the phase shift of the intelligent reflective surface, achieving the goal of improving the energy efficiency of the system, and effectively ensuring the quality of service (QoS) requirements of all ground equipment; the present invention also provides a system, device and medium for implementing the above method.

[0012] In order to achieve the above object, the technical solution adopted by the present invention is:

[0013] An intelligent reflective surface-assisted unmanned aerial vehicle emergency communication method comprises the following steps:

[0014] Step 1: Construct a drone-mounted intelligent reflective surface-assisted energy-saving emergency communication scenario;

[0015] Step 2, modeling the communication process of a UAV-RIS assisting multiple ground devices in the UAV-RIS assisted energy-saving emergency communication scenario constructed in step 1;

[0016] Step 3, modeling the UAV propulsion energy consumption;

[0017] Step 4, combining the UAV-RIS system (UAV-RIS) in the UAV-RIS energy-saving emergency communication scenario in step 2 and the UAV propulsion energy consumption model in step 3, modeling the optimization target of the final energy consumption ratio and constructing the objective function for solving the model;

[0018] Step 5: construct a deep reinforcement learning algorithm model according to the optimization goal proposed in step 4;

[0019] Step 6: construct a deep reinforcement learning training model based on the deep reinforcement learning algorithm model of step 5, and set the state space, action space and reward function of the deep reinforcement learning training model in combination with the UAV equipped with intelligent reflective surface assisted energy-saving emergency communication scenario of step 1 and the objective function of step 4;

[0020] Step 7, training the deep reinforcement learning training model obtained in step 6, obtaining a decision model for energy-saving emergency communication assisted by an intelligent reflective surface mounted on a UAV, and obtaining the optimal solution to the optimization problem.

[0021] The process of step 1 is as follows:

[0022] The energy-saving emergency communication scenario assisted by a UAV-carried intelligent reflective surface includes a UAV, which is equipped with an active reconfigurable intelligent surface (RIS), namely, a UAV-carried intelligent reflective surface system (UAV-RIS), and an aerial base station (ABS) for processing information transmitted from ground equipment. The aerial base station (ABS) serves as a control center and is responsible for interacting with the UAV-carried intelligent reflective surface system (UAV-RIS) and issuing commands. The UAV-carried intelligent reflective surface system (UAV-RIS) assists the uplink transmission between multiple ground equipment and the aerial base station (ABS). The set of ground equipment is denoted as g = {G m ,m=1,2,…,M}, where M is the number of ground devices. It is assumed that there is no direct communication link between the ground devices and the aerial base station (ABS), and the aerial base station (ABS) is located at a higher position.

[0023] The process of step 2 is as follows:

[0024] Step 2.1, construct a channel model;

[0025] In the channel model, each ground device has an omnidirectional antenna, and the airborne base station (ABS) is equipped with Q antenna arrays. The active reconfigurable smart surface (RIS) consists of N reflective elements. Assume that the reflection coefficient of the nth element of RIS is Among them, φ n ∈[0,2π), β represents an amplification factor greater than 1, and the reflection coefficient matrix of the active reconfigurable smart surface (RIS) is defined as Θ=diag([θ1,θ2,…,θ N ]);

[0026] H m,r represents the channel of the link between the mth ground device and the UAV-RIS, H r,b represents the channel between the UAV-RIS and the airborne base station (ABS). Assuming that the channel gain from the UAV-RIS to the airborne base station (ABS) obeys the Rice distribution, the channel is expressed as:

[0027]

[0028] Among them, κ r,b is the Rice coefficient, l is the path loss at the reference distance D = 1m, d r,b is the distance between the UAV-RIS and the airborne base station (ABS), α r,b is the path loss index of the UAV-RIS-ABS link; the line-of-sight component is the first level, expressed as:

[0029]

[0030] in, is the departure angle (AoD) and arrival angle (AoA) of the link from the UAV-RIS to the airborne base station (ABS), and ω is the antenna spacing;

[0031] Non-line-of-sight component Each element of H follows an independent and identically distributed complex Gaussian distribution with a mean of zero and a variance of 1. Similarly, the channel H m,r Following a distribution similar to the above, the LoS component of the mth link is It is expressed as:

[0032]

[0033] in, is the departure angle (AoD) of the link from the mth ground device to UAV-RIS. The angle of arrival (AoA) and the angle of departure (AoD) are determined by their relative position relationship;

[0034] Finally, it is assumed that all channels follow block fading, and the global channel information is known at the air base station (ABS) and remains constant in each time slot, but varies from one time slot to another;

[0035] Step 2.2, constructing a transmission model based on the channel model constructed in step 2.1;

[0036] In the uplink transmission model from the ground equipment to the airborne base station (ABS), no in-band interference is considered. The airborne base station (ABS) uses maximum ratio combining (MRC) for each link. The maximum ratio combining (MRC) beamforming matrix is ​​F = [f1, ..., f M ] indicates that, among which, f m represents the unit norm beamforming vector of the mth link; the signal received by the aerial base station (ABS) from the mth ground device is expressed as:

[0037]

[0038] Among them, P m is the transmission power of the mth ground device, s m is the unit energy signal sample associated with the monitoring data; the noise vector n generated at the air base station (ABS) m , denoted by n m =[n1,...,n M ] T ,in

[0039] Active reconfigurable smart surfaces (RIS) require additional power and each element is equipped with an amplifier, taking into account the thermal noise generated at the UAV-RIS. i ,in, The uplink signal-to-noise ratio (SINR) of the mth link of the air base station (ABS) is expressed as:

[0040]

[0041] Therefore, the rate of the mth link is C m =Blog(1+η m ), where B is the bandwidth.

[0042] The process of step 3 is as follows:

[0043] The UAV-RIS and the airborne base station (ABS) are used at the horizontal position in time slot t. and It indicates that, The vertical heights of the UAV-RIS and the airborne base station (ABS) are set as h r and h b The fixed height of the UAV carrying intelligent reflective surface system (UAV-RIS) at the next time slot t is determined by the flight distance D t and the azimuth variable ξ t Determined, the horizontal coordinate of the UAV-RIS at time t+1 is expressed as:

[0044] According to the setting of the horizontal position of the UAV-RIS, the horizontal flight speed of the UAV-RIS at time t is is the maximum horizontal velocity, Δt is the length of each time period; if It means that at time t, the UAV is in a hovering state. According to the horizontal speed of the UAV, the energy consumption of the UAV propulsion in each time period is:

[0045]

[0046] Among them, P0 and P1 are the constant power and induced power of the blade in the hovering state, respectively, and U tip is the tip speed of the rotor blade, Δt is the duration of each time slot, v0 is the average rotor induced speed in hover, d0 and s are the airframe drag ratio and rotor solidity, respectively, and ρ and G represent the air density and rotor disk area, respectively.

[0047] The process of step 4 is as follows:

[0048] The reflection coefficient matrix of the smart reflector at the tth time slot is denoted as Θ t , C t,m is the communication rate of the mth link at the tth time slot obtained in step 2, The energy consumption of the UAV propulsion in the t-th time slot obtained in step 3, the objective function of maximizing energy efficiency within a time range of length T is expressed as:

[0049]

[0050]

[0051]

[0052]

[0053] Among them, constraint (1) represents the transmission rate requirement that each ground device needs to meet, and Υ represents the rate limit value; constraint (2) limits the range in which the phase of the reconfigurable intelligent surface (RIS) can be adjusted; constraint (3) represents the maximum flight speed limit of the UAV-RIS.

[0054] The process of step 5 is as follows:

[0055] The deep reinforcement learning algorithm model is constructed by using the H-PPO algorithm to control the flight trajectory and reconfigurable intelligent surface (RIS) phase of the unmanned aerial vehicle carrying intelligent reflective surface system (UAV-RIS). The H-PPO algorithm includes two independent proximal policy optimization algorithms PPO; the first proximal policy optimization algorithm PPO is used to optimize the flight action for trajectory control; the second proximal policy optimization algorithm PPO is used for the reconfigurable intelligent surface (RIS) phase value to enhance the channel gain; the independent proximal policy optimization algorithm PPO is used to optimize the control of the flight trajectory and the reconfigurable intelligent surface (RIS) phase;

[0056] The Proximal Policy Optimization (PPO) algorithm is a deep reinforcement learning (DRL) optimization algorithm. The Proximal Policy Optimization (PPO) algorithm learns the optimal strategy by iteratively updating the strategy to maximize the expected cumulative reward. The strategy of the Proximal Policy Optimization (PPO) algorithm is parameterized and represented by two networks, namely the actor network and the and critic network Among them, θ A and θ CRepresents the corresponding parameter instance; the proximal policy optimization algorithm PPO uses a proxy objective function and a clipping mechanism to constrain policy updates; the objective function in the proximal policy optimization algorithm PPO is:

[0057]

[0058] in, Representation strategy With the old strategy The importance sampling weights between ; for The corresponding advantage function, where γ is the discount factor; the clipping mechanism limits the policy update to a predefined range, where clip() represents the clipping function, and λ is a hyperparameter used to adjust ρ t (θ A )) is controlled in the interval [1-ε,1+ε];

[0059] During the training phase, the network is updated using I batches of experience interacting with the environment; for the actor network The parameter update method is:

[0060]

[0061] At the same time, in order to update θ C , using the mean square error function as the loss function, expressed as:

[0062]

[0063] Among them, α C represents the learning rate, represents the target state value function derived by time difference.

[0064] The settings of the state space, action space and reward function in step 6 are as follows:

[0065] 1) State space: At time slot t, the state of the first proximal policy optimization algorithm PPO contains global position information, expressed as represents the horizontal coordinate of the mth ground device at time slot t; in addition, the state of the second proximal policy optimization algorithm PPO contains the global channel information, which is expressed as Before inputting this complex channel state into the network, the real and imaginary parts of the channel are separated and input into the neural network at the same time;

[0066] 2) Action space: At time slot t, the first action of the proximal policy optimization algorithm PPO (upper layer) is the flight distance and flight direction of the UAV-RIS, expressed as The second proximal policy optimization algorithm PPO (lower layer) takes the phase value of active RIS at time slot t, denoted as Θ t ;

[0067] 3) Reward function: The reward structure of the first proximal policy optimization algorithm PPO consists of two parts; the first is the real-time reward, which includes the ratio of the communication volume to the energy consumption in the current time slot and the difference between the horizontal distance between the UAV-RIS and all ground equipment and the previous time slot, expressed as Among them, P t,m represents the additional positive reward value of the mth ground device when the rate threshold is met in time slot t, Secondly, a termination reward is set that reflects the energy efficiency of the entire service cycle, expressed as λ represents a positive fixed value, which is used to adjust the ratio of the termination reward value to the final reward; in addition, the reward function of the first proximal policy optimization algorithm PPO is expressed as:

[0068]

[0069] Finally, at time slot t, the reward function of the second proximal policy optimization algorithm PPO is expressed as:

[0070]

[0071] The process of step 7 is as follows:

[0072] Step 7.1, Initialize the network

[0073] Randomly initialize the actor network θ of each proximal policy optimization algorithm PPO A and the critic network θ C Parameters;

[0074] Step 7.2: Train the deep reinforcement learning model

[0075] Randomly initialize the position of each ground device, the position of the UAV-RIS, and the position of the aerial base station (ABS);

[0076] At each time slot t, the first proximal policy optimization algorithm PPO interacts with the dynamic environment and obtains the state Based on the current state, the first proximal policy optimization algorithm PPO obtains the drone flight action from its Actor network To control the trajectory of the drone;

[0077] The second proximal policy optimization algorithm PPO interacts with the dynamic environment to obtain the state Based on the current state, the second proximal policy optimization algorithm PPO obtains the action of the RIS phase from its Actor network. To control the intelligent reflective surface; finally take action Interact with the environment and calculate the reward r obtained from the environment t ;

[0078] The status state action action and reward r t Stored in the experience pool;

[0079] When the experience in the experience pool accumulates to a certain amount, N d The parameters of the Critic and Actor networks are updated with samples of different sizes. After one training, the experience pool is cleared and the above process is repeated until the model training converges. The decision-making model of energy-saving emergency communication assisted by intelligent reflective surfaces on drones is obtained.

[0080] Step 7.3, Decision-making Phase

[0081] The decision model of UAV-equipped intelligent reflective surface-assisted energy-saving emergency communication is used in random dynamic UAV-equipped intelligent reflective surface-assisted energy-saving emergency communication. The optimal reconfigurable smart surface (RIS) reflection coefficient matrix and UAV flight action are decided in each time slot, so that the energy consumption ratio of the communication system in the whole process is minimized, while ensuring the communication quality of all individual ground users.

[0082] The present invention also provides an intelligent reflective surface-assisted unmanned aerial vehicle emergency communication system, comprising:

[0083] The UAV emergency communication scenario construction module is used to build a UAV equipped with intelligent reflective surfaces to assist energy-saving emergency communication scenarios;

[0084] The communication process establishment module is used to model the communication process of the UAV-RIS assisting multiple ground devices in the UAV-RIS assisted energy-saving emergency communication scenario;

[0085] UAV propulsion energy consumption establishment module, used to model the UAV propulsion energy consumption;

[0086] The optimization target and objective function construction module is used to combine the UAV-RIS system in the UAV-RIS scenario to assist the communication process of multiple ground devices and the UAV propulsion energy consumption model in the energy-saving emergency communication scenario, model the optimization target of the final energy consumption ratio, and construct the objective function when solving the model;

[0087] A deep reinforcement learning algorithm model building module is used to build a deep reinforcement learning algorithm model according to the proposed optimization objectives;

[0088] The state space, action space and reward function setting module is used to build a deep reinforcement learning training model according to the deep reinforcement learning algorithm model, and to set the state space, action space and reward function of the deep reinforcement learning training model in combination with the UAV equipped with intelligent reflective surface assisted energy-saving emergency communication scenario and objective function;

[0089] The deep reinforcement learning algorithm model training module is used to train the deep reinforcement learning training model, obtain the drone-equipped intelligent reflective surface-assisted energy-saving emergency communication decision-making model, and obtain the optimal solution to the optimization problem.

[0090] The present invention also provides an intelligent reflective surface-assisted unmanned aerial vehicle emergency communication device, comprising:

[0091] Memory: a computer program storing the above-mentioned intelligent reflective surface-assisted UAV emergency communication method, which is a computer-readable device;

[0092] Processor: used to implement the intelligent reflective surface-assisted UAV emergency communication method when executing the computer program.

[0093] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the intelligent reflective surface-assisted unmanned aerial vehicle emergency communication method.

[0094] Compared with the prior art, the present invention has the following beneficial effects:

[0095] 1. Existing traditional optimization cannot use mathematical methods to infer low-complexity optimal solutions in the face of high-complexity and high-dynamic scenarios. In addition, most studies use discrete DRL algorithms to optimize the trajectory of drones and the phase shift of reconfigurable smart surfaces (RIS). However, the discrete action algorithm used to process trajectory and phase limits the design space and has a negative impact on its performance. The present invention uses a deep reinforcement learning algorithm to solve continuous optimization problems, while optimizing the flight trajectory of drones and the phase of smart reflective surfaces. It can adapt to real-time decision-making needs in dynamic scenarios, and solve the complex optimization problem of the trajectory of joint unmanned aerial vehicles (UAVs) and the phase shift of reconfigurable smart surfaces (RIS).

[0096] 2. Currently, there is no research on drones equipped with intelligent reflective surfaces that can simultaneously solve the problem of optimizing energy efficiency and ensuring the service quality of individual users in scenarios that require long decision sequences. The H-PPO algorithm proposed in the present invention effectively solves the joint trajectory and phase design, and through the effective design of the reward function of the H-PPO algorithm, it can maximize the energy consumption ratio of the system while ensuring the individual service quality of each ground user.

[0097] 3. The present invention uses a drone to carry an intelligent reflective surface for auxiliary emergency communication, and utilizes the high mobility of the drone to make the deployment range of the intelligent reflective surface as a mobile relay wider, which can provide more flexible and more durable communication services in harsh emergency scenarios.

[0098] In summary, the present invention uses a drone carrying an intelligent reflective surface to assist emergency communication, and proposes an H-PPO algorithm to simultaneously optimize the flight trajectory of the drone and the phase of the intelligent reflective surface. The service quality of individual ground users is considered in the reward function, and the complex optimization problem of the trajectory of the joint unmanned aerial vehicle (UAV) and the phase shift of the reconfigurable intelligent surface (RIS) is solved. While maximizing the energy consumption ratio of the system, the individual service quality requirements of each ground user are guaranteed, which has the advantage of being able to provide more flexible and more durable communication services in harsh emergency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 It is a flow chart of the implementation method of the present invention.

[0100] Figure 2 It is a schematic diagram of a network scenario of an energy-saving emergency communication assisted by an intelligent reflective surface mounted on a drone provided by an embodiment of the present invention.

[0101] Figure 3 is a schematic diagram of the convergence speed based on the H-PPO algorithm in an embodiment of the present invention; wherein, Figure 3(a) is a graph showing the changing trend of the reward value of the first proximal policy optimization algorithm PPO in the H-PPO algorithm as the training progresses; Figure 3(b) is a graph showing the changing trend of the reward value of the second proximal policy optimization algorithm PPO in the H-PPO algorithm as the training progresses; Figure 3(c) is a graph showing the changing trend of the energy consumption ratio of the system during the training of the H-PPO algorithm; and Figure 3(d) is a graph showing the changing trend of the QOS completion rate of the ground equipment during the training of the H-PPO algorithm.

[0102] Figure 4 A schematic diagram of energy efficiency of different solutions provided in an embodiment of the present invention under different QoS thresholds.

[0103] Figure 5 A schematic diagram of the average QoS completion rate of different solutions under different QoS thresholds provided for the examples of the present invention. DETAILED DESCRIPTION

[0104] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0105] The present invention studies the joint trajectory and phase design problem in energy-saving emergency communications assisted by smart reflective surfaces carried by UAVs. The corresponding non-convex optimization problem is proposed through design, and the optimization problem is solved by a deep reinforcement learning algorithm. It can design the phase of the smart reflective surface while planning the trajectory of the UAV, and realize emergency communication in the UAV-carried smart reflective surface communication network. In the model, the present invention takes into account the motion characteristics of the UAV, the quality of service (QoS) of all ground users, the energy efficiency of the UAV-carried smart reflective surface system, and the need for rapid decision-making. The simulation results of the model show that the method proposed by the present invention can maximize the energy efficiency of the UAV-carried smart reflective surface system while ensuring the quality of service (QoS) of all ground users.

[0106] The present invention is implemented as follows: a method for emergency communication of unmanned aerial vehicles assisted by an intelligent reflective surface, which plans the flight trajectory of the unmanned aerial vehicle and designs the phase of the intelligent reflective surface, so that the intelligent reflective surface carried on the unmanned aerial vehicle can serve as a mobile relay to provide emergency communication services for ground users.

[0107] First, a UAV-equipped intelligent reflective surface assisted emergency network scenario is modeled, which is exemplified by a wildfire scenario. It includes a UAV-equipped intelligent reflective surface system (UAV-RIS) equipped with an active intelligent reflective surface, M ground devices (GDs) for fire detection, and an aerial base station (ABS) responsible for processing information transmitted from the ground devices. Due to the unfavorable conditions of the wildfire environment, the present invention assumes that there is no direct connection between the ground setting and the aerial base station (ABS), and the aerial base station (ABS) is set at a higher position to ensure the line-of-sight connection through the cascade link of the RIS. The communication channel is considered to be a Rice distribution with a line-of-sight component and a non-line-of-sight component. Each GD has an omnidirectional antenna, and the aerial base station (ABS) is equipped with a linear array of Q antennas. The active RIS consists of N reflective elements. The height of the UAV-equipped intelligent reflective surface system (UAV-RIS) and the aerial base station (ABS) is considered to be a fixed value, and the horizontal speed of the UAV-equipped intelligent reflective surface system (UAV-RIS) is V h , dividing the entire flight process into T time slots.

[0108] Secondly, based on the cascade channel of active RIS-assisted ground equipment communication, the uplink transmission rate of each user is derived, and then the energy consumption of the UAV-RIS in each time slot is derived according to the UAV propulsion energy consumption model. Finally, it is integrated into the optimization problem of the joint trajectory and reflector phase in the UAV-RIS communication network. The optimization goal is to maximize the energy efficiency after a long period of decision-making while ensuring the service quality of individual ground equipment. It is a multivariate non-convex optimization problem, and a Hierarchical Proximal Policy Optimization (H-PPO) algorithm based on deep reinforcement learning algorithm is proposed to solve it. The algorithm adopts the structure of the upper and lower proximal policy optimization algorithm PPO. The upper and lower layers of H-PPO optimize the UAV trajectory and reflector phase control respectively. The optimal movement route is selected for the UAV and the phase of the intelligent reflector is designed at the same time to improve the service quality. The present invention maximizes the energy efficiency of the intelligent reflective surface system carried by the UAV while ensuring the quality of service (QoS) of all ground users. The proposed optimization algorithm has the advantages of high reliability and high real-time performance, and meets the requirements of the emergency network for delay and accuracy.

[0109] An intelligent reflective surface-assisted unmanned aerial vehicle emergency communication method comprises the following steps:

[0110] Step 1: Construct a drone-mounted intelligent reflective surface-assisted energy-saving emergency communication scenario;

[0111] The process of step 1 is as follows:

[0112] The present invention considers a key UAV-equipped intelligent reflective surface assisted energy-saving emergency communication scenario. Taking the wildfire monitoring scenario as an example, the specific scenario includes a UAV, which is equipped with an active reconfigurable intelligent surface (RIS), namely, an UAV-carried intelligent reflective surface system (UAV-RIS), multiple ground devices for fire detection, and an aerial base station (ABS) for processing information transmitted from the ground devices. The aerial base station (ABS) serves as a control center and is responsible for interacting with the UAV-carried intelligent reflective surface system (UAV-RIS) and issuing commands. The ground device set is denoted as g = {G m ,m=1,2,…,M}, where M is the number of ground devices. Due to the adverse conditions of the wildfire environment, it is assumed that there is no direct communication link between the ground devices and the aerial base station (ABS), and the aerial base station (ABS) is located at a higher position to ensure the line-of-sight connection of the cascade link.

[0113] Step 2, modeling the communication process of a UAV-RIS assisting multiple ground devices in the UAV-RIS assisted energy-saving emergency communication scenario constructed in step 1;

[0114] Further, the process of step 2 is as follows:

[0115] Step 2.1, construct a channel model;

[0116] In the channel model, each ground device has an omnidirectional antenna, and the airborne base station (ABS) is equipped with Q antenna arrays. The active reconfigurable smart surface (RIS) consists of N reflective elements. Assume that the reflection coefficient of the nth element of RIS is Among them, φ n ∈[0,2π), β represents an amplification factor greater than 1, and the reflection coefficient matrix of the active reconfigurable smart surface (RIS) is defined as Θ=diag([θ1,θ2,…,θ N ]);

[0117] H m,r represents the channel of the link between the mth ground device and the UAV-RIS, H r,b represents the channel between the UAV-RIS and the airborne base station (ABS). Assuming that the channel gain from the UAV-RIS to the airborne base station (ABS) obeys the Rice distribution, the channel is expressed as:

[0118]

[0119] Among them, κ r,b is the Rice coefficient, l is the path loss at the reference distance D = 1m, d r,b is the distance between the UAV-RIS and the airborne base station (ABS), α r,b is the path loss index of the UAV-RIS-ABS link; the line-of-sight component is the first level, expressed as:

[0120]

[0121] in, is the departure angle (AoD) and arrival angle (AoA) of the link from the UAV-RIS to the airborne base station (ABS), and ω is the antenna spacing;

[0122] Non-line-of-sight component Each element of H follows an independent and identically distributed complex Gaussian distribution with a mean of zero and a variance of 1. Similarly, the channel Hm,r Following a distribution similar to the above, the LoS component of the mth link is It is expressed as:

[0123]

[0124] in, is the departure angle (AoD) of the link from the mth ground device to UAV-RIS. The angle of arrival (AoA) and the angle of departure (AoD) are determined by their relative position relationship;

[0125] Finally, it is assumed that all channels follow block fading, and the global channel information is known at the air base station (ABS) and remains constant in each time slot, but varies from one time slot to another.

[0126] Step 2.2, constructing a transmission model based on the channel model constructed in step 2.1;

[0127] In the uplink transmission model from the ground equipment to the airborne base station (ABS), no in-band interference is considered. The airborne base station (ABS) uses maximum ratio combining (MRC) for each link. The maximum ratio combining (MRC) beamforming matrix is ​​F = [f1, ..., f M ] indicates that, among which, f m represents the unit norm beamforming vector of the mth link; the signal received by the aerial base station (ABS) from the mth ground device is expressed as:

[0128]

[0129] Among them, P m is the transmission power of the mth ground device, s m is the unit energy signal sample associated with the monitoring data; the noise vector n generated at the air base station (ABS) m , denoted by n m =[n1,...,n M ] T ,in

[0130] In addition, unlike passive RIS, active RIS requires additional power and the thermal noise generated at the UAV-RIS cannot be ignored because each element is equipped with an amplifier. i ,in, Therefore, the uplink signal-to-noise ratio (SINR) of the mth link of the air base station (ABS) is expressed as:

[0131]

[0132] Therefore, the rate of the mth link is C m =Blog(1+η m ), where B is the bandwidth.

[0133] Step 3, modeling the UAV propulsion energy consumption, laying the foundation for the subsequent joint optimization problem proposed in the present invention.

[0134] The process of step 3 is as follows:

[0135] The UAV-RIS and the Aerial Base Station (ABS) are available in the horizontal position at time slot t. and It indicates that, The vertical heights of the UAV-RIS and the airborne base station (ABS) are set as h r and h b The fixed height of the UAV carrying intelligent reflective surface system (UAV-RIS) at the next time slot t is determined by the flight distance D t and the azimuth variable ξ t Determined, the horizontal coordinate of the UAV-RIS at time t+1 is expressed as:

[0136] According to the above setting for the horizontal position of the UAV-RIS, the horizontal flight speed of the UAV-RIS at time t is is the maximum horizontal velocity, Δt is the length of each time period; if It means that at time t, the UAV is in a hovering state. According to the horizontal speed of the UAV, the energy consumption of the UAV propulsion in each time period is:

[0137]

[0138] Among them, P0 and P1 are the constant power and induced power of the blade in the hovering state, respectively, and U tip is the tip speed of the rotor blade, Δt is the duration of each time slot, v0 is the average rotor induced speed in hover, d0 and s are the airframe drag ratio and rotor solidity, respectively, and ρ and G represent the air density and rotor disk area, respectively.

[0139] Step 4, combining the UAV-RIS in the UAV-equipped intelligent reflective surface assisted energy-saving emergency communication scenario in step 2 and the UAV propulsion energy consumption model in step 3, the optimization target of the final energy consumption ratio of the present invention is modeled, and the objective function when the model is solved is constructed; laying the foundation for the subsequent use of deep reinforcement learning for model solving in the present invention;

[0140] The process of step 4 is as follows:

[0141] The goal of this invention is to maximize the energy efficiency of the UAV-RIS assisted emergency communication system by optimizing the trajectory of the UAV and the phase of the smart reflector in each service cycle. At the same time, the minimum communication rate requirement of all ground devices in dynamic scenarios must be ensured. The reflection coefficient matrix of the smart reflector at the tth time slot is denoted as Θ t , C t,m is the communication rate of the mth link at the tth time slot obtained in step 2, The energy consumption of the UAV propulsion in the tth time slot obtained in step 3, the objective function of maximizing energy efficiency within a time range of length T can be expressed as:

[0142]

[0143]

[0144]

[0145]

[0146] Among them, constraint (1) represents the transmission rate requirement that each ground device needs to meet, and Υ represents the rate limit value; constraint (2) limits the range in which the phase of the reconfigurable intelligent surface (RIS) can be adjusted; constraint (3) represents the maximum flight speed limit of the UAV-RIS.

[0147] Step 5: Build a deep reinforcement learning algorithm model based on the optimization goal proposed in step 4; lay a theoretical foundation for the practical problems to be solved and reduce the difficulty of solving the optimization problem;

[0148] The process of step 5 is as follows:

[0149] In order to solve the challenge of UAV-RIS flight trajectory and RIS phase optimization, the H-PPO algorithm is proposed. The deep reinforcement learning algorithm model is constructed by using the H-PPO algorithm to control the flight trajectory of the UAV-RIS and the phase of the reconfigurable intelligent surface (RIS). The H-PPO algorithm includes two independent proximal policy optimization algorithms PPO; the first proximal policy optimization algorithm PPO is used to optimize the flight action for trajectory control; the second proximal policy optimization algorithm PPO is used for the reconfigurable intelligent surface (RIS) phase value to enhance the channel gain; since the state information required for the flight trajectory and RIS phase control is unrelated, the independent proximal policy optimization algorithm PPO is used to optimize the control of the flight trajectory and the reconfigurable intelligent surface (RIS) phase;

[0150] The first proximal policy optimization algorithm PPO collects data containing flight status, actions and rewards, and improves its strategy to pursue the best trajectory; the second proximal policy optimization algorithm PPO specifically optimizes the reconfigurable intelligent surface (RIS) phase, using channel status, phase actions and reward functions to continuously update its strategy and improve performance; through decoupled optimization, the state information of each action is effectively utilized to avoid the instability caused by interaction. This method aims to simplify coordination and optimization challenges and improve the overall performance of the UAV-RIS system.

[0151] The Proximal Policy Optimization (PPO) algorithm is a deep reinforcement learning (DRL) optimization algorithm. The Proximal Policy Optimization (PPO) algorithm learns the optimal strategy by iteratively updating the strategy to maximize the expected cumulative reward. The strategy of the Proximal Policy Optimization (PPO) algorithm is parameterized and represented by two networks, namely the actor network π θA and critic network V θC (s), actor network π θA The role of is to output the predicted action based on different state information, and the role of the value network is to judge the quality of the current state, and then judge the quality of the decision made by the actor network. A and θ C Represents the corresponding parameter instance; the core idea of ​​the proximal policy optimization algorithm PPO is to maximize the objective function and use the shearing mechanism to constrain the policy update; the objective function in the proximal policy optimization algorithm PPO is:

[0152]

[0153] in, Representation strategy With the old strategy The importance sampling weights between ; for The corresponding advantage function, where γ is the discount factor; the clipping mechanism limits the policy update to a predefined range, where clip() represents the clipping function, and λ is a hyperparameter used to adjust ρ t (θ A )) is controlled in the interval [1-ε,1+ε];

[0154] During the training phase, the network is updated using I batches of experience interacting with the environment; for the actor network The parameter update method is:

[0155]

[0156] At the same time, in order to update θ C , using the mean square error function as the loss function, expressed as:

[0157]

[0158] Among them, α C represents the learning rate, represents the target state value function derived by time difference.

[0159] Step 6: construct a deep reinforcement learning training model based on the deep reinforcement learning algorithm model of step 5, combine the UAV equipped with intelligent reflective surface to assist energy-saving emergency communication scenario in step 1 and the objective function of step 4, set the state space, action space and reward function of the deep reinforcement learning training model, and lay the foundation for obtaining the decision model later;

[0160] It is necessary to control the flight trajectory and RIS phase of the UAV-RIS at the same time. According to the deep reinforcement learning algorithm model of step 5, a deep reinforcement learning training model is constructed. The state space of the PPO that optimizes the flight trajectory and RIS phase control in the H-PPO deep reinforcement learning training model is set respectively. Combined with the UAV-RIS assisted energy-saving emergency communication scenario in step 1, the action space of the two PPOs in the H-PPO deep reinforcement learning training model is set. Combined with the objective function of step 4 and constraint (1), the reward functions of the two PPOs in the H-PPO deep reinforcement learning training model are set respectively.

[0161] The settings of the state space, action space and reward function in step 6 are as follows:

[0162] 1) State space: At time slot t, the state of the first proximal policy optimization algorithm PPO contains global position information, expressed as represents the horizontal coordinate of the mth ground device at time slot t; in addition, the state of the second proximal policy optimization algorithm PPO contains the global channel information, which is expressed as Since the neural network receives real numbers as input, before inputting this complex channel state into the network, it is necessary to separate the real and imaginary parts of the channel and input them into the neural network at the same time;

[0163] 2) Action space: At time slot t, the first action of the proximal policy optimization algorithm PPO (upper layer) is the flight distance and flight direction of the UAV-RIS, expressed as The second proximal policy optimization algorithm PPO (lower layer) takes the phase value of active RIS at time slot t, denoted as Θ t ;

[0164] 3) Reward function: The reward structure of the first proximal policy optimization algorithm PPO mainly consists of two parts; the first is the real-time reward, which includes the ratio of the communication volume to the energy consumption in the current time slot and the difference between the horizontal distance between the UAV-RIS and all ground equipment and the previous time slot, expressed as Among them, P t,m represents the additional positive reward value of the mth ground device when the rate threshold is met in time slot t, Secondly, a termination reward is set that reflects the energy efficiency of the entire service cycle, expressed as λ represents a positive fixed value, which is used to adjust the ratio of the termination reward value to the final reward; in addition, the reward function of the first proximal policy optimization algorithm PPO is expressed as:

[0165]

[0166] Finally, at time slot t, the reward function of the second proximal policy optimization algorithm PPO is expressed as:

[0167]

[0168] Step 7, training the deep reinforcement learning training model obtained in step 6, obtaining a decision model for energy-saving emergency communication assisted by an intelligent reflective surface mounted on a UAV, and obtaining the optimal solution to the optimization problem.

[0169] The process of step 7 is as follows:

[0170] Step 7.1, Initialize the network

[0171] Randomly initialize the actor network θ of each proximal policy optimization algorithm PPO A and the critic network θ C Parameters;

[0172] Step 7.2: Train the deep reinforcement learning model

[0173] Randomly initialize the position of each ground device, the position of the UAV-RIS, and the position of the aerial base station (ABS);

[0174] At each time slot t, the first proximal policy optimization algorithm PPO interacts with the dynamic environment and obtains the state Based on the current state, the first proximal policy optimization algorithm PPO obtains the drone flight action from its Actor network To control the trajectory of the drone;

[0175] The second proximal policy optimization algorithm PPO interacts with the dynamic environment to obtain the state Based on the current state, the second proximal policy optimization algorithm PPO obtains the action of the RIS phase from its Actor network. To control the intelligent reflective surface; finally take action Interact with the environment and calculate the reward r obtained from the environment t ;

[0176] The status state action action and reward r t Stored in the experience pool;

[0177] When the experience in the experience pool accumulates to a certain amount, N d The parameters of the Critic and Actor networks are updated with samples of different sizes. After one training, the experience pool is cleared and the above process is repeated until the model training converges. The decision-making model of energy-saving emergency communication assisted by intelligent reflective surfaces on drones is obtained.

[0178] Step 7.3, Decision-making Phase

[0179] The decision model of UAV-equipped intelligent reflective surface-assisted energy-saving emergency communication is used in random dynamic UAV-equipped intelligent reflective surface-assisted energy-saving emergency communication. The optimal reconfigurable smart surface (RIS) reflection coefficient matrix and UAV flight action are decided in each time slot, so that the energy consumption ratio of the communication system in the whole process is minimized, while ensuring the communication quality of all individual ground users.

[0180] The key points and protection points of the present invention are:

[0181] 1. Implement the idea of ​​using drones equipped with intelligent reflective surfaces to assist energy-saving emergency communications.

[0182] 2. The optimization problem of jointly designing trajectory and RIS phase is proposed in the energy-saving emergency communication network assisted by UAV-mounted intelligent reflective surfaces.

[0183] 3. Hierarchical Proximal Policy Optimization (H-PPO) algorithm based on deep reinforcement learning algorithm.

[0184] In the scenario where drones carry intelligent reflectors to assist in communication, there is currently research on joint optimization of drone trajectory planning and RIS phase to maximize the total capacity of the target user link, but the quality of service (QOS) of a single user is not considered. In addition, in a multi-user network, solving the problem of competition between users for RIS resources is crucial to ensuring excellent service for a single user, which is a problem that many studies have overlooked. The use of DRL to jointly design drone trajectories and RIS phases simultaneously solves the problem of optimizing energy efficiency and ensuring the quality of service for a single user in scenarios that require long decision sequences. Therefore, there is currently no other alternative that can fully achieve the purpose of the present invention.

[0185] like Figure 2 As shown, it is a schematic diagram of a scenario of an energy-saving emergency communication network assisted by a UAV equipped with an intelligent reflective surface. The UAV equipped with an intelligent reflective surface provides uplink communication services for ground equipment.

[0186] As shown in Figure 3, the convergence of the H-PPO algorithm is demonstrated. Figure 3(a) and Figure 3(b) respectively show the changing trends of the reward values ​​of the two PPOs in the H-PPO algorithm as the training progresses. Figure 3(c) and Figure 3(d) show the changes in the energy consumption ratio and QOS completion rate of the system during the training process. It can be seen from Figure 3 that as the training progresses, the energy efficiency performance of the system and the individual service quality of all ground equipment have been significantly improved, indicating that after sufficient training, it has strong adaptability in the face of high complexity and high dynamic scenarios.

[0187] like Figure 4 As shown in Figure 2, the system energy consumption ratio under different schemes and different rate threshold requirements is shown. Figure 5 As shown, the completion rate of QOS of ground equipment under different rate threshold requirements under different schemes is demonstrated. By analyzing the data in the above two figures, it can be clearly seen that although the H-PPO scheme of the present invention lags slightly behind the DT-PPO scheme that uses heuristic control of UAV trajectory in terms of QOS completion rate, the H-PPO scheme of the present invention is a better choice when considering energy efficiency. The H-PPO scheme of the present invention innovatively coordinates the phase of the UAV trajectory and the reconfigurable intelligent reflective surface RIS, significantly optimizes energy efficiency, ensures the service quality of individual devices, and effectively solves the problem of UAVs equipped with intelligent reflective surfaces while solving the problem of optimizing energy efficiency and competition between users for reconfigurable intelligent surface (RIS) resources in scenarios that require long decision sequences. This is true even for cascaded channel information associated with time-varying conditions.

[0188] The present invention also provides an intelligent reflective surface-assisted unmanned aerial vehicle emergency communication system, comprising:

[0189] The UAV emergency communication scenario construction module is used to implement the construction of the UAV equipped with intelligent reflective surface assisted energy-saving emergency communication scenario in step 1;

[0190] A communication process establishment module is used to implement the modeling of the communication process of the UAV-RIS assisting multiple ground devices in the UAV-RIS assisted energy-saving emergency communication scenario constructed in step 1 in step 2;

[0191] The UAV propulsion energy consumption establishment module is used to implement the modeling of the UAV propulsion energy consumption in step 3;

[0192] The optimization target and objective function construction module is used to implement the communication process of the UAV-RIS assisting multiple ground devices in the UAV-RIS assisted energy-saving emergency communication scenario in step 2 and the UAV propulsion energy consumption model in step 3, model the optimization target of the final energy consumption ratio, and construct the objective function when solving the model;

[0193] A deep reinforcement learning algorithm model construction module is used to implement the deep reinforcement learning algorithm model constructed in step 5 according to the optimization target proposed in step 4;

[0194] The state space, action space and reward function setting module is used to implement the construction of the deep reinforcement learning training model according to the deep reinforcement learning algorithm model in step 5 in step 6, and to set the state space, action space and reward function of the deep reinforcement learning training model in combination with the UAV equipped with intelligent reflective surface assisted energy-saving emergency communication scenario in step 1 and the objective function of step 4;

[0195] The deep reinforcement learning algorithm model training module is used to implement the training of the deep reinforcement learning training model obtained in step 6 in step 7, obtain the energy-saving emergency communication decision model assisted by the intelligent reflective surface of the UAV, and obtain the optimal solution to the optimization problem.

[0196] The present invention also provides an intelligent reflective surface-assisted unmanned aerial vehicle emergency communication device, comprising:

[0197] Memory: a computer program storing the above-mentioned intelligent reflective surface-assisted UAV emergency communication method, which is a computer-readable device;

[0198] Processor: used to implement the intelligent reflective surface-assisted UAV emergency communication method when executing the computer program.

[0199] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the intelligent reflective surface-assisted unmanned aerial vehicle emergency communication method.

Claims

1. A method for emergency communication of unmanned aerial vehicles assisted by intelligent reflective surfaces, characterized in that: The following steps are involved: Step 1: Construct a drone-mounted intelligent reflective surface-assisted energy-saving emergency communication scenario; Step 2, modeling the communication process of a UAV-RIS assisting multiple ground devices in the UAV-RIS assisted energy-saving emergency communication scenario constructed in step 1; Step 3, modeling the UAV propulsion energy consumption; Step 4, combining the UAV-RIS system (UAV-RIS) in the UAV-RIS energy-saving emergency communication scenario in step 2 and the UAV propulsion energy consumption model in step 3, modeling the optimization target of the final energy consumption ratio and constructing the objective function for solving the model; Step 5: construct a deep reinforcement learning algorithm model according to the optimization goal proposed in step 4; Step 6: construct a deep reinforcement learning training model based on the deep reinforcement learning algorithm model of step 5, and set the state space, action space and reward function of the deep reinforcement learning training model in combination with the UAV equipped with intelligent reflective surface assisted energy-saving emergency communication scenario of step 1 and the objective function of step 4; Step 7, training the deep reinforcement learning training model obtained in step 6, obtaining a decision model for energy-saving emergency communication assisted by an intelligent reflective surface mounted on a UAV, and obtaining the optimal solution to the optimization problem.

2. The method for emergency communication of unmanned aerial vehicles assisted by intelligent reflective surfaces according to claim 1, characterized in that: The process of step 1 is as follows: The energy-saving emergency communication scenario assisted by a UAV-carried intelligent reflective surface includes a UAV, which is equipped with an active reconfigurable intelligent surface (RIS), namely, a UAV-carried intelligent reflective surface system (UAV-RIS), and an aerial base station (ABS) for processing information transmitted from ground equipment. The aerial base station (ABS) serves as a control center and is responsible for interacting with the UAV-carried intelligent reflective surface system (UAV-RIS) and issuing commands. The UAV-carried intelligent reflective surface system (UAV-RIS) assists the uplink transmission between multiple ground equipment and the aerial base station (ABS). The set of ground equipment is denoted as g = {G m ,m=1,2,…,M}, where M is the number of ground devices. It is assumed that there is no direct communication link between the ground devices and the aerial base station (ABS), and the aerial base station (ABS) is located at a higher position.

3. The method for emergency communication of unmanned aerial vehicles assisted by intelligent reflective surfaces according to claim 1, characterized in that: The process of step 2 is as follows: Step 2.1, construct a channel model; In the channel model, each ground device has an omnidirectional antenna, and the airborne base station (ABS) is equipped with Q antenna arrays. The active reconfigurable smart surface (RIS) consists of N reflective elements. Assume that the reflection coefficient of the nth element of RIS is Among them, φ n ∈[0,2π), β represents an amplification factor greater than 1, and the reflection coefficient matrix of the active reconfigurable smart surface (RIS) is defined as Θ=diag([θ1,θ2,…,θ N ]); H m,r represents the channel of the link between the mth ground device and the UAV-RIS, H r,b represents the channel between the UAV-RIS and the airborne base station (ABS). Assuming that the channel gain from the UAV-RIS to the airborne base station (ABS) obeys the Rice distribution, the channel is expressed as: Among them, κ r,b is the Rice coefficient, l is the path loss at the reference distance D = 1m, d r,b is the distance between the UAV-RIS and the aerial base station (ABS), α r,b is the path loss index of the UAV-RIS-ABS link; the line-of-sight component is the first level, expressed as: in, is the departure angle (AoD) and arrival angle (AoA) of the link from the UAV-RIS to the airborne base station (ABS), and ω is the antenna spacing; Non-line-of-sight component Each element of H follows an independent and identically distributed complex Gaussian distribution with a mean of zero and a variance of 1. Similarly, the channel H m,r Following a distribution similar to the above, the LoS component of the mth link is It is expressed as: in, is the departure angle (AoD) of the link from the mth ground device to UAV-RIS. The angle of arrival (AoA) and the angle of departure (AoD) are determined by their relative position relationship; Finally, it is assumed that all channels follow block fading, and the global channel information is known at the air base station (ABS) and remains constant in each time slot, but varies from one time slot to another; Step 2.2, constructing a transmission model based on the channel model constructed in step 2.1; In the uplink transmission model from the ground equipment to the airborne base station (ABS), no in-band interference is considered. The airborne base station (ABS) uses maximum ratio combining (MRC) for each link. The maximum ratio combining (MRC) beamforming matrix is ​​F = [f1, ..., f M ] indicates that, among which, f m represents the unit norm beamforming vector of the mth link; the signal received by the aerial base station (ABS) from the mth ground device is expressed as: Among them, P m is the transmission power of the mth ground device, s m is the unit energy signal sample associated with the monitoring data; the noise vector n generated at the air base station (ABS) m , denoted by n m =[n1,...,n M ] T ,in Active reconfigurable smart surfaces (RIS) require additional power and each element is equipped with an amplifier, taking into account the thermal noise generated at the UAV-RIS. i ,in, The uplink signal-to-noise ratio (SINR) of the mth link of the air base station (ABS) is expressed as: Therefore, the rate of the mth link is C m =Blog(1+η m ), where B is the bandwidth.

4. The method for emergency communication of unmanned aerial vehicles assisted by intelligent reflective surfaces according to claim 1, characterized in that: The process of step 3 is as follows: The UAV-RIS and the airborne base station (ABS) are used at the horizontal position in time slot t. and It indicates that, The vertical heights of the UAV-RIS and the airborne base station (ABS) are set as h r and h b The fixed height of the UAV carrying intelligent reflective surface system (UAV-RIS) at the next time slot t is determined by the flight distance D t and the azimuth variable ξ t Determined, the horizontal coordinate of the UAV-RIS at time t+1 is expressed as: According to the setting of the horizontal position of the UAV-RIS, the horizontal flight speed of the UAV-RIS at time t is is the maximum horizontal velocity, Δt is the length of each time period; if It means that at time t, the UAV is in a hovering state. According to the horizontal speed of the UAV, the energy consumption of the UAV propulsion in each time period is: Among them, P0 and P1 are the constant power and induced power of the blade in the hovering state, respectively, and U tip is the tip speed of the rotor blade, Δt is the duration of each time slot, v0 is the average rotor induced speed in hover, d0 and s are the airframe drag ratio and rotor solidity, respectively, and ρ and G represent the air density and rotor disk area, respectively; The process of step 4 is as follows: The reflection coefficient matrix of the smart reflector at the tth time slot is denoted as Θ t , C t,m is the communication rate of the mth link at the tth time slot obtained in step 2, The energy consumption of the UAV propulsion in the t-th time slot obtained in step 3, the objective function of maximizing energy efficiency within a time range of length T is expressed as: Among them, constraint (1) represents the transmission rate requirement that each ground device needs to meet, and Υ represents the rate limit value; constraint (2) limits the range in which the RIS phase can be adjusted; constraint (3) represents the maximum flight speed limit of the UAV-RIS.

5. The method for emergency communication of unmanned aerial vehicles assisted by intelligent reflective surfaces according to claim 1, characterized in that: The process of step 5 is as follows: The deep reinforcement learning algorithm model is constructed by using the H-PPO algorithm to control the flight trajectory and RIS phase of the UAV-RIS. The H-PPO algorithm includes two independent proximal policy optimization algorithms PPO; the first proximal policy optimization algorithm PPO is used to optimize the flight action for trajectory control; the second proximal policy optimization algorithm PPO is used for RIS phase value to enhance channel gain; the independent proximal policy optimization algorithms PPO are used to optimize the control of flight trajectory and RIS phase; The Proximal Policy Optimization (PPO) algorithm is a deep reinforcement learning (DRL) optimization algorithm. The Proximal Policy Optimization (PPO) algorithm learns the optimal strategy by iteratively updating the strategy to maximize the expected cumulative reward. The policy of the proximal policy optimization algorithm PPO is parameterized and represented by two networks, the actor network and critic network Among them, θ A and θ C Represents the corresponding parameter instance; the proximal policy optimization algorithm PPO uses a proxy objective function and a clipping mechanism to constrain policy updates; the objective function in the proximal policy optimization algorithm PPO is: in, Representation strategy With the old strategy The importance sampling weights between for The corresponding advantage function, where γ is the discount factor; the clipping mechanism limits the policy update to a predefined range, where clip() represents the clipping function, and λ is a hyperparameter used to adjust ρ t (θ A )) is controlled in the interval [1-ε,1+ε]; During the training phase, the network is updated using I batches of experience interacting with the environment; for the actor network The parameter update method is: At the same time, in order to update θ C , using the mean square error function as the loss function, expressed as: Among them, α C represents the learning rate, represents the target state value function derived by time difference.

6. The method for emergency communication of unmanned aerial vehicles assisted by intelligent reflective surfaces according to claim 1, characterized in that: The settings of the state space, action space and reward function in step 6 are as follows: 1) State space: At time slot t, the state of the first proximal policy optimization algorithm PPO contains global position information, expressed as represents the horizontal coordinate of the mth ground device at time slot t; in addition, the state of the second proximal policy optimization algorithm PPO contains the global channel information, which is expressed as Before inputting this complex channel state into the network, the real and imaginary parts of the channel are separated and input into the neural network at the same time; 2) Action space: At time slot t, the first action of the proximal policy optimization algorithm PPO (upper layer) is the flight distance and flight direction of the UAV-RIS, expressed as The second proximal policy optimization algorithm PPO (lower layer) takes the phase value of active RIS at time slot t, denoted as Θ t ; 3) Reward function: The reward structure of the first proximal policy optimization algorithm PPO consists of two parts; the first is the real-time reward, which includes the ratio of the communication volume to the energy consumption in the current time slot and the difference between the horizontal distance between the UAV-RIS and all ground equipment and the previous time slot, expressed as Among them, P t,m represents the additional positive reward value of the mth ground device when the rate threshold is met in time slot t, Secondly, a termination reward is set that reflects the energy efficiency of the entire service cycle, expressed as λ represents a positive fixed value, which is used to adjust the ratio of the termination reward value to the final reward; in addition, the reward function of the first proximal policy optimization algorithm PPO is expressed as: Finally, at time slot t, the reward function of the second proximal policy optimization algorithm PPO is expressed as:

7. The method for emergency communication of unmanned aerial vehicles assisted by intelligent reflective surfaces according to claim 1, characterized in that: The process of step 7 is as follows: Step 7.1, Initialize the network Randomly initialize the actor network θ of each proximal policy optimization algorithm PPO A and the critic network θ C Parameters; Step 7.2: Train the deep reinforcement learning model Randomly initialize the position of each ground device, the position of the UAV-RIS, and the position of the aerial base station (ABS); At each time slot t, the first proximal policy optimization algorithm PPO interacts with the dynamic environment and obtains the state Based on the current state, the first proximal policy optimization algorithm PPO obtains the drone flight action from its Actor network To control the trajectory of the drone; The second proximal policy optimization algorithm PPO interacts with the dynamic environment to obtain the state Based on the current state, the second proximal policy optimization algorithm PPO obtains the action of the RIS phase from its Actor network. To control the intelligent reflective surface; finally take action Interact with the environment and calculate the reward r obtained from the environment t ; The status state action action and reward r t Stored in the experience pool; When the experience in the experience pool accumulates to a certain amount, N d The parameters of the Critic and Actor networks are updated with samples of different sizes. After one training, the experience pool is cleared and the above process is repeated until the model training converges. The decision-making model of energy-saving emergency communication assisted by intelligent reflective surfaces on drones is obtained. Step 7.3, Decision-making Phase The decision model of UAV-equipped intelligent reflective surface-assisted energy-saving emergency communication is used in random dynamic UAV-equipped intelligent reflective surface-assisted energy-saving emergency communication. The optimal reconfigurable smart surface (RIS) reflection coefficient matrix and UAV flight action are decided in each time slot, so that the energy consumption ratio of the communication system in the whole process is minimized, while ensuring the communication quality of all individual ground users.

8. An intelligent reflective surface-assisted drone emergency communication system based on the method according to any one of claims 1 to 7, characterized in that: include: The UAV emergency communication scenario construction module is used to build a UAV equipped with intelligent reflective surfaces to assist energy-saving emergency communication scenarios; The communication process establishment module is used to model the communication process of the UAV-RIS assisting multiple ground devices in the UAV-RIS assisted energy-saving emergency communication scenario; UAV propulsion energy consumption establishment module, used to model the UAV propulsion energy consumption; The optimization target and objective function construction module is used to combine the UAV-RIS system in the UAV-RIS scenario to assist the communication process of multiple ground devices and the UAV propulsion energy consumption model in the energy-saving emergency communication scenario, model the optimization target of the final energy consumption ratio, and construct the objective function when solving the model; A deep reinforcement learning algorithm model building module is used to build a deep reinforcement learning algorithm model according to the proposed optimization objectives; The state space, action space and reward function setting module is used to build a deep reinforcement learning training model according to the deep reinforcement learning algorithm model, and to set the state space, action space and reward function of the deep reinforcement learning training model in combination with the UAV equipped with intelligent reflective surface assisted energy-saving emergency communication scenario and objective function; The deep reinforcement learning algorithm model training module is used to train the deep reinforcement learning training model, obtain the drone-equipped intelligent reflective surface-assisted energy-saving emergency communication decision-making model, and obtain the optimal solution to the optimization problem.

9. An intelligent reflective surface-assisted UAV emergency communication device, characterized in that: include: Memory: a computer program storing the method for emergency communication of unmanned aerial vehicles assisted by an intelligent reflective surface as described in any one of claims 1 to 7, which is a computer-readable device; Processor: used to implement the intelligent reflective surface-assisted drone emergency communication method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement the intelligent reflective surface-assisted drone emergency communication method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Unmanned aerial vehicle auxiliary ground communication method based on trajectory and phase joint optimization

    CN114980169A

  • Unmanned aerial vehicle wireless sensor network data collection method assisted by multiple intelligent reflecting surfaces

    CN116709238A

  • Internet of vehicles power distribution and user scheduling method, system and device assisted by intelligent reflecting surface relay, and medium

    CN117460034A

Cited By

  • Unmanned aerial vehicle enhanced emergency communication and rescue system

    CN119652392A

  • Intelligent metasurface assisted unmanned aerial vehicle communication method, system and device based on deep reinforcement learning, and storage medium

    CN119743216A

  • Anti-interference unmanned aerial vehicle-IOS auxiliary wireless communication energy collection method

    CN119743782A

  • Anti-interference UAV-IOS assisted wireless communication energy harvesting method

    CN119743782B

  • Communication network optimization method for air RIS attitude change

    CN120018159A