High-speed dynamic communication multi-objective optimization method based on unmanned aerial vehicle cooperative beam forming
Through the multi-objective optimization model of collaborative beamforming of drone virtual antenna arrays, long-term short-term memory networks and reinforcement learning algorithms, the problems of low transmission rates and high energy consumption caused by strong user mobility in wireless networks are solved, and high speed is achieved The effect of dynamic communication.
Patent Information
- Application Number
- CN202411022416.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-29
AI Technical Summary
In wireless networks, it is difficult to achieve high-speed dynamic communication when using drones as air base stations, especially in scenarios where user mobility is strong, the transmission rate is low and the energy consumption of drones is high, and the existing technology has not effectively solved it. Time-varying channel and interference issues.
By establishing a multi-objective optimization model for collaborative beamforming of drone virtual antenna arrays, using long and short-term memory networks and reinforcement learning algorithms, the optimal mobile position and optimal excitation current weight of the drone are determined in real time, and the transmission rate and energy consumption are optimized. , realize high-speed dynamic communication.
It improves the transmission rate of mobile users, reduces the energy consumption of drones, enhances the real-time response and anti-interference capabilities of the communication system, and realizes high-speed dynamic communication of wireless networks.
Smart Images

Figure CN119071815A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of wireless network communications, and in particular relates to a high-speed dynamic communication multi-objective optimization method based on unmanned aerial vehicle collaborative beamforming. Background Art
[0002] In the field of wireless networks, drones can be used as aerial base stations and aerial relays to assist ground networks and provide reliable and efficient wireless communication services to ground users. In the scenario where drones are used as aerial base stations, drones located in a specific area need to communicate with remote mobile users. The communication between drones and remote mobile users can be established through drone swarms, and then single-hop or multi-hop methods are used to achieve communication. However, this method may cause link failures and bring additional energy consumption. At the same time, due to time-varying channel conditions and interference from other base stations, it is difficult for remote mobile users to obtain satisfactory communication rates. In addition, the current work on drone cooperative beamforming only considers drones serving static ground terminals, while ignoring the mobility of users, and cannot achieve satisfactory results when serving mobile users. The high dynamics caused by the random movement of users poses a challenge to the real-time response capability of the communication system. Summary of the invention
[0003] The purpose of the present invention is to provide a multi-objective optimization method for high-speed dynamic communication based on cooperative beamforming of unmanned aerial vehicles. By establishing a multi-objective joint optimization model for improving the transmission rate of receiving mobile users and reducing the mobile energy consumption of unmanned aerial vehicles in the cooperative beamforming of virtual antenna arrays of unmanned aerial vehicles, the optimal mobile position and optimal excitation current weight of the unmanned aerial vehicle can be output in real time according to the position of the mobile user, thereby realizing high-speed dynamic communication of wireless network communication.
[0004] The technical solution provided by the present invention is:
[0005] The multi-objective optimization method for high-speed dynamic communication based on UAV cooperative beamforming includes the following steps:
[0006] Step 1: Establish a calculation model for the sum of the transmission rates of the UAV virtual antenna array during the entire communication time as the first objective function, and establish a calculation model for the sum of the UAV energy consumption as the second objective function; and
[0007] In each time slot, the location of the mobile user in the communication system is obtained;
[0008] Step 2: Taking the maximum of the first objective function and the minimum of the second objective function as the optimization objectives, determine the optimal position of the UAV and the optimal excitation current weight of the UAV;
[0009] Step 3: The UAV moves to the optimal position and transmits communication data to the user according to the optimal excitation current weight.
[0010] Preferably, the calculation model of the sum of the transmission rates of the UAV virtual antenna array during the entire communication time is:
[0011]
[0012] In the formula, Indicates the horizontal movement direction of the drone. Indicates the horizontal movement distance of the drone. Indicates the vertical movement distance of the drone. Represents the excitation current weight of the UAV; R UM [t] represents the transmission rate when the UAV virtual antenna array communicates with the mobile user in time slot t; UM [t] represents the signal to interference plus noise ratio when the UAV virtual antenna array communicates with the mobile user in time slot t.
[0013] Preferably, the signal to interference plus noise ratio when the drone virtual antenna array communicates with the mobile user in the t time slot is calculated by the following formula:
[0014]
[0015] Among them, P U [t] represents the total transmission power of the UAV virtual antenna array in time slot t, G UM [t] represents the antenna gain of the UAV virtual antenna array toward the mobile user position at time slot t, g UM [t] represents the channel power gain between the UAV virtual antenna array and the mobile user in time slot t, σ 2 represents the noise power, P B [t] represents the transmission power of the base station in time slot t, g BM [t] represents the channel power gain between the unrelated base station and mobile user in time slot t.
[0016] Preferably, the calculation model of the total energy consumption of the drone is:
[0017]
[0018] In the formula, E i [t] represents the energy consumption of UAV i in time slot t, T represents the end time of flight, v i [t] represents the speed of drone i in time slot t, m UAV represents the mass of the drone, g represents the acceleration due to gravity, and h represents the i [0] and v i [0] represents the initial height and initial speed of UAV i, hi [T] and v i [T] represents the height and speed of UAV i at time T, P(v i [t]) represents the propulsion energy consumption of the UAV during flight in two-dimensional horizontal space within time slot t; N UAV Indicates the number of drones.
[0019] Preferably, the propulsion energy consumption of the drone when flying in a two-dimensional horizontal space is calculated by the following formula:
[0020]
[0021] Among them, P B and P I are constants, v tip 、v 0 ,d 0 , s, ρ and A represent the rotor tip speed, the average rotor induced speed in hovering, the fuselage drag ratio, the rotor solidity, the air density and the rotor disc area, respectively; v i represents the speed of drone i.
[0022] Preferably, in the step 2, determining the optimal position of the drone and the optimal excitation current weight of the drone based on the long short-term memory network comprises the following steps:
[0023] Step 1: randomly initialize the network parameters of multiple long short-term memory networks and the weight vectors of the first objective function and the second objective function to obtain multiple policy networks and weight vectors corresponding to the policy networks as the initialization population; iteratively update the policy networks in the initialization population based on the reinforcement learning algorithm of the long short-term memory network, and all updated populations and the initialization population constitute the initial generation population;
[0024] Step 2: Using a performance buffer method to select multiple policy networks from the initial population to obtain a first population;
[0025] Step 3: Use the hyperplane method to screen the first population and obtain a policy network uniquely corresponding to each weight vector as the second population;
[0026] Step 4: updating the policy networks in the second population using a reinforcement learning method based on a long short-term memory network, and selecting non-dominated policy networks in the updated results according to the first objective function and the second objective function, wherein the set of non-dominated networks constitutes a Pareto solution set;
[0027] Repeat step 4 until the maximum number of iterations is reached and the optimal Pareto solution set is obtained;
[0028] Step 5: Taking the user's position as an input parameter, the strategy network in the optimal Pareto solution set is used to output the optimal position of the UAV and the optimal excitation current weight of the UAV.
[0029] Preferably, the weight vectors of the first objective function and the second objective function are arrays (a, b);
[0030] Among them, the value conditions of a and b satisfy a∈[0,1], b∈[0,1], and a+b=1.
[0031] The beneficial effects of the present invention are:
[0032] The multi-objective optimization method for high-speed dynamic communication based on cooperative beamforming of unmanned aerial vehicles provided by the present invention can output the optimal mobile position and optimal excitation current weight of the unmanned aerial vehicle in real time according to the position of the mobile user, thereby realizing high-speed dynamic communication of wireless network communication, by establishing a multi-objective joint optimization model for improving the transmission rate of receiving mobile users and reducing the mobile energy consumption of the unmanned aerial vehicle in the cooperative beamforming of the virtual antenna array of the unmanned aerial vehicle.
[0033] The present invention combines the reinforcement learning algorithm with the evolutionary learning method to solve multi-objective optimization problems, and combines the long short-term memory network to enhance the algorithm's learning ability for highly dynamic environments; using the performance buffer method and the hyperplane-based task selection method, the diversity of the optimal Pareto solution set obtained is enhanced, so that decision makers can choose the appropriate strategy network according to different demand preferences. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 Schematic diagram of the UAV virtual antenna array collaborative beam dynamic communication scenario.
[0035] Figure 2 The present invention provides a flowchart of the UAV virtual antenna array collaborative beamforming high-speed dynamic communication method.
[0036] Figure 3 This is a schematic diagram of the performance buffer pool method described in the present invention.
[0037] Figure 4 Schematic diagram of the hyperplane-based strategy selection method described in the present invention. DETAILED DESCRIPTION
[0038] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.
[0039] like Figure 1As shown in the figure, during wireless network communications, a single UAV can only serve a limited area due to the limitations of onboard energy and transmission power; these constraints make it impossible for UAVs to meet the communication needs of remote users. In addition, time-varying channels and interference from other base stations make it difficult to achieve satisfactory communication rates. In addition, current work on UAV cooperative beamforming only focuses on UAVs serving static ground terminals, without considering the mobility of the users served (such as pedestrians, robots, or vehicles), which poses a challenge to the real-time response capability of the communication system.
[0040] like Figure 2 As shown, the present invention provides a high-speed dynamic communication multi-objective optimization method based on cooperative beamforming of unmanned aerial vehicles, and establishes a multi-objective joint optimization model for improving the transmission rate of receiving mobile users and reducing the energy consumption of unmanned aerial vehicles. Then, the improved multi-objective evolutionary reinforcement learning algorithm is used to design the optimal position of the unmanned aerial vehicle movement and the optimal excitation current weight of the unmanned aerial vehicle according to the observed environmental state at each moment. The evolutionary reinforcement learning algorithm improves the diversity and effectiveness of the solution by introducing a hyperplane-based task selection method and a long short-term memory network. The specific implementation process is as follows.
[0041] 1. Design two optimization functions based on the rate and energy consumption requirements of long-term dynamic communication, and construct a multi-objective optimization problem based on multi-objective optimization theory;
[0042] First, the overall objective function is designed as follows:
[0043] maxF={f 1 ,f 2} (1)
[0044]
[0045] in,
[0046] R UM [t] = log 2 (1+Υ UM [t]) (4)
[0047]
[0048] g UM [t] = K 0 d UM [t] -α Ω UM [t] (6)
[0049] g BM [t] = K 0 d BM [t] -α Ω BM [t] (7)
[0050]
[0051]
[0052] In the formula, Indicates the horizontal movement direction of the drone. Indicates the horizontal movement distance of the drone. Indicates the vertical movement distance of the drone. Represents the excitation current weight of the drone.
[0053] The first objective function f 1 Represents the sum of the transmission rates of the UAV virtual antenna array during the entire communication time. UM [t] represents the transmission rate when the UAV virtual antenna array communicates with the mobile user in time slot t, Υ UM [t] represents the signal to interference plus noise ratio when the drone virtual antenna array communicates with the mobile user in time slot t, P U [t] represents the total transmission power of the UAV virtual antenna array in time slot t, g UM [t] represents the channel power gain between the UAV virtual antenna array and the mobile user in time slot t, K 0 represents the path loss constant between the UAV virtual antenna array and the mobile user, d UM [t] represents the distance between the center of the UAV virtual antenna array and the mobile user in time slot t, α represents the path loss index, and Ω UM [t] represents the small-scale fading of time slot t, which is simulated by the Rice distribution; g BM [t] represents the channel power gain between the base station and the mobile user that are not related in time slot t, P B [t] represents the transmission power of the base station in time slot t; σ 2 Represents the noise power; G UM [t] represents the antenna gain of the UAV virtual antenna array toward the mobile user position at time slot t, where (θ UM [t],φ UM [t]) represents the direction of the mobile user, θ UM [t] represents the elevation angle of the mobile user in time slot t, φ UM [t] represents the azimuth of the mobile user in time slot t, ω(θ,φ) represents the amplitude of the far-field beam pattern of each unit of the drone, which is usually 1, η∈[0,1] represents the antenna array efficiency; AF is the array factor, represents the position of the i-th UAV in time slot t, I i [t] represents the excitation current weight of the i-th UAV in time slot t, k c =2π / λ represents the phase constant, λ represents the wavelength;
[0054] The second objective function f 2 It represents the total energy consumption of the UAV during the communication process. i [t] represents the energy consumption of UAV i in time slot t, T represents the end time of flight, v i [t] represents the speed of drone i in time slot t, m UAV represents the mass of the drone, g represents the acceleration of gravity, and h[0] represents the initial height of the drone; P(v i [t]) represents the propulsion energy consumption of the UAV when flying in two-dimensional horizontal space, P B and P I Represent two constants, v tip , v 0 , d 0 , s, ρ, and A represent the rotor tip speed, the average rotor induced speed in hovering, the fuselage drag ratio, the rotor solidity, the air density, and the rotor disc area, respectively.
[0055] Based on the above objective function, the evolutionary reinforcement learning algorithm is used to train the optimal strategy network, which can output the optimal position of the UAV movement and the optimal excitation current weight of the UAV according to the environmental state observed at each moment to maximize the above two objectives.
[0056] 2. Initialize the population. Randomly initialize n policy networks and n uniformly distributed weight vectors (one weight vector for each policy network), and use a reinforcement learning algorithm based on long short-term memory networks to optimize these policy networks to generate the first generation population.
[0057] Among them, the weight vector is an array (a, b) representing the weights of the first objective function and the second objective function; the value conditions of a and b satisfy a∈[0,1], b∈[0,1], and a+b=1.
[0058] The specific optimization process of each policy network is as follows:
[0059] (1) The strategy network outputs the action to be performed by the UAV (i.e., the horizontal movement direction, horizontal movement distance, vertical movement distance, and excitation current weight of the UAV) based on the environmental state information, and inputs the action into the environment.
[0060] (2) After the UAV performs an action, the environment returns a reward vector in real time (including the transmission rate of the link between the UAV and the user and the energy consumption of the UAV).
[0061] (3) The reward vector is linearly weighted using the weight vector corresponding to the policy network to obtain a weighted reward value, and the reinforcement learning algorithm based on the long short-term memory network updates the policy network parameters according to the reward value.
[0062] (4) Repeat steps (1) to (3) n iter times, and save the strategy network generated by each update into the population to form a new population.
[0063] All updated populations and the initialized population constitute the initial population. Since the number of policy networks in the initial population is greater than the number of policy networks in the initialized population, one weight vector in the initial population will correspond to multiple policy networks.
[0064] Third, use the performance buffer method to update the population to maintain the diversity and quality of the offspring population. Figure 3 As shown, f 1 and f 2 All possible values constitute the performance space, and the policy network in the population is mapped to the performance space, and the population is saved through the performance buffer. The specific process is as follows:
[0065] (1) The performance space is divided into B num buffers, each of which can store up to B size A strategic network.
[0066] (2) The objective function value corresponding to the policy network is calculated based on the two objective functions established previously. The position of the policy network in the performance space is determined by the values of the two objective functions.
[0067] (3) Each policy network is stored in the nearest buffer in the performance space.
[0068] (4) If the number of policy networks in the buffer exceeds B size , only keep B with the largest distance from the origin size A strategic network.
[0069] (5) A new population (the first population) is formed by all the strategy networks stored in the performance buffer pool.
[0070] Fourth, use the hyperplane-based strategy selection method to select the strategy network to be updated in the population (the second population), such as Figure 4 As shown, in the performance space, with each policy network as the center, r hyper A hyperplane is established for the radius. According to the number of policy networks contained in each hyperplane, the probability of the policy network corresponding to the plane being selected for update is determined (that is, the denser the policy network in the hyperplane, the lower the probability of being selected, and vice versa). The specific steps are as follows:
[0071] (1) Calculate the objective function value corresponding to the policy network based on the two objective functions in step 1, and determine the position of the policy network in the performance space based on the objective function value.
[0072] (2) Calculate the number of policy networks contained in each hyperplane (given a hyperplane, if the Euclidean distance between a policy network and the center of the hyperplane is less than r hyper , then the hyperplane contains the policy network).
[0073] (3) For each weight vector obtained by initialization, a unique corresponding policy network is selected for training. The probability of the policy network being selected is as follows:
[0074]
[0075] Among them, N i It represents the number of policy networks contained in the hyperplane corresponding to the i-th policy network, and c is a constant greater than 1.
[0076] 5. Use the reinforcement learning algorithm based on long short-term memory network to optimize these selected policy networks to generate new offspring populations. The parameters of the policy network are updated according to the following formula:
[0077]
[0078] Among them, J clip is the policy gradient loss function, ∈ is a hyperparameter used to control the interception range, r[t](θ) represents the probability ratio, A ω [t] represents the extended advantage function of time slot t, λ∈[0,1] controls the balance between bias and variance, γ represents the discount factor, V π represents the vectorized value function, J V represents the value function loss, represents the target value function.
[0079] After each update, the objective function value is calculated based on the policy network in the current population and the policy network in the previous generation Pareto set, and the non-dominated policy network is found. The set of non-dominated policy networks obtained in each update is used as the new Pareto set.
[0080] Whether the iteration termination condition reaches the maximum number of iterations; if the iteration termination condition is met, the final Pareto combination is output as the optimal Pareto set, otherwise the iterative update process continues.
[0081] The user selects a policy network from the Pareto set as the final policy network based on the demand preference. For example, if the requirement for energy consumption is higher, the policy network with the lowest energy consumption is selected.
[0082] The drone nodes synchronize communication data and form a virtual antenna array. At the beginning of each time slot, the drone obtains the user's location information and inputs the final strategy network.
[0083] The strategy network outputs the optimal position and optimal excitation current weight of the UAV.
[0084] The above process is repeated in each time slot until the communication ends.
[0085] This paper provides a method based on cooperative beamforming of UAV virtual antenna array to achieve highly reliable dynamic communication. The UAV can move to the optimal position in real time according to the real-time position of the mobile user and adjust the optimal excitation current weight to form a virtual antenna array, thereby forming a beam with high antenna gain and high directivity, and directly communicate with the distant mobile user without multiple hops. The main lobe direction of the beam points to the direction of the mobile user being communicated, thereby achieving a high signal to interference plus noise ratio for the receiving user, and improving the transmission rate and anti-interference ability. In addition, by designing the mobile position of the UAV, a certain trade-off can be achieved between the maximum communication rate and the minimum energy consumption of the UAV. Therefore, the purpose of high-speed dynamic communication of wireless networks can be achieved through a communication method based on cooperative beamforming of UAV virtual antenna array. The present invention combines the reinforcement learning algorithm with the evolutionary learning method to solve the multi-objective optimization problem, and combines the long short-term memory network to enhance the learning ability of the algorithm for highly dynamic environments. In addition, the performance buffer method and the hyperplane-based task selection method are used to enhance the diversity of the final Pareto solution set obtained, providing users with more choices so that users can choose a suitable strategy network according to different situations.
[0086] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.
Claims
1. Multi-objective optimization method for high-speed dynamic communication based on cooperative beamforming of UAVs, It is characterized in that include: Step 1: Establish a calculation model for the sum of the transmission rates of the UAV virtual antenna array during the entire communication time as the first objective function, and establish a calculation model for the sum of the UAV energy consumption as the second objective function; as well as In each time slot, the location of the mobile user in the communication system is obtained; Step 2: Taking the maximum of the first objective function and the minimum of the second objective function as the optimization objectives, determine the optimal position of the UAV and the optimal excitation current weight of the UAV; Step 3: The UAV moves to the optimal position and transmits communication data to the user according to the optimal excitation current weight.
2. According to claim 1, the high-speed dynamic communication multi-objective optimization method based on UAV cooperative beamforming, It is characterized in that The calculation model of the sum of the transmission rates of the UAV virtual antenna array during the entire communication time is: In the formula, Indicates the horizontal movement direction of the drone. Indicates the horizontal movement distance of the drone. Indicates the vertical movement distance of the drone. Represents the excitation current weight of the UAV; R UM [t] represents the transmission rate when the UAV virtual antenna array communicates with the mobile user in time slot t; UM [t] represents the signal to interference plus noise ratio when the UAV virtual antenna array communicates with the mobile user in time slot t.
3. According to claim 2, the high-speed dynamic communication multi-objective optimization method based on UAV cooperative beamforming, It is characterized in that The signal to interference plus noise ratio when the drone virtual antenna array communicates with the mobile user in time slot t is calculated by the following formula: Among them, P U [t] represents the total transmission power of the UAV virtual antenna array in time slot t, G UM [t] represents the antenna gain of the UAV virtual antenna array toward the mobile user position at time slot t, g UM [t] represents the channel power gain between the UAV virtual antenna array and the mobile user in time slot t, σ 2 represents the noise power, P B [t] represents the transmission power of the base station in time slot t, g BM [t] represents the channel power gain between the unrelated base station and mobile user in time slot t.
4. According to claim 3, the high-speed dynamic communication multi-objective optimization method based on UAV cooperative beamforming, It is characterized in that The calculation model of the total energy consumption of the drone is: In the formula, E i [t] represents the energy consumption of UAV i in time slot t, T represents the end time of flight, v i [t] represents the speed of drone i in time slot t, m UAV represents the mass of the drone, g represents the acceleration due to gravity, and h represents the i [0] and v i [0] represents the initial height and initial speed of UAV i, h i [T] and v i [T] represents the height and speed of UAV i at time T, P(v i [t]) represents the propulsion energy consumption of the UAV during flight in two-dimensional horizontal space within time slot t; N UAV Indicates the number of drones.
5. According to claim 4, the high-speed dynamic communication multi-objective optimization method based on UAV cooperative beamforming, It is characterized in that The propulsion energy consumption of a drone flying in a two-dimensional horizontal space is calculated by the following formula: Among them, P B and P I are constants, v tip 、v 0 ,d 0 , s, ρ and A represent the rotor tip speed, the average rotor induced speed in hovering, the fuselage drag ratio, the rotor solidity, the air density and the rotor disc area, respectively; v i represents the speed of drone i.
6. The multi-objective optimization method for high-speed dynamic communication based on cooperative beamforming of unmanned aerial vehicles according to any one of claims 1 to 5, It is characterized in that In the step 2, determining the optimal position of the drone and the optimal excitation current weight of the drone based on the long short-term memory network includes the following steps: Step 1: randomly initialize the network parameters of multiple long short-term memory networks and the weight vectors of the first objective function and the second objective function to obtain multiple policy networks and weight vectors corresponding to the policy networks as the initialization population; iteratively update the policy networks in the initialization population based on the reinforcement learning algorithm of the long short-term memory network, and all updated populations and the initialization population constitute the initial generation population; Step 2: Using a performance buffer method to select multiple policy networks from the initial population to obtain a first population; Step 3: Use the hyperplane method to screen the first population and obtain a policy network uniquely corresponding to each weight vector as the second population; Step 4: updating the policy networks in the second population using a reinforcement learning method based on a long short-term memory network, and selecting non-dominated policy networks in the updated results according to the first objective function and the second objective function, wherein the set of non-dominated networks constitutes a Pareto solution set; Repeat step 4 until the maximum number of iterations is reached and the optimal Pareto solution set is obtained; Step 5: Taking the user's position as an input parameter, the strategy network in the optimal Pareto solution set is used to output the optimal position of the UAV and the optimal excitation current weight of the UAV.
7. The multi-objective optimization method for high-speed dynamic communication based on cooperative beamforming of unmanned aerial vehicles according to claim 6, It is characterized in that The weight vectors of the first objective function and the second objective function are arrays (a, b); Among them, the value conditions of a and b satisfy a∈[0,1], b∈[0,1], and a+b=1.
Citation Information
Patent Citations
IRS-assisted unmanned aerial vehicle communication joint optimization method based on DDPG algorithm
CN113162679A
Unmanned aerial vehicle swarm secure communication method
CN114221694A
Unmanned aerial vehicle self-organizing multi-hop network data transmission method for disaster area
CN115866575A
Marine secret communication method based on unmanned aerial vehicle swarm
CN115942299A
Wireless network weighting and rate maximization method based on UAV intelligent reflecting surface
CN118282449A
