A UAV trajectory planning method based on fusion multiple access
By integrating non-orthogonal multiple access and orthogonal multiple access technologies in drone communication, combined with improved Q-Learning algorithm, the drone track planning is optimized, and the communication rate and coverage in communication cells are solved. The problem of low rate achieves more efficient communication performance.
Patent Information
- Application Number
- CN202311348447.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-18
- Publication Date
- 2025-02-07
- Estimated Expiration
- 2043-10-18
AI Technical Summary
The prior art is difficult to effectively solve the increase in communication load demand caused by multi-user distributed communication in communication cells, and the establishment of more ground base stations performs poorly in terms of timeliness and economics.
Adopt a drone track planning method based on converged multiple access, combined with non-orthogonal multiple access technology and orthogonal multiple access technology, to maximize ground users by obtaining UAV communication environment data. The overall throughput optimization problem model is optimized through user grouping and transmission power allocation optimization model parameters, and iteratively solves the improved Q-Learning algorithm to obtain the optimal path of the drone.
It improves the transmission rate and coverage rate of drone communication, achieves more efficient communication performance, reduces the number of iterations and solution time, and is suitable for drone mobile communication in complex environments.
Smart Images
Figure CN117459954B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of unmanned aerial vehicle communication, and in particular relates to a unmanned aerial vehicle trajectory planning method based on fusion multiple access. Background Art
[0002] In recent years, UAVs have been widely researched and applied in many fields. Their miniaturized aerial platform features make them an irreplaceable advantage in both combat and public areas as transportation and communication tools. Stimulated by excellent performance and a wide range of application scenarios, different requirements for the use of UAVs have been developed for different use environments. Among them, the idea of using UAVs in the field of communications as communication base stations or switching equipment has gradually become a popular direction and has achieved great development. The new application environment has also prompted the thinking and research of new scenarios and new technologies.
[0003] In a communication cell, decentralized communications among multiple users leads to an increase in communication load demand, but establishing more ground base stations to handle decentralized users is less timely and economical. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a UAV trajectory planning method based on fused multiple access in view of the deficiencies of the above-mentioned prior art, which integrates non-orthogonal multiple access technology and orthogonal multiple access technology to realize UAV trajectory planning, solve the communication performance problems such as insufficient communication rate and low communication coverage, and provide communication services with better communication performance for UAV communications covered by communication cells.
[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:
[0006] A UAV trajectory planning method based on fusion multiple access includes:
[0007] Step 1: Obtain the drone communication environment data, including the coordinate set of the maximum flight range allowed by the drone, the channel gain per unit distance of 1 meter, the communication cell user coordinate set, and the communication cell obstacle coordinate set;
[0008] Step 2: Based on the UAV communication environment data, non-orthogonal multiple access and orthogonal multiple access technologies are used to construct an optimization problem model to maximize the overall throughput of ground users;
[0009] Step 3: Optimize the parameters of the optimization problem model for maximizing the overall throughput of terrestrial users by grouping users and allocating transmission power;
[0010] Step 4: The improved Q-Learning algorithm is used to iteratively solve the optimization problem model of maximizing the overall throughput of ground users after optimizing the parameters to obtain the optimal trajectory of the UAV.
[0011] To optimize the above technical solutions, the specific measures taken also include:
[0012] The above step 1 obtains the coordinate set Limit [L_x, L_y, L_z] of the maximum range allowed for the UAV to fly from the UAV as an aerial base station assisted communication environment in a communication cell environment; the standard gain g d represents the channel gain per unit distance of 1 meter; the communication cell user coordinate set M, the user i position coordinate (x i ,y i ,0)∈M, i=1,2,3,……,N_user. N_user is the number of users in the communication cell; the communication cell obstacle coordinate set O, the obstacle o position (x o ,y o ,0)∈O, o=1,2,3,……,N_osc. N_osc is the number of obstacles in the communication cell.
[0013] The above step 2 constructs the following optimization problem model for maximizing the overall throughput of ground users:
[0014]
[0015]
[0016] Among them, C m,i,n is the transmission rate of position m and user i in non-orthogonal multiple access technology channel n;
[0017] M η is the UAV flight trajectory set; P n is the total channel power, P is the total power of the UAV transmitter;
[0018] L is the radius of the circle with the communication cell obstacle coordinate set O as the point o The coordinates of the communication cell, and the minimum flight altitude h FOB The above coordinates establish a feasible coordinate location set for the communication cell; r o is the obstacle radius;
[0019] m is any possible location of the drone, and its coordinates are (x m ,y m ,z m ), m∈L, and z m >h FOB ;
[0020] ξ m,i,k ∈{0,1} The drone is located at position m, the usage status variable of user i in channel k;
[0021] L_x, L_y are the x-axis and y-axis coordinates in the maximum flight range coordinate set Limit[L_x, L_y, L_z] of the drone;
[0022] l is the number of people served, l max To require the minimum number of service users to be;
[0023] is the power loss per coordinate unit moved at position m when the UAV flies within the allowed range, and β is the flight loss attenuation constant.
[0024] The user grouping described in the above step 3 specifically includes: sorting the channel gains of the users to be allocated, comparing the user with the largest channel gain with the user with the smallest channel gain, pairing them if they are different, and assigning the two signals to one channel alone if they are the same; deleting the paired users from the users to be allocated, and looping in sequence until the allocation is completed. If an odd number of users to be allocated is encountered, the individual user is assigned to one channel alone; after completing the channel status pairing, the users who failed to pair are accessed again, and by comparing the sum of the difference in signal-to-interference-to-noise ratio in the channel after the users are accessed, the channel access user with the larger difference is selected, and user access is performed in this way until the user grouping is completed.
[0025] The transmission power allocation in step 3 is to improve the transmission power of users in the grouped channel in a fractional order based on user grouping, thereby increasing the user transmission rate and improving the communication performance by improving the transmission power. The improved user transmission rate C m,i,n for:
[0026]
[0027] Among them, α ftpa is the power allocation factor, P n is the total transmit power of channel n, N p is the number of signals in channel n, is the equivalent channel gain, where d m,i is the Euclidean distance between the drone position m and user i, h m,i is the channel gain between the drone and position m and user i, is the channel Gaussian white noise variance.
[0028] The improved Q-Learning algorithm described in step 4 above solves the Q value table by traversing the environment to obtain the optimal trajectory of the entire environment, thereby obtaining the optimal trajectory. If the optimal trajectory is not obtained or the result does not converge, the traversal is repeated until the optimal trajectory is obtained. In this process, an improved reward function is used, and the discretized environment is regarded as a cube formed by stacking cubes of the same size. Each time a search is performed, only the outermost cube is taken. That is, if the outer surface of the cube formed by the unselected points is painted each time a search is performed, then only the cubes with paint are taken. The directed search method of the algorithm accelerates the convergence speed of the algorithm.
[0029] The improved reward function above is:
[0030]
[0031] Among them, C m,i,n is the transmission rate of position m and user i in non-orthogonal multiple access technology channel n;
[0032] L is the radius of the circle with the communication cell obstacle coordinate set O as the point o The coordinates of the communication cell, and the minimum flight altitude h FOB The above coordinates establish a feasible coordinate location set for the communication cell; r o is the obstacle radius;
[0033] is the power loss per coordinate unit when the UAV flies within the allowed range, β is the flight loss attenuation constant; N p is the number of signals in channel n.
[0034] The present invention has the following beneficial effects:
[0035] The present invention proposes a UAV trajectory planning method that integrates non-orthogonal multiple access technology and orthogonal multiple access technology, focuses on being closer to the real environment and having a higher transmission rate, realizes UAV trajectory planning that maximizes the comprehensive utility, improves the performance problem of NOMA communication, and uses an adaptive user allocation algorithm based on UAV-to-ground communication and a Q-Learning algorithm with an improved search method to make the algorithm have fewer iterations and faster solution time, so that better results can be achieved in trajectory planning for UAV mobile communications in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 This is a workflow diagram of the UAV trajectory planning method based on fusion multiple access;
[0037] Figure 2 It is a schematic diagram of the track trajectory in the coordinate system;
[0038] Figure 3 This is a simulation result diagram comparing the adaptive user allocation algorithm based on UAV-to-ground communication with random allocation;
[0039] Figure 4 The simulation results of converged, non-orthogonal and orthogonal multiple access on average frequency band utilization are shown in the figure;
[0040] Figure 5 The figure shows the simulation results of converged, non-orthogonal and orthogonal multiple access on average communication coverage. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0042] like Figure 1-5 As shown, the present invention is a UAV trajectory planning method based on fused multiple access. According to the obtained UAV communication cell environment data, the environmental parameters are obtained by discretization; according to the existing environment, an optimization problem model for maximizing the overall throughput of ground users is proposed; on this basis, an adaptive user allocation algorithm optimization model based on UAV-to-ground communication is proposed to improve the user transmission rate; according to the proposed improved Q-Learning algorithm, the optimal trajectory is obtained, and the communication performance is evaluated by the average frequency band utilization and communication coverage, so that the user communication performance in the communication cell is optimized. Specifically including:
[0043] Step 1: Obtain the drone communication environment data, including the coordinate set of the maximum flight range allowed by the drone, the channel gain per unit distance of 1 meter, the communication cell user coordinate set, and the communication cell obstacle coordinate set;
[0044] This step obtains the UAV communication environment data by obtaining and discretizing the communication cell area and the standard distance channel gain.
[0045] Step 2: Based on the UAV communication environment data, non-orthogonal multiple access and orthogonal multiple access technologies are used to construct an optimization problem model to maximize the overall throughput of ground users;
[0046] This step uses existing data to build the communication environment: By using the existing UAV communication environment data, an optimization problem model is constructed to maximize the overall throughput of ground users.
[0047] Step 3: Optimize the parameters of the optimization problem model for maximizing the overall throughput of terrestrial users by grouping users and allocating transmission power;
[0048] This step proposes an adaptive user allocation algorithm: Based on the optimization problem, an adaptive user allocation algorithm based on UAV-to-ground communication is proposed to optimize the communication performance through user grouping and transmission power allocation methods.
[0049] Step 4: The improved Q-Learning algorithm is used to iteratively solve the optimization problem model of maximizing the overall throughput of ground users after optimizing the parameters to obtain the optimal trajectory of the UAV.
[0050] This step iteratively solves the optimization problem: According to the improved Q-Learning algorithm, the proposed optimization problem of maximizing the overall throughput of ground users after the adaptive user allocation algorithm is iteratively solved to obtain the optimal trajectory and communication performance analysis.
[0051] In the embodiment, the step 1 comprises:
[0052] 1.1 Obtain location information from the drone as an aerial base station in a communication cell environment, and set the environmental location accuracy to L pre , forming the coordinate position, the maximum range coordinate set Limit[L_x, L_y, L_z] allowed for the drone to fly;
[0053] 1.2 Obtaining the standard gain g d Indicates the channel gain per unit distance of 1 meter;
[0054] 1.3 Obtain the communication cell user coordinate set M, user i position (x i ,y i ,0)∈M,i=1,2,3,……,N_user. N_user is the number of users in the communication cell;
[0055] 1.4 Obtain the communication cell obstacle coordinate set O, obstacle o position (x o ,y o ,0)∈O, o=1,2,3,……,N_osc. N_osc is the number of obstacles in the communication cell;
[0056] In an embodiment, the step 2 comprises:
[0057] 2.1 According to the communication cell obstacle coordinate set O and obstacle radius r o , except for the communication cell obstacle coordinate set O as the circle with a radius of r o The coordinates of the communication cell, and the minimum flight altitude h FOB The above coordinates establish the feasible coordinate position set L of the communication cell.
[0058] 2.2 Based on the communication environment data, the optimization problem model for the communication environment to maximize the overall throughput of ground users is:
[0059]
[0060]
[0061] For any possible location of the drone m(x m ,y m ,z m ), m∈L, and z m >h FOB For the subchannel set N, the usage variable of user i in channel n when the drone is located at position m is defined as ξ m,i,n ∈{0,1}, total channel power P n The total power of the UAV transmitter is P, the flight strategy of the UAV is η, and the flight trajectory set of the UAV is M η The total power of the UAV transmitter is P, the flight strategy of the UAV is η, and the flight trajectory set of the UAV is M η The maximum range of coordinates allowed for drone flight is Limit[L_x, L_y, L_z], the number of people served is l, and the minimum number of users required to be served is l max At the same time, the UAV flight loss is further introduced. When the UAV flies within the allowed range, the power loss per coordinate unit moved at position m is β is the flight loss attenuation constant. C m,i,n is the transmission rate for position m and user i in non-orthogonal multiple access technology channel n:
[0062]
[0063] α k (k=1,2,3……,N P ) is the power allocation factor of signal k in NOMA channel, where d m,i is the Euclidean distance between the drone position m and user i.
[0064] In the embodiment, step 3 proposes an adaptive user allocation algorithm based on UAV-to-ground communication based on the optimization problem, and optimizes the communication performance through user grouping and transmission power allocation method. Specifically, it includes:
[0065] 3.1 User grouping first uses the channel status pairing scheme:
[0066] By sorting the channel gains of the users to be assigned, the user with the largest channel gain is compared with the user with the smallest channel gain. If they are different, they are paired. If they are the same, the two signals need to be assigned to one channel alone.
[0067] The paired user is deleted from the users to be allocated, and the cycle is repeated until the allocation is completed. If an odd number of users to be allocated is encountered, the individual user is allocated to a channel alone.
[0068] After completing the channel status pairing, if the user fails to pair, the user access is adopted. By comparing the sum of the difference in the signal-to-interference-to-noise ratio in the channel after the access user, the channel access user with the larger difference is selected.
[0069] User access is performed in this way until user grouping is completed.
[0070] 3.2 Transmit power allocation scheme Based on user grouping, the transmit power of users in the grouped channel is improved in fractional order. By improving the transmit power, the user transmission rate is increased, thereby improving the communication performance. The improved user transmission rate C m,i,n for
[0071]
[0072] α ftpa is the power allocation factor, P n is the total transmit power of channel n, N p is the number of signals in channel n, g m,i =|h m,i | 2 is the equivalent channel gain, h m,i The channel gain between the drone and position m and user i, is the channel Gaussian white noise variance.
[0073] The simulation performance comparison of the adaptive user allocation algorithm based on comprehensive user grouping and transmission power allocation and random allocation is shown in the figure below. Figure 3 shown.
[0074] In the embodiment, the step 4 comprises:
[0075] For the non-convex optimization model of the optimization problem, the Q-Learning method is used to solve it. The optimal strategy is selected through the optimization strategy, and the selected strategy is marked as *
[0076]
[0077]
[0078]
[0079] The environmental state S t The value of v(S t ), in the environmental state S t Action A t The value is q(S t ,A t), the environment state St takes action A t Change the environment state to S t+1 The probability is The decay rate is γ.
[0080] 4.1 The value update equation is
[0081]
[0082] The value update learning rate is α.
[0083] The reward function is
[0084] 4.2 In actual implementation, due to different positions m, C m,i,n and There are different gaps in the number of sets. Consider the relationship between the total number of channels and the number of users. The improved reward function is:
[0085]
[0086] 4.3 Solve the Q value table by traversing the environment, that is, the state value table and v * Table, get the best trajectory of the entire environment, and then get the optimal trajectory. If the optimal trajectory is not obtained or the result does not converge, repeat the traversal until the optimal trajectory is obtained.
[0087] 4.4 In the iterative solution of the Q-Learning algorithm, due to the performance limitations of the physical calculation components, a directed search method is proposed, which regards the discretized environment as a cube formed by stacking cubes of the same size from the outside to the inside. Each time the search takes points, only the outermost cube is taken. That is, if the outer surface of the cube formed by the unselected points is painted each time the search is performed, then only the cubes with paint are taken to speed up the convergence of the algorithm.
[0088] like Figure 4-5As shown, the simulation results of the algorithm in fusion, non-orthogonal, and orthogonal multiple access on average frequency band utilization and communication coverage are respectively shown. The present invention improves the performance problem of non-orthogonal multiple access technology communication by introducing the fusion of non-orthogonal multiple access technology and orthogonal multiple access technology and an adaptive user allocation algorithm based on UAV-to-ground communication, and proposes a UAV trajectory planning method that maximizes the comprehensive utility by integrating non-orthogonal multiple access technology and orthogonal multiple access technology, and realizes the UAV trajectory planning that maximizes the comprehensive utility by improving the Q-Learning algorithm of the search method with fewer iterations and faster solution time. The performance problem of non-orthogonal multiple access technology communication is improved, and a UAV trajectory planning method that focuses on being closer to the real environment and has a higher transmission rate is proposed. And through an adaptive user allocation algorithm based on UAV-to-ground communication and an improved Q-Learning algorithm of the search method, the algorithm has fewer iterations and faster solution time.
[0089] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
[0090] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. A UAV trajectory planning method based on fusion multiple access, characterized in that: include: Step 1: Obtain the drone communication environment data, including the coordinate set of the maximum flight range allowed by the drone, the channel gain per unit distance of 1 meter, the communication cell user coordinate set, and the communication cell obstacle coordinate set; Step 2: Based on the UAV communication environment data, non-orthogonal multiple access and orthogonal multiple access technologies are used to construct an optimization problem model to maximize the overall throughput of ground users; Step 3: Optimize the parameters of the optimization problem model for maximizing the overall throughput of terrestrial users by grouping users and allocating transmission power; Step 4: The optimization problem model of maximizing the overall throughput of ground users after optimizing parameters is iteratively solved using the improved Q-Learning algorithm to obtain the optimal trajectory of the UAV; The step 2 constructs the following optimization problem model for maximizing the overall throughput of ground users: Among them, C m,i,n is the transmission rate of position m and user i in non-orthogonal multiple access technology channel n; M η is the UAV flight trajectory set; P n is the total channel power, P is the total power of the UAV transmitter; L is the radius of the circle with the communication cell obstacle coordinate set O as the point o The coordinates of the communication cell, and the minimum flight altitude h FOB The above coordinates establish a feasible coordinate location set for the communication cell; r o is the obstacle radius; m is any possible location of the drone, and its coordinates are (x m ,y m ,z m ), m∈L, and z m >h FOB ; ξ m,i,k ∈{0,1} The drone is located at position m, the usage status variable of user i in channel k; L_x, L_y are the x-axis and y-axis coordinates in the maximum flight range coordinate set Limit[L_x, L_y, L_z] of the drone; l is the number of people served, l max To require the minimum number of service users to be; is the power loss per coordinate unit moved at position m when the UAV flies within the allowed range, and β is the flight loss attenuation constant; The transmission power allocation in step 3 is to improve the transmission power of users in the grouped channel in a fractional order based on user grouping, and to increase the user transmission rate by improving the transmission power, thereby improving the communication performance. The improved user transmission rate C m,i,n for: Among them, α ftpa is the power allocation factor, P n is the total transmit power of channel n, N p is the number of signals in channel n, is the equivalent channel gain, where d m,i is the Euclidean distance between the drone position m and user i, h m,i is the channel gain between the drone and position m and user i, is the channel Gaussian white noise variance; The improved Q-Learning algorithm in step 4 solves the Q value table by traversing the environment to obtain the optimal trajectory of the entire environment, thereby obtaining the optimal trajectory. If the optimal trajectory is not obtained or the result does not converge, the traversal is repeated until the optimal trajectory is obtained. In this process, the improved reward function is used, and the discretized environment is regarded as a cube formed by stacking cubes of the same size. Each time the search is performed, only the outermost cube is selected. That is, if the outer surface of the cube formed by the unselected points is painted each time the search is performed, the directed search method of the algorithm that only takes the cubes with paint accelerates the convergence speed of the algorithm. The improved reward function is: Among them, C m,i,n is the transmission rate of position m and user i in non-orthogonal multiple access technology channel n; L is the radius of the circle with the communication cell obstacle coordinate set O as the point o The coordinates of the communication cell, and the minimum flight altitude h FOB The above coordinates establish a feasible coordinate location set for the communication cell; r o is the obstacle radius; is the power loss per coordinate unit when the UAV flies within the allowed range, β is the flight loss attenuation constant; N p is the number of signals in channel n.
2. The method for UAV trajectory planning based on fusion multiple access according to claim 1 is characterized in that: The step 1 obtains the coordinate set Limit [L_x, L_y, L_z] of the maximum range allowed for the UAV to fly from the UAV as an aerial base station assisted communication environment in a communication cell environment; the standard gain g d represents the channel gain per unit distance of 1 meter; the communication cell user coordinate set M, the user i position coordinate (x i ,y i ,0)∈M, i=1,2,3,……,N_user;N_user is the number of users in the communication cell; the communication cell obstacle coordinate set O, obstacle o position (x o ,y o ,0)∈O,o=1,2,3,……,N_osc; N_osc is the number of obstacles in the communication cell.
3. The method for UAV trajectory planning based on fusion multiple access according to claim 1 is characterized in that: The user grouping described in step 3 specifically includes: sorting the channel gains of the users to be allocated, comparing the user with the largest channel gain with the user with the smallest channel gain, pairing them if they are different, and assigning the two signals to one channel alone if they are the same; deleting the paired users from the users to be allocated, and looping in sequence until the allocation is completed. If an odd number of users to be allocated is encountered, the individual users are assigned to one channel alone; after completing the channel status pairing, the users who failed to be paired are accessed again, and by comparing the sum of the difference in signal-to-interference-to-noise ratio in the channel after the users are accessed, the channel access users with the larger difference are selected, and user access is performed in this way until the user grouping is completed.