An unmanned aerial vehicle path planning method under energy constraint

Abstract

The application discloses a kind of unmanned plane path planning methods under energy constraint condition, including the following steps: obtaining environment and unmanned plane information, using probability map to carry out real-time update;Considering task requirements and energy constraint, generate multiple possible paths, and estimate the energy consumption of each path, select the optimal initial path;On the basis of optimal initial path, according to the feedback of probability map, learn and dynamically adjust flight path;Continuously monitor the energy level of unmanned plane, real-time adjust the task or return path of unmanned plane.The method of the application not only improves the energy use efficiency of unmanned plane when executing task, but also enhances its navigation ability in complex and uncertain environment, such a comprehensive scheme enables unmanned plane to efficiently and safely complete multiple tasks, while improving the intelligent level and adaptability of the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles (UAVs), and in particular relates to a UAV path planning method under energy constraint conditions. Background Art

[0002] As an emerging intelligent flight platform, drones are increasingly being used in military, logistics, surveillance, agriculture, and other fields. With the advancement of drone technology, their autonomous navigation and mission execution capabilities in complex environments have gradually become a research focus. Traditional drone path planning methods rely on pre-defined rules or path optimization techniques based on graph search algorithms. While these methods can handle relatively simple static environments, they exhibit significant limitations when faced with dynamic, changing, and complex environments full of uncertainty.

[0003] UAV missions in complex environments often present numerous challenges, including limited energy resources, complex terrain obstacles, unstable weather conditions, and dynamically changing mission requirements. To address these challenges, path planning must not only consider the shortest path or lowest energy consumption, but also handle environmental uncertainty, avoid obstacles, and ensure safe flight. Simultaneously, UAVs must balance energy efficiency with the real-time nature of their missions. Therefore, achieving efficient and safe mission planning within energy constraints has become a key research topic in UAV path planning.

[0004] Therefore, we study the efficient, safe and stable planning method of UAV paths under energy constraints, so that when UAVs face complex environments, they can optimize energy usage while completing their tasks. Summary of the Invention

[0005] In view of this, the present invention aims to propose a UAV path planning method under energy constraints, comprising the following steps:

[0006] Step 1: Obtain environment and drone information and use probability map to update in real time;

[0007] Step 2: Considering the task requirements and energy constraints, multiple possible paths are generated, the energy consumption of each path is estimated, and the optimal initial path is selected;

[0008] Step 3: Based on the optimal initial path, learn and dynamically adjust the flight path according to the feedback from the probability map;

[0009] Step 4: Continuously monitor the drone's energy level and adjust the drone's mission and return path in real time.

[0010] Specifically, the real-time updating using the probability map includes the following steps:

[0011] Initialize the map. The map is represented by a grid map or a continuous space, where each grid or area contains an initial probability indicating whether the location is occupied.

[0012] Collect surrounding environment data in real time through sensors as the basis for environmental perception;

[0013] Based on the sensor's measurement value, the probability of whether there is an obstacle at each position is updated through the Bayesian update rule;

[0014] As the drone moves and environmental information is updated, the probability map is continuously adjusted and updated.

[0015] Specifically, the Bayesian update rule is used to update the probability of whether there is an obstacle at each position, including the following steps: when the UAV is flying, the sensor collects the obstacle information corresponding to a certain grid m. i Related measurement value z t , the measured value z t Indicates that at time t, grid m i Whether it is occupied or not, at this time, the probability of each grid is updated using the Bayesian formula, and the Bayesian update formula is:

[0016]

[0017] Among them, m i Indicates the position of the i-th grid, s i Indicates the state of the i-th grid, whether it is occupied, P(s i |z t ) indicates that the sensor observes z t After that, grid m i The posterior probability of being occupied, P(z t |s i ) represents a given grid m i The sensor observes z t The probability of i ) represents the grid m i Prior probability of being occupied, P(z t ) indicates that the sensor observes z t The edge probability is obtained by weighted calculation of all possible grid states.

[0018] Furthermore, the method of considering task requirements and energy constraints, generating multiple possible paths, estimating the energy consumption of each path, and selecting the optimal initial path includes the following steps:

[0019] Using a probabilistic path search algorithm, the state space is defined as a set of nodes in a grid map. State transition is the process of moving from one grid to the next. The goal is to find the path from the initial position x0 to the target position x g Multiple paths of

[0020] The cumulative risk of a path is estimated by the grid occupancy probability in the probability map. The risk R(π) of a path π is expressed as the sum of the occupancy probabilities of all grids passed by the path:

[0021]

[0022] Among them, P(s i =1) indicates grid m i The probability of being occupied by an obstacle, i represents a grid on the path π;

[0023] Calculate the energy consumption of flight distance and speed. The energy consumption per unit time of the UAV on the path is expressed as flight power P, which is proportional to the flight speed v and the flight distance d. The energy consumption E(π) of path π is calculated by the following formula:

[0024]

[0025] Among them, P(v i ) represents the flight power of the UAV in the i-th segment on the path, which depends on the flight speed v i , represents the flight time of the i-th segment, d i is the flight distance between two locations on the path, and n represents the number of segments of the path;

[0026] Calculate the energy consumption of terrain and wind speed. When considering the influence of terrain, the energy consumption is calculated by adding the vertical height change h caused by the terrain. i Make corrections:

[0027]

[0028] Where γ is a constant related to the weight of the drone, h i represents the height difference on the i-th path, and the wind speed w i The effect of is taken into account by correcting the flight power:

[0029] P(v i ,w i )=P(v i )·(1+αw i )

[0030] Among them, α is the coefficient of wind speed on flight power, w i represents the wind speed of the i-th path;

[0031] When the energy consumption E(π) and risk R(π) of multiple paths are calculated, an optimization algorithm is used to select the optimal path, and a comprehensive objective function J(π) is defined to evaluate the pros and cons of each path:

[0032] J(π)=λ1E(π)+λ2R(π)

[0033] Among them, λ1 and λ2 are the weight coefficients of energy and risk, respectively, reflecting the priority of the task for different goals. By adjusting the weights, the system finds a balance between energy saving and risk avoidance.

[0034] Preferably, the process of considering task requirements and energy constraints, generating multiple possible paths, estimating the energy consumption of each path, and selecting the optimal initial path comprises the following steps:

[0035] Step 201, initialize a tree structure T = (V, E), where V is a node set, initially including the starting point V = {x start}, E is the set of edges, initially empty; set the initial position of the drone to x start and the target position is x goal , build a probability map Each position m in the map i The probability of including obstacles P(s i ), the map is used in the path evaluation process to determine whether it is safe to pass through a certain area, and to initialize the energy constraint condition E max , represents the maximum available energy of the UAV;

[0036] Step 202: randomly sample a point x from the space rand , the point is within the range of the configuration space, which is defined as the flight area of ​​the UAV;

[0037] Step 203, find the distance x in the current tree T rand The nearest node x nearest , the distance calculation formula is as follows: ‖·‖ represents the Euclidean distance;

[0038] Step 204, from x nearest To x rand Expand and generate a new node x new , the new node is at x nearest to x rand In the direction of , and expand with a step size Δd:

[0039]

[0040] The new node x after expansion newAdd the node set V of the tree and add the edge (x nearest ,x new )Add edge set E;

[0041] Step 205, when expanding the tree, check nearest to x new Whether the path passes through the obstacle area in the probability map, calculate the risk of the path, the risk R(x nearest ,x new ) is estimated by the occupancy probability of the grid on the path:

[0042]

[0043] Among them, π(x nearest ,x new ) means from x nearest to x new All grids passed by on the path, P(s i =1) is the grid s i The probability of being occupied by obstacles; if the cumulative risk of the path exceeds the preset risk threshold, the extension is discarded and x is not new Add to the tree;

[0044] Step 206: For each newly generated path segment, consider the flight distance and speed and evaluate the energy consumption. The energy consumption E(x nearest ,x new ) is:

[0045]

[0046] Where P(v) is the flight power corresponding to the flight speed v, ‖x new -x nearest ‖ is the distance of the path segment, v is the flight speed, if the cumulative energy consumption of the path exceeds the maximum energy E of the drone max , then stop expanding the path;

[0047] Step 207, repeat steps 202 to 206, and continue to expand the tree until the connection starting point x is found start and the target point x goal A feasible path of x, or reaching the preset maximum number of iterations; generate multiple potential paths, each path corresponds to start to x goal The energy consumption and risk of each path on a branch of the tree are calculated and recorded during the expansion process.

[0048] Step 208: Select the optimal path that meets the energy constraint from among the generated multiple paths. The path is selected using an optimization function that comprehensively considers energy consumption E(π) and risk R(π):

[0049] J(π)=λ1E(π)+λ2R(π)

[0050] Where E(π) represents the total energy consumption of path π, R(π) represents the risk of the path, and λ1 and λ2 are the weight coefficients of energy consumption and risk, which are adjusted according to task requirements.

[0051] Preferably, in step 206, considering the influence of terrain height change h and wind speed w, the energy consumption of the path segment is corrected to:

[0052]

[0053] Where: γ is the coefficient of altitude change on energy consumption; α is the coefficient of wind speed on energy consumption.

[0054] Furthermore, when considering the flight distance and speed, the energy consumption is calculated as follows: P(v) = c1v 3 +c2v 2 +c3v+c4, where c1, c2, c3, c4 are constants related to the drone model, c1v 3 represents aerodynamic drag, c2v 2 represents propulsion power, c3v represents mechanical loss, and c4 represents fixed power loss.

[0055] Furthermore, the energy consumption also takes into account the energy consumption of attitude adjustment, including steering energy consumption and energy consumption of acceleration and deceleration. During the steering process of the UAV in the path, due to the effect of inertia, the additional energy consumption is related to the steering angle θ: E turn (θ)=k turn ·θ, where E turn (θ) represents the energy consumption when the steering angle is θ, k turn is a constant related to the UAV model and flight dynamics, representing the energy consumption coefficient of steering. If the UAV needs to accelerate or decelerate on the path, additional power is required to overcome the change in speed. Assuming that the speed of the UAV changes from v1 to v2 between the path segment x1 and x2, the energy consumption is expressed as: Among them, E acc / dec is the additional energy consumption during the speed change, m is the mass of the UAV, and v1 and v2 are the velocities at the beginning and end of the path segment, respectively.

[0056] Specifically, the feedback according to the probability map is used to learn and dynamically adjust the flight path, including the following steps:

[0057] Define the state and action space, the state space S includes: the current position information (x, y, z) of the UAV, the current flight speed v, the current flight direction θ, the remaining energy E remaining , the environmental information includes the obstacle position, the wind speed, the terrain height environmental characteristics Env, the state is represented by a vector s t = x t ,y t ,z t ,v t ,θ t ,E remaining ,Env t ); the action space A includes speed adjustment, direction adjustment, height adjustment, each action is represented by a vector a t = (a v ,a θ ,a h ), a v represents the adjustment of flight speed, a θ represents the adjustment of flight direction, and a h represents the adjustment of flight height;

[0058] Define the reward function, the reward function includes energy efficiency reward, safety reward, task completion reward, the energy efficiency reward R E (s,a), assuming that the energy consumed in state s is E(s,a) when action a is performed, the energy reward can be designed as a negative value, indicating that the greater the energy consumption, the lower the reward, R E (s,a) = -α E ·E(s,a), where α E is the weight coefficient of energy consumption, and E(s,a) is the energy consumption required for performing action a; the safety reward R S (s,a) is designed based on the distance d obs (s) between the UAV and the obstacle, the closer the distance, the lower the reward, where α S is the safety weight coefficient, and ∈ is a small positive number to prevent division by zero error; the task completion reward R T (s) gives a high reward R T (s) = β T when the UAV successfully reaches the target point, indicating task completion; the comprehensive reward function, the total reward R(s,a) of the UAV in state s performing action a is represented as the weighted sum of the above rewards, R(s,a) = R E (s,a) + R S (s,a) + R T (s).

[0059] State transfer and update, the drone is in state s t Next, perform action a t After that, based on the state transition equation of the dynamic model and the environment, it enters the next state s t+1 , the state transition model is expressed as: s t+1 =f(s t ,a t )+ω t , where f(s t ,a t ) is the state transition function of the drone, which describes the state change after performing an action in the current state, ω t is the process noise, which represents the influence of environmental uncertainty;

[0060] Energy consumption calculation, in each state s t Next, perform action a t When calculating the energy consumption E(s t ,a t );

[0061] Strategy update and optimization, using reinforcement learning to maximize cumulative rewards through strategy updates;

[0062] Through reinforcement learning, the speed, direction and altitude of the drone are autonomously adjusted to achieve the optimal path planning goal.

[0063] Furthermore, the method of using reinforcement learning to maximize the cumulative reward by updating the strategy includes the following steps:

[0064] Initialize the parameters θ, set the learning rate α, discount factor γ, entropy regularization coefficient β, and trust region limit step size δ;

[0065] Sampling generation path, from strategy π θ (a|s) Sampling generates multiple flight paths, recording states, actions, rewards, and state transitions;

[0066] Calculate the advantage function A(s t ,a t ), using a generalized dominance-based estimation method to reduce variance.

[0067] Update the policy gradient and update the policy parameters under the trust region constraint: J(θ) is the expected cumulative reward;

[0068] Adding entropy regularization term to encourage the exploration of the strategy and prevent premature convergence to suboptimal solutions;

[0069] Strategy update and iteration, repeat the above steps until the strategy converges, and finally obtain the optimized path planning strategy;

[0070] The advantage function A(s t ,a t ) represents the advantage of the current action compared with the expected action in the strategy, A(s t ,a t )=Q(s t ,a t )-V(s t ), where Q(s t ,a t ) is the state action value function, indicating that in state s t Next, perform action a t The cumulative reward, V(s t ) is the state value function, indicating that in state s t The expected cumulative reward;

[0071] The generalized odds estimate The calculation formula is: Among them, δ t =R(s t ,a t )+γV(s t+1 )-V(s t ) is the time difference error, λ is a parameter that weighs the bias and variance, and is between [0,1].

[0072] Specifically, the continuous monitoring of the drone's energy level and real-time adjustment of the drone's mission and return path include the following steps: during flight, the drone's energy level is continuously monitored and the path planning is adjusted in real time. When the energy approaches the warning threshold, the system will automatically calculate the shortest path back to the charging station and trigger the drone to return to the charging station when necessary. If the drone's energy is insufficient to complete the current mission, other drones will be notified to take over the mission, or the formation will be coordinated to adapt to the current energy conditions.

[0073] The beneficial effects of the present invention are as follows: The present invention takes into account the UAV path planning under energy constraints, provides full-time and full-domain energy consumption calculation and monitoring, and also adopts a path planning method that combines pre-path planning and real-time path planning, which not only improves the energy utilization efficiency of the UAV when performing tasks, but also enhances its navigation ability in complex and uncertain environments. Such a comprehensive solution enables the UAV to complete multiple tasks efficiently and safely, while improving the intelligence level and adaptability of the system, providing a good foundation for more complex UAV application scenarios in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The overall flow chart of the UAV path planning method under energy constraints is shown. DETAILED DESCRIPTION

[0075] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely some, rather than all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0076] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0077] like Figure 1 As shown, this embodiment proposes a UAV path planning method under energy constraints, including the following steps:

[0078] Step 1: Obtain environment and drone information and use probability map to update in real time;

[0079] Step 2: Considering the task requirements and energy constraints, multiple possible paths are generated, the energy consumption of each path is estimated, and the optimal initial path is selected;

[0080] Step 3: Based on the optimal initial path, learn and dynamically adjust the flight path according to the feedback from the probability map;

[0081] Step 4: Continuously monitor the drone's energy level and adjust the drone's mission or return path in real time.

[0082] The basic concept of this embodiment is as follows: First, each drone is assigned a mission, and its starting and target locations, along with relevant constraints (such as energy and obstacles), are determined. Upon startup, the drone records its initial energy and calculates the maximum range allowed by its current energy. After startup, the drone acquires environmental information through sensors and uses a probabilistic map construction method to update the map, including the locations of obstacles, targets, and charging stations. In a dynamic environment, environmental information is updated in real time to ensure that the drone's path planning decisions are based on the latest data. Taking into account mission requirements and energy constraints, multiple possible paths are generated, and the energy consumption of each path is estimated. The optimal initial path is selected by evaluating the mission completion rate and energy consumption of each path. Based on the path planning, the reinforcement learning module dynamically adjusts the path based on real-time feedback. Using the reinforcement learning model, the drone can autonomously adjust its flight speed, direction, and altitude to improve the energy efficiency and safety of the path. Reinforcement learning uses simulations and historical data to gradually improve path selection, ensuring even better performance in subsequent flight missions. During flight, the drone's energy level is continuously monitored, and the path planning is adjusted in real time. When the energy level approaches the warning threshold, the system automatically calculates the shortest path back to the charging station and triggers the drone to return to the charging station if necessary. If the drone's energy level is insufficient to complete the current mission, it notifies other drones to take over the mission or coordinate the formation to adapt to the current energy level.

[0083] Probabilistic map construction methods are used to handle uncertainty in the environment, describing the locations of obstacles, targets, and other important landmarks through continuously updated probability distributions. Specifically, the real-time updating of the probabilistic map includes the following steps:

[0084] Initialize the map. The map is represented by a grid map or a continuous space, where each grid or area contains an initial probability indicating whether the location is occupied.

[0085] Collect surrounding environment data in real time through sensors as the basis for environmental perception;

[0086] Based on the sensor's measurement value, the probability of whether there is an obstacle at each position is updated through the Bayesian update rule;

[0087] As the drone moves and environmental information is updated, the probability map is continuously adjusted and updated.

[0088] Specifically, the Bayesian update rule is used to update the probability of whether there is an obstacle at each position, including the following steps: when the UAV is flying, the sensor collects the obstacle information corresponding to a certain grid m. i Related measurement value z t , the measured value z t Indicates that at time t, grid m iWhether it is occupied or not, at this time, the probability of each grid is updated using the Bayesian formula, and the Bayesian update formula is:

[0089]

[0090] Among them, m i Indicates the position of the i-th grid, s i Indicates the state of the i-th grid, whether it is occupied, P(s i |z t ) indicates that the sensor observes z t After that, grid m i The posterior probability of being occupied, P(z t |s i ) represents a given grid m i The sensor observes z t The probability of i ) represents the grid m i Prior probability of being occupied, P(z t ) indicates that the sensor observes z t The edge probability is obtained by weighted calculation of all possible grid states.

[0091] The probability map provides a clear probability estimate of the occupancy of each location. Based on the drone’s current location, target location, and energy status, the updated probability map is used to plan the optimal path, ensuring the drone can complete its mission safely and efficiently.

[0092] Furthermore, the method of considering task requirements and energy constraints, generating multiple possible paths, estimating the energy consumption of each path, and selecting the optimal initial path includes the following steps:

[0093] Using a probabilistic path search algorithm, the state space is defined as a set of nodes in a grid map. State transition is the process of moving from one grid to the next. The goal is to find the path from the initial position x0 to the target position x g Multiple paths of

[0094] The cumulative risk of a path is estimated by the grid occupancy probability in the probability map. The risk R(π) of a path π is expressed as the sum of the occupancy probabilities of all grids passed by the path:

[0095]

[0096] Among them, P(s i =1) indicates grid m i The probability of being occupied by an obstacle, i represents a grid on the path π;

[0097] The energy consumption of flight distance and speed, the energy consumption per unit time of the UAV on the path is represented as flight power P, which is proportional to flight speed v and flight distance d. The energy consumption E(π) of the path π is calculated by the following formula:

[0098]

[0099] where P(v i ) represents the flight power of the UAV on the i-th segment of the path, which depends on the flight speed v i , represents the flight time of the i-th segment, d i is the flight distance between two positions on the path, and n represents the number of segments of the path;

[0100] The energy consumption of terrain and wind speed is calculated, and when considering the influence of terrain, the energy consumption is corrected by adding the vertical height change h i :

[0101]

[0102] where γ is a constant related to the weight of the UAV, h i represents the height difference on the i-th segment of the path, and the influence of wind speed w i is considered by correcting the flight power:

[0103] P(v i ,w i )=P(v i )·(1+αw i )

[0104] where α is the coefficient of the influence of wind speed on flight power, and w i represents the wind speed on the i-th segment of the path;

[0105] When the energy consumption E(π) and risk R(π) of multiple paths are calculated, an optimization algorithm is used to select the optimal path, and a comprehensive objective function J(π) is defined to evaluate the pros and cons of each path:

[0106] J(π)=λ1E(π)+λ2R(π)

[0107] where λ1 and λ2 are the weight coefficients of energy and risk, respectively, reflecting the priority of the task for different objectives. By adjusting the weights, the system finds a balance between energy saving and risk avoidance.

[0108] By generating multiple potential paths and estimating the energy consumption of each path, the system effectively helps drones select the optimal path in complex environments, taking into account probability maps, mission requirements, and energy constraints. Combining path risk assessment with multi-objective optimization ensures that drones can complete their missions efficiently while avoiding unnecessary energy consumption and risk.

[0109] This embodiment proposes a novel approach: generating multiple potential paths for a drone and estimating the energy consumption and risk of each path. Then, through a multi-objective optimization method, an optimal path with minimal energy consumption and optimal safety is selected. This approach is applicable to complex and uncertain environments and can ensure that drones successfully complete their missions under limited energy conditions.

[0110] Preferably, the process of considering task requirements and energy constraints, generating multiple possible paths, estimating the energy consumption of each path, and selecting the optimal initial path comprises the following steps:

[0111] Step 201, initialize a tree structure T = (V, E), where V is a node set, initially including the starting point V = {x start}, E is the set of edges, initially empty; set the initial position of the drone to x start and the target position is x goal , build a probability map Each position m in the map i The probability of including obstacles P(s i ), the map is used in the path evaluation process to determine whether it is safe to pass through a certain area, and to initialize the energy constraint condition E max , represents the maximum available energy of the UAV;

[0112] Step 202: randomly sample a point x from the space rand , the point is within the range of the configuration space, which is defined as the flight area of ​​the UAV;

[0113] Step 203, find the distance x in the current tree T rand The nearest node x nearest , the distance calculation formula is as follows: ‖·‖ represents the Euclidean distance;

[0114] Step 204, from x nearest To x rand Expand and generate a new node x new , the new node is at x nearest to x rand In the direction of , and expand with a step size Δd:

[0115]

[0116] The new node x after expansion new Add the node set V of the tree and add the edge (x nearest ,x new )Add edge set E;

[0117] Step 205, when expanding the tree, check nearest to x new Whether the path passes through the obstacle area in the probability map, calculate the risk of the path, the risk R(x nearest ,x new ) is estimated by the occupancy probability of the grid on the path:

[0118]

[0119] Among them, π(x nearest ,x new ) means from x nearest to x new All grids passed by on the path, P(s i =1) is the grid s i The probability of being occupied by obstacles; if the cumulative risk of the path exceeds the preset risk threshold, the extension is discarded and x is not new Add to the tree;

[0120] Step 206: For each newly generated path segment, consider the flight distance and speed and evaluate the energy consumption. The energy consumption E(x nearest ,x new ) is:

[0121]

[0122] Where P(v) is the flight power corresponding to the flight speed v, ‖x new -x nearest ‖ is the distance of the path segment, v is the flight speed, if the cumulative energy consumption of the path exceeds the maximum energy E of the drone max , then stop expanding the path;

[0123] Step 207, repeat steps 202 to 206, and continue to expand the tree until the connection starting point x is found start and the target point x goal A feasible path of x, or reaching the preset maximum number of iterations; generate multiple potential paths, each path corresponds to start to x goal The energy consumption and risk of each path on a branch of the tree are calculated and recorded during the expansion process.

[0124] Step 208, in the generated multiple paths, the optimal path meeting the energy constraint is selected, and the path is selected by an optimization function which comprehensively considers the energy consumption E(π) and the risk R(π):

[0125] J(π) = λ1E(π) + λ2R(π)

[0126] Wherein, E(π) represents the total energy consumption of the path π, R(π) represents the risk of the path, λ1, λ2 are the weight coefficients of energy consumption and risk, which are adjusted according to the task demand.

[0127] Preferably, in step 206, the energy consumption of the path segment is corrected considering the influence of the terrain height change h and the wind speed w:

[0128]

[0129] Wherein: γ1 is the coefficient of height change on energy consumption; α is the influence coefficient of wind speed on energy consumption.

[0130] Further, the energy consumption considers that the flight power P(v) changes with the speed when considering the flight distance and the speed, and the calculation formula is: P(v) = c1v 3 +c2v 2 +c3v+c4, wherein c1, c2, c3, c4 are constants related to the model of the unmanned aerial vehicle, c1v 3 represents the aerodynamic resistance, c2v 2 represents the propulsion power, c3v represents the mechanical loss, and c4 is the fixed power loss.

[0131] Further, the energy consumption also considers the energy consumption of attitude adjustment, including the energy consumption of turning and the energy consumption of acceleration and deceleration; in the turning process of the unmanned aerial vehicle in the path, due to the action of inertial force, the additional energy consumption is related to the turning angle θ: E turn (θ) = k turn ·θ, wherein E turn (θ) represents the energy consumption when the turning angle is θ, k turn is a constant related to the model of the unmanned aerial vehicle and flight dynamics, representing the energy consumption coefficient of turning; if the unmanned aerial vehicle needs to accelerate or decelerate on the path, additional power is needed to overcome the change of speed, assuming that the speed of the unmanned aerial vehicle changes from v1 to v2 between path segments x1 to x2, the energy consumption is represented as: Wherein, E acc / dec is the additional energy consumption in the process of speed change, m is the mass of the unmanned aerial vehicle, and v1 and v2 are the speeds at the beginning and end of the path segment respectively.

[0132] In this embodiment, reinforcement learning-driven path optimization is an intelligent method that enables drones to autonomously learn optimal paths in dynamic and uncertain environments. Through reinforcement learning, the drone can adjust its flight speed, direction, and altitude based on environmental feedback to improve the energy efficiency and safety of the path.

[0133] Specifically, learning and dynamically adjusting the flight path based on the feedback from the probability map includes the following steps:

[0134] Define the state and action space. The state space S includes: the current position information of the drone (x, y, z), the current flight speed v, the current flight direction θ, and the remaining energy E. remaining , environmental information includes obstacle location, wind speed, terrain height environmental characteristics Env, and the state is represented by a vector s t =x t ,y t ,z t ,v t ,θ t ,E remaining ,Env t ); the action space A includes speed adjustment, direction adjustment, and height adjustment, and each action is represented by a vector a t =(a v ,a θ ,a h ), a v Indicates adjusting the flight speed, a θ Indicates adjusting the flight direction, a h Indicates adjusting the flight altitude;

[0135] Define the reward function, which includes energy efficiency reward, safety reward, task completion reward, and energy efficiency reward R E (s, a), assuming that the energy consumed by executing action a in state s is E(s, a), then the energy reward can be designed to be a negative value, indicating that the greater the energy consumption, the lower the reward, R E (s,a)=-α E E(s,a), where α E is the weight coefficient of energy consumption, E(s,a) is the energy consumption required to perform action a; safety reward R S (s,a), based on the distance d between the drone and the obstacle obs (s) are designed, the closer the distance, the lower the reward. Among them, α S is the safety weight coefficient, ∈ is a small positive number used to prevent division by zero errors; the task completion reward R T (s), when the drone successfully reaches the target point, it is given a high reward R T (s) = βT , indicating that the task is completed; the comprehensive reward function, the total reward R(s,a) for the drone to perform action a in state s is expressed as the weighted sum of the above rewards, R(s,a)=R E (s,a)+R S (s,a)+R T (s);

[0136] State transfer and update, the drone is in state s t Next, perform action a t After that, based on the state transition equation of the dynamic model and the environment, it enters the next state s t+1 , the state transition model is expressed as: s t+1 =f(s t ,a t )+ω t , where f(s t ,a t ) is the state transition function of the drone, which describes the state change after performing an action in the current state, ω t is the process noise, which represents the influence of environmental uncertainty;

[0137] Energy consumption calculation, in each state s t Next, perform action a t When calculating the energy consumption E(s t ,a t );

[0138] Strategy update and optimization, using reinforcement learning to maximize cumulative rewards through strategy updates;

[0139] Through reinforcement learning, the speed, direction and altitude of the drone are autonomously adjusted to achieve the optimal path planning goal.

[0140] In reinforcement learning-driven path optimization, the improved policy gradient method (policy-based reinforcement learning) can improve computational efficiency and reduce energy consumption, including the balance between reducing sample complexity, improving learning efficiency and optimizing the exploration and exploitation of policies.

[0141] Furthermore, the method of using reinforcement learning to maximize the cumulative reward by updating the strategy includes the following steps:

[0142] Initialize the parameters θ, set the learning rate α, discount factor γ, entropy regularization coefficient β, and trust region limit step size δ;

[0143] Sampling generation path, from strategy π θ (a|s) Sampling generates multiple flight paths, recording states, actions, rewards, and state transitions;

[0144] Calculate the advantage function A(s t ,a t ), using a generalized dominance-based estimation method to reduce variance.

[0145] Update the policy gradient and update the policy parameters under the trust region constraint: J(θ) is the expected cumulative reward;

[0146] Adding entropy regularization term to encourage the exploration of the strategy and prevent premature convergence to suboptimal solutions;

[0147] Strategy update and iteration, repeat the above steps until the strategy converges, and finally obtain the optimized path planning strategy;

[0148] The advantage function A(s t ,a t ) represents the advantage of the current action compared with the expected action in the strategy, A(s t ,a t )=Q(s t ,a t )-V(s t ), where Q(s t ,a t ) is the state action value function, indicating that in state s t Next, perform action a t The cumulative reward, V(s t ) is the state value function, indicating that in state s t The expected cumulative reward;

[0149] The generalized odds estimate The calculation formula is: Among them, δ t =R(s t ,a t )+γV(s t+1 )-V(s t ) is the time difference error, λ is a parameter that weighs the bias and variance, and is between [0,1].

[0150] Traditional policy gradient methods directly use reward signals for optimization, but this approach has high sample complexity and is susceptible to noise. The improved method introduces an advantage function to replace the direct use of cumulative reward signals, thereby reducing the computational burden and improving learning efficiency.

[0151] Specifically, the continuous monitoring of the drone's energy level and real-time adjustment of the drone's mission and return path include the following steps: during flight, the drone's energy level is continuously monitored and the path planning is adjusted in real time. When the energy approaches the warning threshold, the system will automatically calculate the shortest path back to the charging station and trigger the drone to return to the charging station when necessary. If the drone's energy is insufficient to complete the current mission, other drones will be notified to take over the mission, or the formation will be coordinated to adapt to the current energy conditions.

[0152] The advantages and benefits of the present invention include: real-time monitoring of the drone's remaining battery life and optimizing the flight path based on energy consumption and mission requirements, thereby maximizing the likelihood of mission completion. This ensures that the drone effectively utilizes every bit of energy during mission execution and avoids unnecessary energy waste. Dynamic energy assessment: By evaluating the drone's energy status in real time, its endurance can be predicted, ensuring that flight is not interrupted due to insufficient energy during mission execution. Path planning enables effective decision-making in the face of environmental uncertainties, such as dynamic obstacles and target position changes, enabling the drone to navigate safely and efficiently in complex and unstable environments. By considering multiple factors such as energy consumption, mission completion probability, and collision risk, the path with the highest expected return is calculated, thereby improving the efficiency and success rate of mission completion. Dynamic adjustment of the planned path based on real-time environmental feedback ensures that the drone can quickly respond to changes during flight and maintain its mission objectives. Deep reinforcement learning algorithms, by continuously learning from historical data and real-time feedback, enable the drone to autonomously optimize its flight strategy, improving the intelligence of path planning. This capability enables the drone to make more efficient decisions when faced with complex environments and changing missions.

[0153] As used herein, the word "preferred" is intended to serve as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word "preferred" is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any of the naturally inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing examples.

[0154] Moreover, although the present disclosure has been shown and described with respect to one or implementation, those skilled in the art will think of equivalent variations and modifications based on reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if structurally different from the disclosed structure that performs the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that can be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0155] The functional units in the embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or multiple or more units may be integrated into a single module. The aforementioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc. The aforementioned devices or systems may execute the storage method in the corresponding method embodiment.

[0156] In summary, the above embodiment is one implementation method of the present invention, but the implementation method of the present invention is not limited to the described embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A UAV path planning method under energy constraints, characterized by: The steps include: Step 1: Obtain environment and drone information and use probability map to update in real time; Step 2: Considering the task requirements and energy constraints, multiple possible paths are generated, the energy consumption of each path is estimated, and the optimal initial path is selected; Step 3: Based on the optimal initial path, learn and dynamically adjust the flight path according to the feedback from the probability map; Step 4: Continuously monitor the drone's energy level and adjust the drone's mission or return path in real time; The real-time updating using the probability map includes the following steps: Initialize the map. The map is represented by a grid map or a continuous space, where each grid or area contains an initial probability indicating whether the location is occupied. Collect surrounding environment data in real time through sensors as the basis for environmental perception; Based on the sensor's measurement value, the probability of whether there is an obstacle at each position is updated through the Bayesian update rule; As the drone moves and environmental information is updated, the probability map is continuously adjusted and updated; The method of learning and dynamically adjusting the flight path based on the feedback from the probability map includes the following steps: Define the state and action space. The state space S includes: the current position information of the drone (x, y, z), the current flight speed v, the current flight direction θ, the remaining energy E remaining , environmental information includes obstacle location, wind speed, terrain height, environmental characteristics Env, and the state is represented by a vector s t =(x t ,y t ,z t ,v t ,θ t ,E remaining ,Env t ); the action space A includes speed adjustment, direction adjustment, and height adjustment, and each action is represented by a vector a t =(a v ,a θ ,a h ), a v Indicates adjusting the flight speed, a θ Indicates adjusting the flight direction, a h Indicates adjusting the flight altitude; Define the reward function, which includes energy efficiency reward, safety reward, task completion reward, and energy efficiency reward R E (s, a), assuming that the energy consumed by executing action a in state s is E(s, a), then the energy reward can be designed to be a negative value, indicating that the greater the energy consumption, the lower the reward, R E (s,a)=-α E E(s,a), where α E is the weight coefficient of energy consumption, E(s,a) is the energy consumption required to perform action a; safety reward R S (s,a), based on the distance d between the drone and the obstacle obs (s) are designed, the closer the distance, the lower the reward. Among them, α S is the safety weight coefficient, ∈ is a small positive number used to prevent division by zero errors; the task completion reward R T (s), when the drone successfully reaches the target point, it is given a high reward R T (s) = β T , indicating that the task is completed; the comprehensive reward function, the total reward R(s,a) for the drone to perform action a in state s is expressed as the weighted sum of the above rewards, R(s,a)=R E (s,a)+R S (s,a)+R T (s); State transfer and update, the drone is in state s t Next, perform action a t After that, based on the state transition equation of the dynamic model and the environment, it enters the next state s t+1 , the state transition model is expressed as: s t+1 =f(s t ,a t )+ω t , where f(s t ,a t ) is the state transition function of the drone, which describes the state change after performing an action in the current state, ω t is the process noise, which represents the influence of environmental uncertainty; Energy consumption calculation, in each state s t Next, perform action a t When calculating the energy consumption E(s t ,a t ); Strategy update and optimization, using reinforcement learning to maximize cumulative rewards through strategy updates; Through reinforcement learning, the speed, direction and altitude of the drone are autonomously adjusted to achieve the optimal path planning goal.

2. The method for UAV path planning under energy constraints according to claim 1, characterized in that: The Bayesian update rule is used to update the probability of whether there is an obstacle at each position, which includes the following steps: when the UAV is flying, the sensor collects the obstacle information corresponding to a certain grid m. i Related measurement value z t , the measured value z t Indicates that at time t, grid m i Whether it is occupied or not, at this time, the probability of each grid is updated using the Bayesian formula, and the Bayesian update formula is: Among them, m i Indicates the position of the i-th grid, s i Indicates the state of the i-th grid, whether it is occupied, P(s i |z t ) indicates that the sensor observes z t After that, grid m i The posterior probability of being occupied, P(z t |s i ) represents a given grid m i The sensor observes z t The probability of i ) represents the grid m i Prior probability of being occupied, P(z t ) indicates that the sensor observes z t The edge probability is obtained by weighted calculation of all possible grid states.

3. The method for UAV path planning under energy constraints according to claim 1, characterized in that: The method of considering task requirements and energy constraints, generating multiple possible paths, estimating the energy consumption of each path, and selecting the optimal initial path includes the following steps: Using a probabilistic path search algorithm, the state space is defined as a set of nodes in a grid map. State transition is the process of moving from one grid to the next. The goal is to find the path from the initial position x0 to the target position x g Multiple paths of The cumulative risk of a path is estimated by the grid occupancy probability in the probability map. The risk R(π) of a path π is expressed as the sum of the occupancy probabilities of all grids passed by the path: Among them, P(s i =1) indicates grid m i The probability of being occupied by an obstacle, i represents a grid on the path π; Calculate the energy consumption of flight distance and speed. The energy consumption per unit time of the UAV on the path is expressed as flight power P, which is proportional to the flight speed v and the flight distance d. The energy consumption E(π) of path π is calculated by the following formula: Among them, P(v i ) represents the flight power of the UAV in the i-th segment on the path, which depends on the flight speed v i , represents the flight time of the i-th segment, d i is the flight distance between two locations on the path, and n represents the number of segments of the path; Calculate the energy consumption of terrain and wind speed. When considering the influence of terrain, the energy consumption is calculated by adding the vertical height change h caused by the terrain. i Make corrections: Where γ is a constant related to the weight of the drone, h i represents the height difference on the i-th path, and the wind speed w i The effect of is taken into account by correcting the flight power: P(v i ,w i )=P(v i )·(1+αw i ) Among them, α is the coefficient of wind speed on flight power, w i represents the wind speed of the i-th path; When the energy consumption E(π) and risk R(π) of multiple paths are calculated, an optimization algorithm is used to select the optimal path, and a comprehensive objective function J(π) is defined to evaluate the pros and cons of each path: J(π)=λ1E(π)+λ2R(π) Among them, λ1 and λ2 are the weight coefficients of energy and risk, respectively, reflecting the priority of the task for different goals. By adjusting the weights, the system finds a balance between energy saving and risk avoidance.

4. The method for UAV path planning under energy constraints according to claim 1, characterized in that: The method of considering task requirements and energy constraints, generating multiple possible paths, estimating the energy consumption of each path, and selecting the optimal initial path includes the following steps: Step 201, initialize a tree structure T = (V, E), where V is a node set, initially including the starting point V = {x start }, E is the set of edges, initially empty; set the initial position of the drone to x start and the target position is x goal , build a probability map Each position m in the map i The probability of including obstacles P(s i ), the map is used in the path evaluation process to determine whether it is safe to pass through a certain area, and to initialize the energy constraint condition E max , represents the maximum available energy of the UAV; Step 202: randomly sample a point x from the space rand , the point is within the range of the configuration space, which is defined as the flight area of ​​the UAV; Step 203, find the distance x in the current tree T rand The nearest node x nearest , the distance calculation formula is as follows: ‖·‖ represents the Euclidean distance; Step 204, from x nearest To x rand Expand and generate a new node x new , the new node is at x nearest to x rand In the direction of , and expand with a step size Δd: The new node x after expansion new Add the node set V of the tree and add the edge (x nearest ,x new )Add edge set E; Step 205, when expanding the tree, check nearest to x new Whether the path passes through the obstacle area in the probability map, calculate the risk of the path, the risk R(x nearest ,x new ) is estimated by the occupancy probability of the grid on the path: Among them, π(x nearest ,x new ) means from x nearest to x new All grids passed by on the path, P(s i =1) is the grid s i The probability of being occupied by obstacles; if the cumulative risk of the path exceeds the preset risk threshold, the extension is discarded and x is not new Add to the tree; Step 206: For each newly generated path segment, consider the flight distance and speed and evaluate the energy consumption. The energy consumption E(x nearest ,x new ) is: Where P(v) is the flight power corresponding to the flight speed v, ‖x new -x nearest ‖ is the distance of the path segment, v is the flight speed, if the cumulative energy consumption of the path exceeds the maximum energy E of the drone max , then stop expanding the path; Step 207, repeat steps 202 to 206, and continue to expand the tree until the connection starting point x is found start and the target point x goal A feasible path of x, or reaching the preset maximum number of iterations; generate multiple potential paths, each path corresponds to start to x goal The energy consumption and risk of each path on a tree are calculated and recorded during the expansion process; Step 208: Select the optimal path that meets the energy constraint from among the generated multiple paths. The path is selected using an optimization function that comprehensively considers energy consumption E(π) and risk R(π): J(π)=λ1E(π)+λ2R(π) Where E(π) represents the total energy consumption of path π, R(π) represents the risk of the path, and λ1 and λ2 are the weight coefficients of energy consumption and risk, which are adjusted according to task requirements.

5. The method for UAV path planning under energy constraints according to claim 4, characterized in that: In step 206, considering the influence of terrain height change h and wind speed w, the energy consumption of the path segment is corrected to: Where: γ1 is the coefficient of altitude change on energy consumption; α is the coefficient of wind speed on energy consumption.

6. The method for UAV path planning under energy constraints according to claim 3 or 4, characterized in that: When considering the flight distance and speed, the energy consumption is calculated as follows: P(v) = c1v 3 +c2v 2 +c3v+c4, where c1, c2, c3, c4 are constants related to the drone model, c1v 3 represents aerodynamic drag, c2v 2 represents propulsion power, c3v represents mechanical loss, and c4 represents fixed power loss.

7. The method for UAV path planning under energy constraints according to claim 6, characterized in that: The energy consumption mentioned above also takes into account the energy consumption of attitude adjustment, including steering energy consumption and energy consumption of acceleration and deceleration. During the steering process of the UAV in the path, due to the effect of inertial force, the additional energy consumption is related to the steering angle θ: E turn (θ)=k turn ·θ, where E turn (θ) represents the energy consumption when the steering angle is θ, k turn is a constant related to the UAV model and flight dynamics, representing the energy consumption coefficient of steering. If the UAV needs to accelerate or decelerate on the path, additional power is required to overcome the change in speed. Assuming that the speed of the UAV changes from v1 to v2 between the path segment x1 and x2, the energy consumption is expressed as: Among them, E acc / dec is the additional energy consumption during the speed change, m is the mass of the UAV, and v1 and v2 are the velocities at the beginning and end of the path segment, respectively.

8. The method for UAV path planning under energy constraints according to claim 7, characterized in that: The method of using reinforcement learning to maximize the cumulative reward through strategy update includes the following steps: Initialize the parameters θ, set the learning rate α, discount factor γ, entropy regularization coefficient β, and trust region limit step size δ; Sampling generation path, from strategy π θ (a|s) Sampling generates multiple flight paths, recording states, actions, rewards, and state transitions; Calculate the advantage function A(s t ,a t ), using a generalized advantage-based estimation method to reduce variance; Update the policy gradient and update the policy parameters under the trust region constraint: J(θ) is the expected cumulative reward; Adding entropy regularization term to encourage the exploration of the strategy and prevent premature convergence to suboptimal solutions; Strategy update and iteration, repeat the above steps until the strategy converges, and finally obtain the optimized path planning strategy; The advantage function A(s t ,a t ) represents the advantage of the current action compared with the expected action in the strategy, A(s t ,a t )=Q(s t ,a t )-V(s t ), where Q(s t ,a t ) is the state action value function, indicating that in state s t Next, perform action a t The cumulative reward, V(s t ) is the state value function, indicating that in state s t The expected cumulative reward; The generalized odds estimate The calculation formula is: Among them, δ t is the temporal difference error, and λ is a parameter that weighs the bias and variance, which is between [0,1].

9. The method for UAV path planning under energy constraints according to claim 1, characterized in that: The continuous monitoring of the drone's energy level and real-time adjustment of the drone's mission and return path include the following steps: during flight, the drone's energy level is continuously monitored and the path planning is adjusted in real time. When the energy approaches the warning threshold, the shortest path back to the charging station is calculated, and the drone is triggered to return to the charging station when necessary. If the drone's energy is insufficient to complete the current mission, other drones are notified to take over the mission, or the formation is coordinated to adapt to the current energy conditions.

Images