Unmanned aerial vehicle path planning method, equipment and medium

Abstract

The invention discloses an unmanned aerial vehicle path planning method and device and a medium, and the method comprises the steps: carrying out the rasterization of environment image information obtained by an unmanned aerial vehicle, building a raster map, and determining a current position, obstacle information and a target point position in the raster map; constructing a Q table based on the current position, the obstacle information and the target point position; selecting a specified action and a corresponding state in the Q table based on a self-adaptive greedy mechanism; obtaining path smoothness information in the grid map and height information of the unmanned aerial vehicle; performing fuzzification processing based on the current position, the target point position, the path smoothness information and the height information of the unmanned aerial vehicle to obtain a reward value; updating a Q value in a Q table based on the reward value, the specified action and the corresponding state; and performing iterative execution based on the updated Q table to obtain an iterative Q table, and determining an optimal path based on the iterative Q table.

Description

Technical Field

[0001] This specification relates to the field of path planning technology, and in particular to a method, device, and medium for unmanned aerial vehicle path planning. Background Art

[0002] As a crucial component of mobile UAV navigation systems, path planning essentially involves finding an optimal or near-optimal feasible route from a starting point to a destination, within specific spatiotemporal constraints, using an algorithm to achieve specific mission objectives. The autonomous planning capabilities of path planning algorithms determine whether mobile UAVs can autonomously complete various complex missions. This plays a crucial role in the design and application of mobile UAVs and represents one of the key technical bottlenecks hindering the development of intelligent UAVs.

[0003] Traditional drone path planning methods primarily include single algorithms such as graph search-based algorithms (e.g., A*, Dijkstra) and sampling-based algorithms (e.g., RRT, RRT*), which can lead to inaccurate path planning. Therefore, a more accurate drone path planning method is needed to better meet user needs. Summary of the Invention

[0004] One or more embodiments of this specification provide a method, device, and medium for drone path planning to solve the technical problems raised by the background technology.

[0005] One or more embodiments of this specification adopt the following technical solutions:

[0006] One or more embodiments of this specification provide a method for drone path planning, the method comprising:

[0007] S1: rasterizing the environmental image information obtained by the UAV to create a raster map, and determining the current position, obstacle information and target point position on the raster map;

[0008] S2: Based on the current position, the obstacle information, and the target point position, a Q table is constructed. The Q table is a data structure used to track and record states, actions, and expected rewards. Each value in the Q table corresponds to a Q value for a state and action pair.

[0009] S3: Select a specified action and corresponding state in the Q table;

[0010] S4: Obtaining path smoothness information in the grid map and altitude information of the UAV;

[0011] S5: performing fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the drone to obtain a reward value;

[0012] S6: updating the Q value in the Q table based on the reward value, the designated action and the corresponding state;

[0013] S7: Iteratively execute steps S3, S4, S5, and S6 based on the updated Q-table to obtain an iterative Q-table, and determine the optimal path based on the iterative Q-table.

[0014] It should be noted that traditional UAV path planning relies on a single algorithm (such as A* only considers the geometric shortest path, RRT focuses on random exploration), which makes it difficult to simultaneously optimize multiple objectives such as path efficiency, flight stability and high safety in complex environments, resulting in planning results that may deviate from the actual mission intention (such as steep turns increase energy consumption, low-altitude flight endangers safety); and this method integrates the reinforcement learning framework and the fuzzy reasoning mechanism. Based on the grid map established by S1, the Q table constructed by S2 associates the environmental state and the action value, and through the fuzzy processing of S5, the core innovation is to combine the target proximity (current position and target point position), path Heterogeneous metrics such as smoothness and altitude safety constraints are converted into unified semantic rules (e.g., "rewards are suppressed if altitude is too low" and "penalties are triggered if path curvature changes suddenly"). These rewards drive the Q-value updates in S6, enabling the reinforcement learning iterations (S3-S7) to no longer rely solely on geometric distance information. Instead, the reward function reflects multidimensional task demands in real time. For example, when a drone encounters a low-altitude obstacle, altitude information significantly reduces the reward for risky maneuvers through fuzzy rules, forcing the Q-table to prioritize climbing over the shortest path in subsequent iterations. Furthermore, the weighting of rewards by path smoothness suppresses the cumulative value of sharp turns, naturally generating a smoother route. This mechanism ultimately generates the optimal path in S7, essentially mathematically encapsulating and continuously reinforcing the human expert experience (e.g., the priority logic of safety > efficiency > energy consumption) pre-set in the fuzzy rule base through the reward function. This allows the system to autonomously generate a globally optimized path that combines goal-directedness, flight safety, and operational stability in complex environments. This effectively addresses the problem of traditional methods, where planning results deviate from user needs due to the fragmented handling of multiple objectives.

[0015] Furthermore, constructing a Q table based on the current position, the obstacle information and the target point position includes:

[0016] Determining a first distance between the current position and the target position based on the current position and the target point position;

[0017] determining a second distance between the current position and the obstacle based on the current position and the obstacle information;

[0018] determining the effective range of the obstacle based on the obstacle information;

[0019] A Q table is constructed based on the first distance, the second distance, and the range of the obstacle.

[0020] It is important to note that this method integrates the three-dimensional dynamic parameters of the first distance (the distance between the current position and the target point), the second distance (the distance between the current position and the obstacle), and the obstacle range into the Q-table construction. This integrates goal-directedness (the first distance drives convergence toward the target), obstacle avoidance safety (the second distance quantifies collision risk), and environmental dynamics (the obstacle range characterizes the potential threat area) into a unified decision-making framework. This enables the Q-table to quantify the comprehensive benefits of different actions in real time during reinforcement learning. When the drone approaches the target, the shortening of the first distance increases the positive reward. However, if it simultaneously enters the obstacle range, the reduction of the second distance triggers a negative penalty, forcing the algorithm to automatically balance the conflicting needs of approaching the target and avoiding dynamic obstacles during path selection. This dynamic coupling mechanism enables the Q-table to not only adapt to real-time changes in obstacle positions (for example, a sudden decrease in the second distance triggers a strategy adjustment when a moving vehicle suddenly approaches), but also predict threat areas based on the obstacle range (for example, a pre-set buffer around temporary buildings). This allows the algorithm to generate paths that balance efficiency and safety in complex dynamic environments, effectively solving the path rigidity problem caused by static discrete modeling in traditional grid methods.

[0021] Furthermore, the range of action based on the first distance, the second distance, and the obstacle includes:

[0022] Based on the formula with U all =U att +U rep , construct the Q table, where ρ(q,q g ) is the distance between the current position of the UAV and the target point, ρ(q,q o ) is the distance between the current position of the UAV and the obstacle. ξ and η are the attraction factor and repulsion factor respectively, and ρ0 is the range of the obstacle.

[0023] Furthermore, the selecting of a specified action and a corresponding state in the Q table based on the adaptive greedy mechanism includes:

[0024] Based on a pre-set adaptive greedy strategy, a specified action and a corresponding state are selected in the Q table.

[0025] It should be noted that the fixed exploration-exploitation strategy in traditional path planning is difficult to cope with the uncertainty brought by dynamic obstacles and complex terrain, and often leads to a decrease in path reliability due to over-conservatism (trapping into a local optimal path) or blind exploration (increasing the risk of collision). However, this method dynamically adjusts the action selection mechanism through an adaptive greedy strategy: when the drone is in a known safe area (for example, the Q table shows that there is a high-reward path in a certain direction), the strategy automatically increases the "exploitation" probability to quickly approach the target along the verified path, reducing the energy consumption of ineffective exploration; when the sensor detects a sudden change in the environment (for example, a new obstacle causes the historical Q value to become invalid) or the path reward continues to fluctuate, the strategy immediately increases the "exploration" probability and actively tries new actions such as detours and climbs to update environmental cognition. This self-regulating capability enables the system to inherit the obstacle avoidance experience accumulated by the Q table in the decision-making process of S3 (such as using a wall-flying strategy in densely built-up areas), while also maintaining responsiveness to sudden threats (such as quickly exploring emergency climbing maneuvers when encountering a moving vehicle). This allows the drone to always maintain the adaptability and stability of its strategy during the iterative learning process - avoiding path interruptions caused by environmental changes, reducing task delays caused by meaningless exploration, and fundamentally improving its continuous operation capabilities in complex scenarios.

[0026] Furthermore, based on a pre-set adaptive greedy strategy, selecting an action and a corresponding state in the Q table includes:

[0027] Based on the adaptive greedy strategy Select the specified action and corresponding state in the Q table, where ε(n+1) is the latest greed factor, ε(n) is the greed factor of the previous iteration, n is the number of iterations, ε max The preset maximum greed value.

[0028] Furthermore, the fuzzy processing is performed based on the current position, the target point position, the path smoothness information, and the altitude information of the drone to obtain a reward value, including:

[0029] Performing fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the UAV, and performing reasoning using a fuzzy rule base to obtain a fuzzy output;

[0030] The fuzzy output is defuzzified to obtain the reward value.

[0031] It should be noted that traditional path planning methods often struggle to adapt to the uncertainty of dynamic environments due to the use of rigid thresholds or linear weighting strategies when processing multi-source heterogeneous environmental information (such as path smoothness and altitude constraints). However, this method utilizes a fuzzy processing mechanism to transform multidimensional indicators such as the distance between the current position and the target, obstacle avoidance requirements, flight altitude restrictions, and path curvature into fuzzy sets (such as "near / far," "safe / dangerous," and "smooth / tortuous"). This method then simulates human decision-making and performs nonlinear reasoning based on a fuzzy rule base. When the UAV approaches the target but path smoothness decreases, the fuzzy rule base dynamically balances the priorities of approaching the target and maintaining flight stability based on expert experience (e.g., "If the distance to the target is close and altitude is safe, a moderate increase in path curvature is allowed for faster arrival"). At the same time, altitude information is incorporated into fuzzy reasoning to avoid low-altitude obstacles or airspace restrictions. The defuzzification process transforms this multi-objective collaborative decision-making into a single reward value, enabling the reinforcement learning Q-table update to not only reflect the physical feasibility of the path but also incorporate implicit constraints such as flight safety and energy consumption control. This mechanism enables the reward function to have semantic-level environmental understanding capabilities, which can not only handle uncertainties such as sensor noise and dynamic changes in obstacles, but also achieve multi-objective optimization trade-offs through the flexible logic of the fuzzy rule base. It can be deduced that the path generated by the UAV in complex scenarios not only meets geometric connectivity, but also can adaptively balance the competing demands of mission efficiency, flight safety and operational stability, significantly improving the system's cognition and decision-making robustness in unknown dynamic environments.

[0032] Furthermore, fuzzy processing is performed based on the current position, the target point position, the path smoothness information, and the altitude information of the UAV, including: determining distance information between the current position and the target point position based on the current position and the target point position; and converting the distance information into a membership function of the distance information: Among them, μ F (L), μ M (L), μ N (L) are the membership functions of the distance information L, the preset long distance, the preset medium distance, and the preset short distance, respectively, d max is the maximum distance between all path points and the target point; the path smoothness information is converted into the membership function of the path smoothness information:

[0033]

[0034] Among them, ν T (θ), ν M (θ), ν S(θ) is the path smoothness information θ, which is the membership function of preset tortuous, preset medium smooth, and preset smooth; the altitude information of the UAV is converted into the membership function of altitude information:

[0035]

[0036] Among them, ω L (η), ω M (η), ω H (η) are the height information η is the membership function of the preset low height, preset medium height, and preset high height, h max It is the maximum altitude at which the drone is preset to fly.

[0037] Furthermore, defuzzifying the fuzzy output to obtain the reward value includes:

[0038] By defuzzifying the formula Defuzzifying the fuzzy output to obtain the reward value;

[0039] Among them, R is the reward value obtained after defuzzification, φ i is the membership degree corresponding to the i-th rule in the fuzzy rule base, O i is the output value corresponding to the i-th rule in the fuzzy rule base.

[0040] It should be noted that traditional defuzzification methods (such as the center of gravity method or the maximum membership method) tend to lead to decision rigidity when converting fuzzy reasoning results into precise reward values ​​due to ignoring the rule weights and situational relevance. This method, on the other hand, constructs a dynamic reward generation mechanism through a weighted defuzzification formula based on rule membership and rule output value: each rule in the fuzzy rule base corresponds to expert experience under a specific environmental state (such as "prioritize lifting when the altitude is too low" or "penalize curvature when the path turns sharply"). When multiple rules are activated at the same time due to environmental complexity, the membership quantifies the applicability of each rule in the current state (such as the membership of the obstacle avoidance rule increases sharply when an obstacle suddenly approaches). The defuzzification process integrates the output values ​​of different rules by weight, so that the reward value inherits both high membership and low membership. The dominant decision of the high-degree rules (such as strengthening negative penalties in emergency obstacle avoidance) and the auxiliary correction of low-membership rules (such as maintaining path smoothness rewards during stable flight) are absorbed. This mechanism enables the Q-table update of reinforcement learning to dynamically reflect the multi-dimensional constraints of the environment (safety, efficiency, energy consumption) and their priority changes. For example, when the drone passes through a dense obstacle area, the membership of the obstacle avoidance-related rules is cumulatively improved, driving the reward function to focus on safety; while in open areas, it automatically switches to the dominance of the target approach rule, thereby deducing that the path strategy has context-awareness, which can not only respond to the immediate intervention needs of sudden threats, but also maintain the global optimality of long-term mission goals, and ultimately achieve the unity of multi-objective collaborative optimization and strategy stability in a dynamic uncertain environment.

[0041] One or more embodiments of this specification provide a drone path planning device, including:

[0042] At least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the following steps: S1: rasterizing the environmental image information obtained by the drone, establishing a raster map, and determining the current position, obstacle information, and target point position in the raster map; S2: constructing a Q table based on the current position, the obstacle information, and the target point position, wherein the Q table is used to track and record the data structure of the state, action, and expected reward, and ... Each value corresponds to a Q value of a state and action pair; S3: select a specified action and corresponding state in the Q table; S4: obtain the path smoothness information in the grid map and the altitude information of the drone; S5: perform fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the drone to obtain a reward value; S6: update the Q value in the Q table based on the reward value, the specified action and the corresponding state; S7: iteratively execute steps S3, S4, S5 and S6 based on the updated Q table to obtain an iterative Q table, and determine the optimal path based on the iterative Q table.

[0043] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, which can implement the following steps when executed by a computer: S1: rasterizing the environmental image information obtained by the drone to create a raster map, and determining the current position, obstacle information, and target point position on the raster map; S2: constructing a Q table based on the current position, the obstacle information, and the target point position, wherein the Q table is used to track and record the data structure of the state, action, and expected reward, and each value in the Q table corresponds to a state and an action. S3: select a specified action and a corresponding state in the Q table; S4: obtain the path smoothness information in the grid map and the altitude information of the UAV; S5: perform fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the UAV to obtain a reward value; S6: update the Q value in the Q table based on the reward value, the specified action and the corresponding state; S7: iteratively execute steps S3, S4, S5 and S6 based on the updated Q table to obtain an iterative Q table, and determine the optimal path based on the iterative Q table.

[0044] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0045] Traditional UAV path planning relies on a single algorithm (such as A* only considers the geometric shortest path, RRT focuses on random exploration), which makes it difficult to simultaneously optimize multiple objectives such as path efficiency, flight stability and high safety in complex environments, resulting in planning results that may deviate from the actual mission intention (such as steep turns increase energy consumption, and low-altitude flight endangers safety); this method integrates the reinforcement learning framework and the fuzzy reasoning mechanism. Based on the grid map established by S1, the Q table constructed by S2 associates the environmental state and the action value, and through the fuzzy processing core innovation of S5 - the target approach degree (current position and target point position), path smoothness, Heterogeneous metrics, such as altitude safety constraints, are converted into unified semantic rules (e.g., "rewards are suppressed if altitude is too low" and "penalties are triggered if path curvature changes suddenly"). These rewards drive the Q-value updates in S6, enabling the reinforcement learning iterations (S3-S7) to no longer rely solely on geometric distance information. Instead, the reward function reflects multidimensional task demands in real time. For example, when a drone encounters a low-altitude obstacle, altitude information significantly reduces the reward for risky maneuvers through fuzzy rules, forcing the Q-table to prioritize climbing over the shortest path in subsequent iterations. Furthermore, the weighting of reward values ​​by path smoothness suppresses the cumulative value of sharp turns, naturally generating a smoother route. This mechanism ultimately generates the optimal path in S7, essentially mathematically encapsulating and continuously reinforcing the human expert experience (e.g., the priority logic of safety > efficiency > energy consumption) pre-set in the fuzzy rule base through the reward function. This allows the system to autonomously generate a globally optimized path that combines goal-orientation, flight safety, and operational stability in complex environments. This effectively addresses the problem of traditional methods, where planning results deviate from user needs due to the fragmented handling of multiple objectives. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some of the embodiments described in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without inventive work. In the drawings:

[0047] Figure 1 A flowchart of a drone path planning method provided in one or more embodiments of this specification;

[0048] Figure 2 A schematic diagram of a membership function for distance information provided in one or more embodiments of this specification;

[0049] Figure 3 A schematic diagram of a membership function for path smoothness information provided in one or more embodiments of this specification;

[0050] Figure 4 A schematic diagram of a membership function for height information provided in one or more embodiments of this specification;

[0051] Figure 5 An experimental trajectory diagram under a 10x10 grid map provided for one or more embodiments of this specification;

[0052] Figure 6 This is an algorithm trajectory diagram for an environment with an obstacle ratio of 0.1 provided in one or more embodiments of this specification;

[0053] Figure 7 Algorithm trajectory diagram for an environment with an obstacle ratio of 0.2 provided in one or more embodiments of this specification;

[0054] Figure 8 A diagram showing the iterative results of the Q-Learning algorithm in a three-dimensional environment provided by one or more embodiments of this specification;

[0055] Figure 9 A diagram showing the iterative results of the IFQ-Learning algorithm in a three-dimensional environment provided by one or more embodiments of this specification;

[0056] Figure 10 A flowchart of an improved Q-Learning path planning algorithm based on fuzzy logic provided in one or more embodiments of this specification;

[0057] Figure 11 A schematic diagram of the structure of a drone path planning device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0058] The embodiments of this specification provide a method, device, and medium for drone path planning.

[0059] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative work should fall within the scope of protection of this specification.

[0060] Figure 1 This is a flowchart of a method for drone path planning provided in one or more embodiments of this specification. This process can be executed by a drone path planning system. Certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0061] The method steps of the embodiment of this specification are as follows:

[0062] S1: Perform rasterization processing on the environmental image information obtained by the UAV, establish a raster map, and determine the current position, obstacle information and target point position on the raster map.

[0063] In the embodiments of this specification, a drone-mounted visual or laser sensor acquires an image of the environment and then divides the continuous environment into uniform grid cells, with each grid cell labeled as a free area, an obstacle, or a target area. An image segmentation algorithm is used to identify obstacle outlines, and target detection technology is combined to locate the drone's current position and target coordinates, ultimately generating a grid map that includes the location, obstacle distribution, and target points.

[0064] S2: Based on the current position, the obstacle information, and the target point position, a Q table is constructed. The Q table is used to track and record the data structure of the state, action, and expected reward. Each value in the Q table corresponds to the Q value of a state and action pair.

[0065] In the embodiments of this specification, the environmental state can be abstracted into a unique identifier (e.g., a combination of coordinates and obstacle distances) based on the drone's current position in a grid map, the distance to nearby obstacles, and the orientation of the target point. The action set is defined as the drone's optional movement directions (e.g., forward, backward, left, right, etc.). The Q table is initialized as a two-dimensional matrix, with rows representing all possible states and columns representing all actions. The initial Q values ​​are set to a preset empirical value or zero, forming a record of the expected benefits of each state-action pair.

[0066] S3: Select a specified action and a corresponding state in the Q table.

[0067] In the embodiments of this specification, an adaptive greedy strategy can be used in conjunction with the Q-value distribution of the current state in the Q-table: a dynamic probability is used to select the highest Q-value action (using the known optimal path) or a random action (exploring new paths). The exploration probability is automatically adjusted based on the number of iterations, path convergence, or environmental complexity (such as obstacle density). For example, a high exploration rate is used in the early stages to cover unknown areas, while a bias is used to use verified paths later.

[0068] S4: Obtain path smoothness information in the grid map and altitude information of the UAV.

[0069] In an embodiment of the present specification, altitude sensor data can be obtained in real time from the drone flight log, and the path smoothness (such as the mean angle between adjacent path segments or the curvature change) can be calculated through the turning point sequence of the planned path in the raster map to form a quantitative index of altitude constraint and path smoothness, which serves as an input parameter for subsequent reward calculation.

[0070] S5: Perform fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the drone to obtain a reward value.

[0071] In the embodiments of this specification, a fuzzy inference system can be designed, defining the distance between the current position and the target, obstacle proximity, path smoothness, and altitude deviation as input variables. Using a pre-defined fuzzy rule base (e.g., "If the distance to the target is close and the height is safe, the reward increases"), semantic inference is performed to output a fuzzy reward level. The defuzzification process weights and fuses the results of multiple rule activations to convert them into a precise reward value, reflecting the overall benefit of the current action.

[0072] S6: Update the Q value in the Q table based on the reward value, the designated action and the corresponding state.

[0073] In the embodiment of this specification, after executing the selected action according to the current state, the Q value of the corresponding state-action pair in the Q table can be adjusted according to the reinforcement learning update rule in combination with the reward value generated by S5: if the action leads to a higher reward, its Q value is increased to strengthen the strategy; otherwise, the Q value is reduced to suppress inefficient behavior, and the action preference of the drone in a specific state is gradually optimized.

[0074] It should be noted that it can be based on the formula:

[0075] Update the Q value in the Q table. Among them, (s t ,a t ) is the state and action pair at the current moment, Q(s t ,a t ) is the current state (s t ,a t ) corresponds to the Q value, Q(s t+1 ,a t+1 ) is the state at the next moment (s t+1 ,a t+1 ) corresponding to the Q value, r t is the reward value obtained for the current state and action, and α and γ are discount factors.

[0076] S7: Iteratively execute steps S3, S4, S5, and S6 based on the updated Q-table to obtain an iterative Q-table, and determine the optimal path based on the iterative Q-table.

[0077] In the embodiments of this specification, S3 through S6 can be repeated, with each iteration updating the Q-table and recording the path trajectory. When the Q-table update amplitude falls below a threshold or reaches a preset number of iterations, the learning process is terminated. The Q-table with the highest cumulative reward, successful obstacle avoidance, and continuous path is selected from the historical Q-tables. The state-action sequence of this Q-table is then traced back to generate the final flight path, ensuring global optimality and dynamic adaptability of the path.

[0078] It should be noted that traditional UAV path planning relies on a single algorithm (such as A* only considers the geometric shortest path, RRT focuses on random exploration), which makes it difficult to simultaneously optimize multiple objectives such as path efficiency, flight stability and high safety in complex environments, resulting in planning results that may deviate from the actual mission intention (such as steep turns increase energy consumption, low-altitude flight endangers safety); and this method integrates the reinforcement learning framework and the fuzzy reasoning mechanism. Based on the grid map established by S1, the Q table constructed by S2 associates the environmental state and the action value, and through the fuzzy processing of S5, the core innovation is to combine the target proximity (current position and target point position), path Heterogeneous metrics such as smoothness and altitude safety constraints are converted into unified semantic rules (e.g., "rewards are suppressed if altitude is too low" and "penalties are triggered if path curvature changes suddenly"). These rewards drive the Q-value updates in S6, enabling the reinforcement learning iterations (S3-S7) to no longer rely solely on geometric distance information. Instead, the reward function reflects multidimensional task demands in real time. For example, when a drone encounters a low-altitude obstacle, altitude information significantly reduces the reward for risky maneuvers through fuzzy rules, forcing the Q-table to prioritize climbing over the shortest path in subsequent iterations. Furthermore, the weighting of rewards by path smoothness suppresses the cumulative value of sharp turns, naturally generating a smoother route. This mechanism ultimately generates the optimal path in S7, essentially mathematically encapsulating and continuously reinforcing the human expert experience (e.g., the priority logic of safety > efficiency > energy consumption) pre-set in the fuzzy rule base through the reward function. This allows the system to autonomously generate a globally optimized path that combines goal-directedness, flight safety, and operational stability in complex environments. This effectively addresses the problem of traditional methods, where planning results deviate from user needs due to the fragmented handling of multiple objectives.

[0079] Furthermore, when constructing the Q table based on the current position, the obstacle information, and the target point position, the first distance between the current position and the target position can be determined based on the current position and the target point position; the second distance between the current position and the obstacle can be determined based on the current position and the obstacle information; the range of the obstacle can be determined based on the obstacle information; and the Q table can be constructed based on the first distance, the second distance, and the range of the obstacle.

[0080] It should be noted that the Euclidean distance between the drone's current position and the target point can be calculated in real time as the first distance, reflecting the remaining path length. The grid map is simultaneously scanned for nearby obstacles, and the distance to the closest obstacle from the drone is extracted as the second distance to quantify the collision risk. Furthermore, based on the obstacle type (e.g., moving vehicle, fixed building), the range of action (e.g., vehicle radius + safety buffer distance) is preset or estimated in real time. The first distance is discretized into distance intervals (e.g., near / medium / far), and the second distance is divided into safety levels (e.g., dangerous / warning / safe). Combined with the obstacle range, a composite state identifier is generated (e.g., "medium distance to target - dangerous obstacle - large range"). Each unique identifier corresponds to a row of states in the Q-table. Furthermore, basic actions that the drone can perform (e.g., forward, left turn, climb, etc.) can be defined to form an action set. A two-dimensional Q-table matrix is ​​constructed, with state identifiers as rows and actions as columns.

[0081] If the second distance in the state is in the dangerous range, the Q value of the associated collision risk action (such as going straight) is set to a negative value; if the first distance is close to the target and the range of action is not threatening, the action of approaching the target is given a higher initial Q value.

[0082] When the environment changes (such as the movement of obstacles) causing the second distance or range to change, the state identifier is updated in real time, and the Q value of the original state is inherited or reset according to the new threat level to ensure that the Q table always reflects the current environmental situation.

[0083] It should be noted that traditional path planning methods are prone to failure in dynamic obstacle scenarios due to their reliance on static environment modeling. However, this method integrates the three-dimensional dynamic parameters of the first distance (the distance between the current position and the target point), the second distance (the distance between the current position and the obstacle), and the obstacle range when constructing the Q-table. It incorporates goal orientation (the first distance drives convergence to the target), obstacle avoidance safety (the second distance quantifies the collision risk), and environmental dynamics (the obstacle range characterizes the potential threat area) into a unified decision-making framework, allowing the Q-table to quantify the comprehensive benefits of different actions in real time during the reinforcement learning process: when the drone approaches the target, the shortening of the first distance increases the positive reward, but if it enters the obstacle range at the same time, the reduction of the second distance triggers a negative penalty, forcing the algorithm to automatically balance the conflicting needs of approaching the target and avoiding dynamic obstacles when selecting a path. This dynamic coupling mechanism enables the Q-table to not only adapt to real-time changes in obstacle positions (such as a sudden decrease in the second distance triggering strategy adjustment when a moving vehicle suddenly approaches), but also predict threat areas based on the range of obstacles (such as a preset buffer range around temporary buildings), thereby generating a path that balances efficiency and safety in complex dynamic environments. This can effectively solve the path rigidity problem caused by static discrete modeling in traditional grid methods.

[0084] Furthermore, the Q table is a data structure used to track and record states, actions, and their expected rewards. Each value in the table corresponds to the Q value of a state-action pair. In traditional Q learning algorithms, the Q table is initialized to all zeros or random values, and a large amount of trial and error is required to collect effective experience, and the convergence process is slow. In response to the above problems, the embodiment of this specification proposes the idea of ​​using an improved artificial potential field method to initialize the Q table. Based on the formula with U all =U att +U rep , construct the Q table, where ρ(q,q g ) is the distance between the current position of the UAV and the target point, ρ(q,q o ) is the distance between the current position of the UAV and the obstacle. ξ and η are the attraction factor and repulsion factor respectively, and ρ0 is the range of the obstacle.

[0085] Compared with the traditional artificial potential field method, the above content ensures that there is only one zero potential energy point globally, ensuring that the Q value distribution will not be unreasonable due to the local optimal point, thereby increasing the number of iterations.

[0086] From the perspective of the global map, as the map grows, the range of the total potential energy value also increases. If it is directly used in the iterative process of Q learning, it is easy to fall into the local optimal solution and ignore other potential paths. The reason is that although a better path is found through the greedy strategy, the Q value of the path point cannot become the maximum value among all actions in the current state. Therefore, the total potential energy value is normalized as follows:

[0087]

[0088] Furthermore, selecting a specified action and a corresponding state in the Q table includes:

[0089] Based on a pre-set adaptive greedy strategy, a specified action and a corresponding state are selected in the Q table.

[0090] It should be noted that the fixed exploration-exploitation strategy in traditional path planning is difficult to cope with the uncertainty brought by dynamic obstacles and complex terrain, and often leads to a decrease in path reliability due to over-conservatism (trapping into a local optimal path) or blind exploration (increasing the risk of collision). However, this method dynamically adjusts the action selection mechanism through an adaptive greedy strategy: when the drone is in a known safe area (for example, the Q table shows that there is a high-reward path in a certain direction), the strategy automatically increases the "exploitation" probability to quickly approach the target along the verified path, reducing the energy consumption of ineffective exploration; when the sensor detects a sudden change in the environment (for example, a new obstacle causes the historical Q value to become invalid) or the path reward continues to fluctuate, the strategy immediately increases the "exploration" probability and actively tries new actions such as detours and climbs to update environmental cognition. This self-regulating capability enables the system to inherit the obstacle avoidance experience accumulated by the Q table in the decision-making process of S3 (such as using a wall-flying strategy in densely built-up areas), while also maintaining responsiveness to sudden threats (such as quickly exploring emergency climbing maneuvers when encountering a moving vehicle). This allows the drone to always maintain the adaptability and stability of its strategy during the iterative learning process - avoiding path interruptions caused by environmental changes, reducing task delays caused by meaningless exploration, and fundamentally improving its continuous operation capabilities in complex scenarios.

[0091] Furthermore, the Ant Colony Algorithm (ACO) is an optimization algorithm that simulates the foraging behavior of ants. This algorithm is based on the natural behavior of ants, which release pheromones to mark their paths and find the shortest route while searching for food. The ACO algorithm includes a state transition probability formula, a pheromone increment formula, and a pheromone update formula.

[0092] This application selects actions and corresponding states in the Q table based on a pre-set adaptive greedy strategy, including:

[0093] Based on the adaptive greedy strategy Select the specified action and corresponding state in the Q table, where ε(n+1) is the latest greed factor, ε(n) is the greed factor of the previous iteration, n is the number of iterations, ε max The preset maximum greed value.

[0094] In the ACO algorithm, the concentration of pheromones is proportional to the quality of the path. Ants tend to choose paths with higher pheromone concentrations, thus forming a positive feedback mechanism. The maximum-minimum ant strategy is an improved strategy of the ant colony algorithm. Its idea is to set a concentration range [τ min ,τ max ]. Among them, τ max and τ min They are the upper and lower bounds of pheromone concentration, respectively, to prevent the algorithm from falling into a local optimal solution due to large differences in pheromone concentration.

[0095] In the greedy strategy, ε (greediness value) is a fixed value. If the value of ε is too large, the algorithm will have difficulty converging in the later stages of the iteration; if the value of ε is too small, the algorithm will not be able to fully explore the environment in the early stages and will easily fall into a local minimum. Therefore, this paper proposes a maximum-minimum adaptive greedy strategy based on the idea of ​​the maximum-minimum ant strategy. By combining it with the number of iterations, it ensures that the algorithm can fully explore the environment in the early stages while achieving rapid convergence in the later stages. At the same time, the upper and lower bounds of the greed factor are set to ensure the completeness of the algorithm. The maximum-minimum adaptive greedy strategy is as follows:

[0096]

[0097] Furthermore, when the reward value is obtained by performing fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the drone, fuzzy processing can be performed based on the current position, the target point position, the path smoothness information, and the altitude information of the drone, and reasoning can be performed using a fuzzy rule base to obtain a fuzzy output; the fuzzy output is defuzzified to obtain the reward value.

[0098] It should be noted that traditional path planning methods often struggle to adapt to the uncertainty of dynamic environments due to the use of rigid thresholds or linear weighting strategies when processing multi-source heterogeneous environmental information (such as path smoothness and altitude constraints). However, this method utilizes a fuzzy processing mechanism to transform multidimensional indicators such as the distance between the current position and the target, obstacle avoidance requirements, flight altitude restrictions, and path curvature into fuzzy sets (such as "near / far," "safe / dangerous," and "smooth / tortuous"). This method then simulates human decision-making and performs nonlinear reasoning based on a fuzzy rule base. When the UAV approaches the target but path smoothness decreases, the fuzzy rule base dynamically balances the priorities of approaching the target and maintaining flight stability based on expert experience (e.g., "If the distance to the target is close and altitude is safe, a moderate increase in path curvature is allowed for faster arrival"). At the same time, altitude information is incorporated into fuzzy reasoning to avoid low-altitude obstacles or airspace restrictions. The defuzzification process transforms this multi-objective collaborative decision-making into a single reward value, enabling the reinforcement learning Q-table update to not only reflect the physical feasibility of the path but also incorporate implicit constraints such as flight safety and energy consumption control. This mechanism enables the reward function to have semantic-level environmental understanding capabilities, which can not only handle uncertainties such as sensor noise and dynamic changes in obstacles, but also achieve multi-objective optimization trade-offs through the flexible logic of the fuzzy rule base. It can be deduced that the path generated by the UAV in complex scenarios not only meets geometric connectivity, but also can adaptively balance the competing demands of mission efficiency, flight safety and operational stability, significantly improving the system's cognition and decision-making robustness in unknown dynamic environments.

[0099] Furthermore, when fuzzy processing is performed based on the current position, the target point position, the path smoothness information, and the altitude information of the UAV, the distance information between the current position and the target point position can be determined based on the current position and the target point position; the distance information is converted into a membership function of the distance information. The schematic diagram of the membership function of the distance information is shown as follows: Figure 2 As shown:

[0100] Among them, μ F (L), μ M (L), μ N (L) are the membership functions of the distance information L, the preset long distance, the preset medium distance, and the preset short distance, respectively, d max is the maximum distance between all path points and the target point;

[0101] The path smoothness information is converted into a membership function of the path smoothness information. The membership function diagram of the path smoothness information is as follows: Figure 3 As shown:

[0102]

[0103] Among them, ν T (θ), ν M (θ), ν S (θ) are the membership functions of path smoothness information θ, which are preset tortuous, preset moderately smooth, and preset smooth;

[0104] The altitude information of the UAV is converted into a membership function of the altitude information. The diagram of the membership function of the altitude information is as follows: Figure 4 As shown:

[0105]

[0106] Among them, ω L (η), ω M (η), ω H (η) are the height information η is the membership function of the preset low height, preset medium height, and preset high height, h max It is the maximum altitude at which the drone is preset to fly.

[0107] At the same time, the embodiments of this specification use the rules in the fuzzy rule base to analyze and reason the input parameters to obtain fuzzy output. The rules in the fuzzy rule base are shown in Table 1 Fuzzy Rules (Height Information = H), Table 2 Fuzzy Rules (Height Information = M), and Table 3 Fuzzy Rules (Height Information = L). The specific rules in the fuzzy rule base are:

[0108] Rule 1: If the height information is high, the distance information is long, and the path smoothness information is winding, the reward value is penalty.

[0109] Rule 2: If the height information is high, the distance information is long, and the path smoothness information is medium, the reward value is penalty.

[0110] Rule 3: If the altitude information is High, the distance information is Long, and the path smoothness information is Smooth, the reward value is Medium.

[0111] Rule 4: If the height information is high, the distance information is medium, and the path smoothness information is winding, the reward value is penalty.

[0112] Rule 5: If the height information is high, the distance information is medium, and the path smoothness information is medium, the reward value is penalty.

[0113] Rule 6: If the height information is high, the distance information is medium, and the path smoothness information is smooth, the reward value is medium.

[0114] Rule 7: If the altitude information is high, the distance information is short, and the path smoothness information is winding, the reward value is medium.

[0115] Rule 8: If the altitude information is high, the distance information is short, and the path smoothness information is medium smooth, the reward value is medium.

[0116] Rule 9: If the height information is High, the distance information is Short, and the path smoothness information is Smooth, then the reward value is Reward.

[0117] Rule 10: If the altitude information is medium altitude, the distance information is long distance, and the path smoothness information is winding, the reward value is penalty.

[0118] Rule 11: If the height information is medium height, the distance information is long distance, and the path smoothness information is medium smooth, the reward value is penalty.

[0119] Rule 12: If the altitude information is medium altitude, the distance information is long distance, and the path smoothness information is smooth, the reward value is reward.

[0120] Rule 13: If the height information is medium height, the distance information is medium distance, and the path smoothness information is winding, the reward value is penalty.

[0121] Rule 14: If the height information is medium height, the distance information is medium distance, and the path smoothness information is medium smooth, then the reward value is medium.

[0122] Rule 15: If the height information is medium height, the distance information is medium distance, and the path smoothness information is smooth, the reward value is reward.

[0123] Rule 16: If the altitude information is medium altitude, the distance information is short distance, and the path smoothness information is winding, the reward value is medium.

[0124] Rule 17: If the altitude information is medium altitude, the distance information is short distance, and the path smoothness information is medium smooth, the reward value is reward.

[0125] Rule 18: If the altitude information is medium altitude, the distance information is short distance, and the path smoothness information is smooth, the reward value is reward.

[0126] Rule 19: If the altitude information is low altitude, the distance information is long distance, and the path smoothness information is winding, the reward value is penalty.

[0127] Rule 20: If the altitude information is low altitude, the distance information is long distance, and the path smoothness information is medium smooth, the reward value is medium.

[0128] Rule 21: If the altitude information is low altitude, the distance information is long distance, and the path smoothness information is smooth, the reward value is reward.

[0129] Rule 22: If the altitude information is low altitude, the distance information is medium distance, and the path smoothness information is winding, the reward value is medium.

[0130] Rule 23: If the height information is low height, the distance information is medium distance, and the path smoothness information is medium smooth, the reward value is reward.

[0131] Rule 24: If the height information is low, the distance information is medium, and the path smoothness information is smooth, the reward value is reward.

[0132] Rule 25: If the altitude information is low altitude, the distance information is short distance, and the path smoothness information is winding, the reward value is medium.

[0133] Rule 26: If the altitude information is low altitude, the distance information is short distance, and the path smoothness information is medium smooth, the reward value is reward.

[0134] Rule 27: If the altitude information is low altitude, the distance information is short distance, and the path smoothness information is smooth, the reward value is reward.

[0135] Table 1 Fuzzy rules (height information = H)

[0136]

[0137] Table 2 Fuzzy rules (height information = M)

[0138]

[0139] Table 3 Fuzzy rules (height information = L)

[0140]

[0141] Furthermore, defuzzifying the fuzzy output to obtain the reward value includes:

[0142] By defuzzifying the formula Defuzzify the fuzzy output to obtain the reward value; where R is the reward value obtained after defuzzification, φ i is the membership degree corresponding to the i-th rule in the fuzzy rule base, O i is the output value corresponding to the i-th rule in the fuzzy rule base, and the values ​​corresponding to penalty, medium, and reward can be set to -0.5, 0, and 0.5 respectively. Through the above calculation process, the reward value can be obtained.

[0143] It should be noted that traditional defuzzification methods (such as the center of gravity method or the maximum membership method) tend to lead to decision rigidity when converting fuzzy reasoning results into precise reward values ​​due to ignoring the rule weights and situational relevance. This method, on the other hand, constructs a dynamic reward generation mechanism through a weighted defuzzification formula based on rule membership and rule output value: each rule in the fuzzy rule base corresponds to expert experience under a specific environmental state (such as "prioritize lifting when the altitude is too low" or "penalize curvature when the path turns sharply"). When multiple rules are activated at the same time due to environmental complexity, the membership quantifies the applicability of each rule in the current state (such as the membership of the obstacle avoidance rule increases sharply when an obstacle suddenly approaches). The defuzzification process integrates the output values ​​of different rules by weight, so that the reward value inherits both high membership and low membership. The dominant decision of the high-degree rules (such as strengthening negative penalties in emergency obstacle avoidance) and the auxiliary correction of low-membership rules (such as maintaining path smoothness rewards during stable flight) are absorbed. This mechanism enables the Q-table update of reinforcement learning to dynamically reflect the multi-dimensional constraints of the environment (safety, efficiency, energy consumption) and their priority changes. For example, when the drone passes through a dense obstacle area, the membership of the obstacle avoidance-related rules is cumulatively improved, driving the reward function to focus on safety; while in open areas, it automatically switches to the dominance of the target approach rule, thereby deducing that the path strategy has context-awareness, which can not only respond to the immediate intervention needs of sudden threats, but also maintain the global optimality of long-term mission goals, and ultimately achieve the unity of multi-objective collaborative optimization and strategy stability in a dynamic uncertain environment.

[0144] 1. 10x10 grid map simulation experiment

[0145] Table 4 Iteration results of five algorithms (10x10). Figure 5 The experimental trajectory diagram for a 10x10 grid map is shown in Table 4. Through comparative experiments, we can see that compared with the traditional Q-Learning algorithm, the proposed IFQ-Learning algorithm has a significant advantage in path length and running speed. The other four algorithms all show varying degrees of improvement in these two data points, indicating that the three improvements proposed in this paper have a positive effect on improving algorithm performance.

[0146] Table 4 Iteration results of five algorithms (10x10)

[0147]

[0148]

[0149] 2. Two-dimensional comparison experiment

[0150] Combine Figure 6 From the algorithm trajectory diagram with an obstacle ratio of 0.1 and the algorithm iteration results with an obstacle ratio of 0.1 in Table 5, it can be seen that when the map becomes larger, the IFQ-Learning algorithm still maintains a good performance, and the gap between the path length and running speed of the IFQ-Learning algorithm and the Q-Learning algorithm gradually widens. Figure 7 By comparing the algorithm trajectory diagram in the environment with an obstacle ratio of 0.2 and the algorithm iteration results in Table 6 with an obstacle ratio of 0.2, it can be concluded that when the number of obstacles increases, the operation effect of the IFQ-Learning algorithm is relatively stable, and the running time of the Q-Learning algorithm increases significantly.

[0151] Table 5 Algorithm iteration results with obstacle ratio 0.1

[0152]

[0153] Table 6 Algorithm iteration results with obstacle ratio 0.2

[0154]

[0155] 3. Three-dimensional comparison experiment

[0156] The effectiveness of the IFQ-Learning algorithm was verified through simulation experiments in various 3D maps. To avoid the curse of dimensionality and enhance the authenticity of the experiment, this section restricts the maximum flight altitude of the drone in the 3D map, effectively reducing the Q-table storage requirements and the computational power consumption during the Q-value update process.

[0157] As shown in the experimental data of the algorithm iteration results in Table 7, it can be seen that in this map, the IFQ-Learning algorithm has certain advantages over the Q-Learning algorithm in terms of comprehensive performance. Figure 8 The iterative results of the Q-Learning algorithm in a three-dimensional environment are shown in the figure and Figure 9 The IFQ-Learning algorithm iteration results in a three-dimensional environment show that the two algorithms use different strategies for altitude information. The Q-Learning algorithm focuses on finding the shortest path and does not make special considerations regarding flight altitude. However, the IFQ-Learning algorithm prefers to find the optimal route at lower altitudes.

[0158] Table 7 Algorithm iteration results

[0159]

[0160] It should be noted that path planning, as a crucial component of mobile UAV navigation systems, essentially involves using an algorithm to find the optimal or near-optimal feasible route from a starting point to a destination, while satisfying specific spatiotemporal constraints, in order to achieve specific mission objectives. The autonomous planning capabilities of path planning algorithms determine whether mobile UAVs can autonomously complete various complex tasks. This plays a crucial role in the design and application of mobile UAVs and represents one of the key technical bottlenecks restricting the development of intelligent UAVs. Traditional path planning algorithms, which rely on established rules, mathematical models, or geometric properties to plan paths, are increasingly unable to meet the requirements of complex tasks. Consequently, reinforcement learning theory has been widely applied in the field of path planning.

[0161] Unlike deep learning, reinforcement learning employs a trial-and-error learning paradigm. It doesn't require extensive, validated empirical data for model training. Through continuous trial-and-error interaction between the drone and its environment, it autonomously optimizes its trajectory with the goal of maximizing the expected cumulative reward, ultimately achieving the optimal route. Compared to traditional path planning algorithms, reinforcement learning-based algorithms offer model-free adaptability and online evolution, eliminating the need for precise environmental modeling and better adapting to complex tasks.

[0162] From the above analysis, it can be seen that although the existing technology has certain advantages over the standard Q-learning algorithm, problems such as the difficulty in finding the initial path and slow convergence caused by defects such as the inability to fully utilize prior knowledge and the sparse reward function still need to be further solved and optimized.

[0163] It should be noted that the embodiment of this specification provides an improved Q-Learning path planning algorithm based on fuzzy logic, see Figure 10The flowchart of the improved Q-Learning path planning algorithm based on fuzzy logic is shown, which includes the following steps:

[0164] S21, establish a grid map, give the starting state and target point position.

[0165] S22, using the improved artificial potential field method to initialize the state value and construct the Q table.

[0166] S23, the maximum-minimum adaptive greedy mechanism selects action a t .

[0167] S24, execute the action and reach the next state s t+1 .

[0168] S25, calculate the reward value r using the improved reward and punishment mechanism t+1 .

[0169] S26, update the Q value.

[0170] S27, judging whether the termination condition is satisfied.

[0171] S28, if yes, obtain the optimal path.

[0172] S29, if not, return to execute the maximum-minimum adaptive greedy mechanism to select action a t .

[0173] In step S21, the environmental image information obtained by the mobile drone is rasterized to create a grid map. If an obstacle is found in the grid, that grid is defined as the obstacle location, and the drone cannot move through that grid. If a target point is found in the grid, that grid is defined as the target point location, which is the drone's desired final location. All other grids are unobstructed feasible areas, and the drone can move freely.

[0174] In step S26, the core goal is to evaluate the current state-action pair through the reward function, iteratively update the value of the state-action pair, and make it close to the optimal action-value function Q*(s,a). The Q value represents the expected long-term cumulative reward that can be obtained by following the current strategy after executing action a in state s. Update the Q value in the Q table. Among them, (s t ,a t ) is the state and action pair at the current moment, Q(s t ,a t ) is the current state (s t ,a t ) corresponds to the Q value, Q(s t+1 ,a t+1 ) is the state at the next moment (s t+1 ,at+1 ) corresponding to the Q value, r t is the reward value obtained for the current state and action, and α and γ are discount factors.

[0175] Figure 11 A schematic diagram of a UAV path planning device provided for one or more embodiments of this specification includes:

[0176] At least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the following steps: S1: rasterizing the environmental image information obtained by the drone, establishing a raster map, and determining the current position, obstacle information, and target point position in the raster map; S2: constructing a Q table based on the current position, the obstacle information, and the target point position, wherein the Q table is used to track and record the data structure of the state, action, and expected reward, and ... Each value corresponds to a Q value of a state and action pair; S3: select a specified action and corresponding state in the Q table; S4: obtain the path smoothness information in the grid map and the altitude information of the drone; S5: perform fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the drone to obtain a reward value; S6: update the Q value in the Q table based on the reward value, the specified action and the corresponding state; S7: iteratively execute steps S3, S4, S5 and S6 based on the updated Q table to obtain an iterative Q table, and determine the optimal path based on the iterative Q table.

[0177] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, which can implement the following steps when executed by a computer: S1: rasterizing the environmental image information obtained by the drone to create a raster map, and determining the current position, obstacle information, and target point position on the raster map; S2: constructing a Q table based on the current position, the obstacle information, and the target point position, wherein the Q table is used to track and record the data structure of the state, action, and expected reward, and each value in the Q table corresponds to a state and an action. S3: select a specified action and a corresponding state in the Q table; S4: obtain the path smoothness information in the grid map and the altitude information of the UAV; S5: perform fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the UAV to obtain a reward value; S6: update the Q value in the Q table based on the reward value, the specified action and the corresponding state; S7: iteratively execute steps S3, S4, S5 and S6 based on the updated Q table to obtain an iterative Q table, and determine the optimal path based on the iterative Q table.

[0178] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0179] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0180] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0183] In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above units may be implemented in the form of hardware or software.

[0184] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

Claims

1. A UAV path planning method, characterized in that: The method comprises: S1: rasterizing the environmental image information obtained by the UAV to create a raster map, and determining the current position, obstacle information and target point position on the raster map; S2: Based on the current position, the obstacle information, and the target point position, a Q table is constructed. The Q table is a data structure used to track and record states, actions, and expected rewards. Each value in the Q table corresponds to a Q value for a state and action pair. S3: Select a specified action and corresponding state in the Q table; S4: Obtaining path smoothness information in the grid map and altitude information of the UAV; S5: performing fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the drone to obtain a reward value; S6: updating the Q value in the Q table based on the reward value, the designated action and the corresponding state; S7: Iteratively execute steps S3, S4, S5, and S6 based on the updated Q-table to obtain an iterative Q-table, and determine the optimal path based on the iterative Q-table.

2. The method according to claim 1, characterized in that The constructing of a Q table based on the current position, the obstacle information and the target point position includes: Determining a first distance between the current position and the target position based on the current position and the target point position; determining a second distance between the current position and the obstacle based on the current position and the obstacle information; determining the effective range of the obstacle based on the obstacle information; A Q table is constructed based on the first distance, the second distance, and the range of the obstacle.

3. The method according to claim 2, characterized in that The range of action based on the first distance, the second distance, and the obstacle includes: Based on the formula with U all =U att +U rep , construct the Q table, where ρ(q,q g ) is the distance between the current position of the UAV and the target point, ρ(q,q o ) is the distance between the current position of the UAV and the obstacle. ξ and η are the attraction factor and repulsion factor respectively, and ρ0 is the range of the obstacle.

4. The method according to claim 1, wherein The selecting of a specified action and a corresponding state in the Q table includes: Based on a pre-set adaptive greedy strategy, a specified action and a corresponding state are selected in the Q table.

5. The method according to claim 4, characterized in that Based on a pre-set adaptive greedy strategy, a specified action and corresponding state are selected in the Q table, including: Based on the adaptive greedy strategy Select the specified action and corresponding state in the Q table, where ε(n+1) is the latest greed factor, ε(n) is the greed factor of the previous iteration, n is the number of iterations, ε max The preset maximum greed value.

6. The method according to claim 1, characterized in that The fuzzy processing is performed based on the current position, the target point position, the path smoothness information, and the altitude information of the UAV to obtain a reward value, including: Performing fuzzy processing based on the current position, the target point position, the path smoothness information, and the altitude information of the UAV, and performing reasoning using a fuzzy rule base to obtain a fuzzy output; The fuzzy output is defuzzified to obtain the reward value.

7. The method according to claim 6, characterized in that Fuzzy processing is performed based on the current position, the target point position, the path smoothness information, and the altitude information of the drone, including: Determine, based on the current position and the target point position, distance information between the current position and the target point position; The distance information is converted into a membership function of the distance information: Among them, μ F (L), μ M (L), μ N (L) are the membership functions of the distance information L, which are preset long distance, preset medium distance, and preset short distance, respectively. d max is the maximum distance between all path points and the target point; The path smoothness information is converted into a membership function of the path smoothness information: Among them, ν T (θ), ν M (θ), ν S (θ) are the membership functions of path smoothness information θ, which are preset tortuous, preset moderately smooth, and preset smooth; The altitude information of the UAV is converted into a membership function of altitude information: Among them, ω L (η), ω M (η), ω H (η) are the height information η is the membership function of the preset low height, preset medium height, and preset high height, h max It is the maximum altitude at which the drone is preset to fly.

8. The method according to claim 7, characterized in that Defuzzifying the fuzzy output to obtain the reward value includes: By defuzzifying the formula Defuzzifying the fuzzy output to obtain the reward value; Among them, R is the reward value obtained after defuzzification, φ i is the membership degree corresponding to the i-th rule in the fuzzy rule base, O i is the output value corresponding to the i-th rule in the fuzzy rule base.

9. A drone path planning device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the drone path planning method according to any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that Computer-executable instructions are stored, and when the computer-executable instructions are executed by a computer, the drone path planning method according to any one of claims 1 to 8 can be implemented.

Images