Unmanned aerial vehicle path planning method and system suitable for three-dimensional terrain

Abstract

The application belongs to the technical field of unmanned aerial vehicle path planning. A kind of unmanned aerial vehicle path planning method and system suitable for three-dimensional terrain are provided, in view of the problems of two-dimensional limitation, low search efficiency and easy to fall into local optimum of traditional ant colony algorithm when carrying out unmanned aerial vehicle path planning, based on DEM modeling, Q-learning is integrated into ant colony algorithm, heuristic function and pheromone strategy are optimized, optimal path is fused and smoothed, the efficiency and safety of path planning under complex terrain are improved, suitable for mountain inspection, agricultural plant protection and other scenes.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) path planning technology, and specifically to a UAV path planning method and system applicable to three-dimensional terrain. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of drone technology, its applications in mountainous areas, agricultural plant protection, and power line inspection are becoming increasingly widespread. In these applications, achieving efficient and safe three-dimensional spatial full-coverage path planning is a key technical challenge. When drones perform coverage tasks in three-dimensional space, they must continuously retrieve accurate location data from the original terrain model. Flying at a constant altitude limits their effectiveness in covering complex terrains (such as mountainous and valley areas) (because safety constraints prevent them from getting closer to the Earth's surface). The drone's altitude is optimized based on its climb capability, sensor capacity, and the terrain features of the area, and its three-dimensional flight path is adjusted accordingly.

[0004] Ant Colony Optimization (ACO) mimics the foraging behavior of ants in nature, seeking optimal paths through a positive feedback mechanism of pheromones. However, it also has some limitations in solving path planning problems. When performing path planning, ACO focuses on selecting each path node based on pheromone concentration. After each generation of ants completes its path search, pheromones are released into all paths traversed, resulting in identical pheromone levels across all paths. Even if only some paths are optimal, this increases path planning time and slows convergence to the optimal path. Furthermore, the feedback effect is not significant with each generation of ants updating their strategies. Additionally, ants may get stuck on obstacle nodes, reducing optimization capability. Moreover, in complex environments, when the globally optimal path appears late, the pheromone concentration on the current optimal path is too high, causing the population to ignore the optimal solution, get trapped in local optima, reduce search efficiency, and converge prematurely. All of these factors make existing ant algorithms unsuitable for the path planning needs of 3D terrain UAVs. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for UAV path planning applicable to three-dimensional terrain. By combining terrain data and modeling the terrain within the planning space based on a digital elevation model, the three-dimensional UAV path planning problem is formulated as a constrained optimization problem with path length and threat as objective functions and collision and turning angle as constraints. By combining reinforcement learning and ant colony algorithm in a three-dimensional environment, higher accuracy UAV path planning is achieved.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for UAV path planning applicable to three-dimensional terrain.

[0008] A UAV path planning method suitable for 3D terrain includes the following process:

[0009] Based on the acquired terrain data, the terrain of the planning space is modeled using a digital elevation model to obtain a planning space model. An objective function with the path length and threat minimization is constructed, and the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor are obtained.

[0010] Based on the Q-Learning algorithm, a heuristic function model is constructed, incorporating the initial Q value, pheromone weight, heuristic information scaling factor, and Q value scaling factor, to obtain an improved state transition formula;

[0011] Based on the objective function, ants whose path length is less than the average path length of the current iteration are identified as elite ants. The pheromone concentration of the path is updated according to the pheromone increment equation. At the same time, the pheromone volatility value is dynamically adjusted according to the adaptive pheromone fluctuation factor equation, taking into account the initial pheromone volatility value and the current iteration number.

[0012] Based on the improved state transition formula and pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative reward and node visit count.

[0013] Based on the cumulative return and the number of node visits, the maximum cumulative return frequency is calculated. Then, the Q-values ​​between nodes are updated according to the Q-value update formula, taking into account the learning rate and discount factor.

[0014] After each generation of ants completes the path planning task, based on the objective function and the updated Q-value between nodes, the three ants with the best combination of path length, threat coefficient and cumulative benefit are selected, and the local paths of the three ants are merged to obtain the fused path.

[0015] A third-order Bézier curve is used to smooth the fusion path, generating the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

[0016] Secondly, the present invention provides a UAV path planning system suitable for three-dimensional terrain.

[0017] A UAV path planning system suitable for three-dimensional terrain includes:

[0018] The modeling initialization unit is configured to: model the terrain of the planning space based on the acquired terrain data and the digital elevation model to obtain the planning space model; construct an objective function with path length and threat minimization; and obtain the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor.

[0019] The state transition formula improvement unit is configured to: construct a heuristic function model based on the Q-Learning algorithm, incorporate the initial Q value, pheromone weight, heuristic information scaling factor and Q value scaling factor, and obtain the improved state transition formula;

[0020] The pheromone update unit is configured to: based on the objective function, identify ants whose path length is less than the average path length of the current iteration as elite ants, update the pheromone concentration of the path according to the pheromone increment equation, and dynamically adjust the pheromone volatility value according to the adaptive pheromone fluctuation factor equation, in combination with the initial pheromone volatility value and the current iteration number.

[0021] The path search recording unit is configured to: based on the improved state transition formula and pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative reward and number of node visits.

[0022] The Q-value update unit is configured to: calculate the maximum cumulative return frequency based on the cumulative return and the number of node visits, and update the Q-values ​​between nodes according to the Q-value update formula by combining the learning rate and the discount factor;

[0023] The path fusion unit is configured to: after each generation of ants completes the path planning task, select the three ants with the best combination of path length, threat coefficient and cumulative benefit based on the objective function and the updated inter-node Q value, and merge the local paths of the three ants to obtain the fused path.

[0024] The path smoothing unit is configured to smooth the fused path using a third-order Bézier curve to generate the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

[0025] Thirdly, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium;

[0026] A processor, adapted to execute computer programs;

[0027] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the UAV path planning method applicable to three-dimensional terrain according to the first aspect of the present invention.

[0028] Fourthly, the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed the UAV path planning method applicable to three-dimensional terrain according to the first aspect of the present invention.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] This invention is based on terrain data and models the terrain within the planning space using a Digital Elevation Model (DEM). It formulates the 3D UAV path planning problem as a constrained optimization problem with path length and threat as objective functions and collision and turning angle as constraints. By combining reinforcement learning and ant colony algorithm in a 3D environment, it achieves higher accuracy UAV path planning.

[0031] This invention incorporates the Q-value from the Q-learning algorithm in reinforcement learning into the ant colony algorithm, improving the heuristic function model in the traditional ant colony algorithm, increasing the accuracy of ant search direction and the global search rate, and introducing an elite ant search strategy and an adaptive pheromone fluctuation factor strategy, thereby increasing the sufficient number of ants during the search process and the flexibility of back-and-forth iteration between algorithms.

[0032] This invention updates the Q-value based on the frequency at which each ant obtains the maximum cumulative benefit after completing the pathfinding task, thereby enhancing the selection of optimal path nodes and improving the convergence speed and exploration ability of ants. The method of integrating the optimal path with the combination of distance and cumulative benefit involves selecting the three ants with the best combination of distance and cumulative benefit after each generation of ants completes the path planning task, and the three ants with the least threat in the objective function. The local paths with the best combination of distance and cumulative benefit are then integrated, and the paths are reorganized into potentially better paths.

[0033] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0034] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0035] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present invention for a UAV path planning method applicable to three-dimensional terrain, where Y represents yes and N represents no;

[0036] Figure 2 A schematic diagram of a digital elevation model provided as an exemplary embodiment of the present invention, wherein, Represents the maximum height. Represents the minimum height. Representing the At height;

[0037] Figure 3 A schematic diagram of a three-dimensional terrain projection provided as an exemplary embodiment of the present invention;

[0038] Figure 4 A schematic diagram of path planning results provided for an exemplary embodiment of the present invention;

[0039] Figure 5 A schematic diagram of an unmanned aerial vehicle (UAV) path planning system suitable for three-dimensional terrain, provided as an exemplary embodiment of the present invention;

[0040] Figure 6 A schematic diagram of a computer device provided for an exemplary embodiment of the present invention. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0043] Traditional path planning algorithms, such as A* and Dijkstra's algorithms, while effective in static and simple settings, often fail to deliver optimal performance in dynamic and complex environments. This is primarily due to their inherent limitations, including inefficiency, slow convergence, and a tendency to get trapped in local optima. To address these issues, this invention formulates the 3D UAV path planning problem as a constrained optimization problem with path length and threat as objective functions and collision and turning angle as constraints. Furthermore, by intelligently configuring ant colony optimization parameters through reinforcement learning, dependence on empirical parameters can be eliminated. Simultaneously, using an improved ACO algorithm, the pheromone matrix obtained in the moving environment is fused with the Q-matrix in the QL algorithm, improving the heuristic learning evaluation model in the QL algorithm. This allows for dynamic adjustment of learning factors, providing guidance for the search path. This invention combines reinforcement learning (RL) with ant colony optimization (ACO) and a digital elevation model (DEM), significantly improving the operational efficiency of UAVs in complex and dynamic 3D environments.

[0044] This implementation provides a UAV path planning method suitable for 3D terrain, such as... Figure 1As shown, the process includes the following steps: The terrain within the planning space is modeled based on a Digital Elevation Model (DEM) and transformed into a 3D path planning problem. Variables and parameters are then initialized, the Q-value from the Q-learning algorithm in reinforcement learning is added, and the heuristic function model in the traditional ACO algorithm is improved. Next, an elite ant search strategy and an adaptive pheromone fluctuation factor strategy are introduced. Then, the ants select the next node based on the pheromone concentration, heuristic information, and Q-value on the path, and save the corresponding path information and reward / penalty information. It is then determined whether the ant has reached the target node. If not, the node selection process continues. If it has reached the target node and the step size is less than the maximum step size, the maximum cumulative reward obtained by each node in the ant path, the frequency of obtaining the maximum cumulative reward of each adjacent node, and the Q-value are updated. After the update, it is determined whether all ants have completed the path search. If not, the process returns to the ant node selection step. If all ants have completed the path search, three globally advantageous paths are selected to achieve optimal local path fusion, and the pheromone on the fused path is enhanced.

[0045] More specifically, the present invention uses a step size of A regular grid of latitude and longitude is interpolated across all terrain nodes to generate a three-dimensional grid of latitude, longitude, and elevation data. This indicates that its corresponding point set in the relative coordinate system is Using a bilinear interpolation model can effectively guarantee the interpolation accuracy of DEM discrete grid data. However, directly in Interpolation may introduce significant errors. Therefore, Relative coordinate system The z-coordinate of the geographical location in the image is based on the inclusion of The coordinates of the four vertices of the grid cell are calculated, specifically including:

[0046] (1);

[0047] in, For the UAV in the relative coordinate system coordinate; The x-coordinates of two adjacent points; The ordinates of two adjacent points; The coordinates of the target interpolation point for which the calculation results are needed. These are the original data values ​​corresponding to four adjacent sampling points. For target point The resulting data value is obtained through bilinear interpolation. These represent the horizontal axis, vertical axis, and vertical shaft, respectively.

[0048] Using B-spline curves to represent the generated path allows for more flexible path depiction with fewer parameters, thus saving computation time. Assume the number of control points is... The drone's flight path This can be represented as a discrete coordinate sequence, as shown below:

[0049] (2);

[0050] in, and These represent the pre-arranged start and end points, respectively. This represents an intermediate control point; therefore, the algorithm only needs to optimize the remaining points. The coordinates of the three-dimensional control points are sufficient; the control points can be represented as follows:

[0051] (3);

[0052] in, These represent coordinates in three-dimensional space.

[0053] The 3D UAV path planning problem is formulated as a constrained optimization problem. First, the path length and threats of the UAV flight path are analyzed in detail and then used as indicators of the objective function. Second, constraints are formulated based on these rules: the UAV must not collide with obstacles or violate the maximum turning angle during flight.

[0054] The objective function of the 3D UAV path planning problem is formulated as a single objective function considering path length (distance) and threat. The UAV flight path is a discrete coordinate sequence, which is determined by introducing weighting coefficients. (Used to balance the importance of distance and threat). Thus, the path length can be calculated as:

[0055] (4);

[0056] in, Fitness value representing flight distance and Indicates the first position in the sequence and the Each element (usually a vector or a point, such as adjacent coordinates in a path). express point to The magnitude of the vector, This represents a comprehensive consideration of all threats along the path (which can be obtained by integrating or summing the threat values ​​at each point along the path). Weights representing path length Indicates the weight of the threat.

[0057] 3D terrain near path nodes The projection is a two-dimensional grid terrain, and the corresponding projection nodes on the horizontal plane are named... ; to node The distance is greater than the minimum safe distance The grid points are non-threatening. Therefore, only points on the safety ring should be considered threat points and must be included when calculating terrain threats, such as... Figure 4 As shown.

[0058] Terrain affects the flight path of drones The threat can be calculated as follows:

[0059] (5);

[0060] This represents the terrain threat assessment function, with the following input: Typically, it displays the system status (path parameters, target location), and the output is a threat level value. This represents the total number of objects to be evaluated (number of path segments, number of individual targets). The total number of threat sources (obstacles, number of threat areas); The index of the first summation corresponds to the ordinal number of the object to be evaluated (values ​​range from 1 to N). The index of the second summation corresponds to the sequence number of the threat source, with values ​​ranging from 1 to... ; This is the safe distance threshold that drones should adhere to (a threat is considered to exist when the distance between the object being evaluated and the threat source is less than this value). The object to be evaluated and the source of the threat The actual distance between them.

[0061] when hour, This indicates that the threat is within a safe distance and contributes to the cost of the threat.

[0062] when When the distance exceeds the safe distance, this item is calculated by squared the proportion of the distance exceeding the safe distance, reflecting the logic that "the greater the distance exceeds the safe distance, the smaller (or no) the cost of the threat", which is more in line with the actual perception of threats (a threat only arises when the drone enters the threat area).

[0063] Given a drone's flight path, it should be subject to two constraints:

[0064] Constraint 1: No collision;

[0065] Constraint 2: The turning angle is less than or equal to the maximum turning angle.

[0066] The above constraints require that the number of path nodes generated below the terrain boundary be equal to zero, and require that all The turning angle is less than or equal to the predefined maximum turning angle The turning angle can be calculated as follows:

[0067] (6);

[0068] in, For the first The turning angle at each bend. The maximum permissible turning angle is predefined; , and These are three consecutive nodes in the path. From point to The magnitude of the vector, From point to The magnitude of the vector.

[0069] To incorporate it into the optimization objective (e.g., as a penalty term), we can further construct the following objective function term (used to minimize the portion exceeding the maximum turning angle):

[0070] (7);

[0071] Where N is the path node coefficient. This is the maximum allowable included angle threshold. When... This item is 0; when When this term is used, it represents the sum of the excess portion and is used to penalize violations of the maximum turning angle constraint during optimization.

[0072] After formulating the 3D path planning problem, it is necessary to initialize the variables and parameters, such as... Value, frequency of obtaining maximum benefit ant count Number of iterations pheromone scaling factor Heuristic information scaling factor Sum scaling factor We will then proceed with the optimization plan.

[0073] When the distance between two adjacent nodes in a motion environment is short, local optima are easily generated. Therefore, this paper designs a new heuristic function model. The new heuristic function model fully considers the positional relationship between the starting point and the ending point:

[0074] (8);

[0075] in, The current node To the next node Euclidean distance, Endpoint To the next node The Euclidean distance. When the global Euclidean distance decreases during the search process, the directionality of the path search increases, thus increasing the probability of selecting a node and improving the accuracy of the ant search.

[0076] After completing the path search, the ants leave pheromones along the path, which in turn creates a positive feedback loop that strengthens the search. The average path length in each iteration is used as the basis for this process. Using a threshold, ants traversing paths shorter than the average path length are considered elite ants, while those traversing paths longer than the average path length are considered inferior ants. Inferior ants are not updated with pheromones; instead, adaptive weights are used to update the pheromone concentration on the elite ants' search paths. The pheromone increment equation is as follows:

[0077] (9);

[0078] (10);

[0079] In the formula, It is the first The pheromone increment left by the ant in the current iteration. It refers to the number of ants. It is the pheromone constant. It is the first The path length searched by an ant. It is the first Average path length at iteration 1 It is the shortest path length at the t-th iteration. It is an adaptive weight value that follows the trend of the rising half-ridge curve, which makes the mid-term path change trend of elite ants update pheromones more quickly, thereby improving the search efficiency of the algorithm.

[0080] In ant colony optimization, pheromone fluctuation factor The value of this factor has a significant impact on the global searchability and convergence of the algorithm. If the value is too large, the pheromone evaporates too quickly, meaning the residual pheromone concentration on the path is insufficient to attract ants to search, thus slowing down the convergence speed. If the value is too small, when ants find a better path, they may become confused due to the residual pheromone concentration on that path, potentially searching for a local optimum and then ending the algorithm's search, leading to a decrease in global search efficiency. Therefore, an adaptive pheromone fluctuation factor is introduced. To improve global search efficiency, the initial value is made large, and the later value... To improve convergence efficiency, as shown in the equation:

[0081] (11);

[0082] in, It is the introduced normal distribution factor. It is the scale parameter of the normal distribution. It is a location parameter of a normal distribution, reaching its peak. The place, This is the initial pheromone evaporation value. It is the current iteration number. This is the lower bound for the number of iterations. This refers to the setting of the number of iterations. Initially, the fluctuation coefficient is relatively large to improve search efficiency, and when the number of iterations is reached... When performing multiple iterations, reduce the fluctuation coefficient to improve convergence efficiency.

[0083] Incorporating a Q-value into the state transition rule, updating the Q-value based on the frequency with which each ant obtains the maximum cumulative reward after completing the pathfinding task, enhances the selection of optimal path nodes and improves the ants' convergence speed and exploration ability. The Q-value is updated based on the frequency of obtaining the maximum cumulative reward. For critical nodes in the graph, i.e., nodes where two paths overlap, the Q-value for different actions is updated to varying degrees based on the maximum cumulative reward, thus strengthening the Q-value for actions with shorter paths to the target node. In some critical nodes, the Q-value can be used to make better decisions. The state transition formula for the reinforcement learning-ant colony algorithm is shown below:

[0084] (12);

[0085] In the formula, It is the first Only one ant starts from node t at time t. The probability transition equation to the next node j, It is the pheromone weight, representing the expected pheromone concentration during the search path. These are heuristic weights, representing the expected path distance when searching for a path. Ants Can be found in the current search node Reach the next set of nodes Place, From node To the next node pheromone concentration value, From node To the next node Heuristic information. The value is updated based on the frequency at which the maximum cumulative return is achieved. The current node To the next node cumulative value, yes Value scaling factor. When each ant reaches the target position in fewer steps than the maximum step size, the value is updated based on the frequency of maximizing the reward. value, The value update formula is as follows:

[0086] (13);

[0087] In the formula, It is the learning rate; Representing ants from nodes Move to node Afterwards, Instant rewards earned at any time; Discount factor Selecting a node nodes The frequency of obtaining the maximum cumulative reward, which represents the frequency at which the ant finds a partially optimal path, is calculated using the following formula:

[0088] (14);

[0089] In the formula, It is at the node Select node Number of times, It is at the node Select node The number of times to obtain the maximum cumulative profit is calculated as follows:

[0090] (15);

[0091] In the formula, This indicates that the ant is at the node. The historical maximum cumulative return obtained upon reaching the target point. This indicates that the ant starts from the node. Select node The cumulative reward gained upon reaching the target point. The ant's cumulative reward is as follows:

[0092] (16);

[0093] in, express The original sequence value at time (position), This is a weight decay factor used to weight the "sequence items at future time steps". represent Weighted cumulative results at any given time (cumulative discount reward). It is the total stride length of the ant. express The original sequence value at time (position), express The original sequence value at time (position), express The original sequence value at time (position).

[0094] In conventional ant colony optimization algorithms, after each generation of ants completes its pathfinding task, the pheromones of all ant paths need to be updated, resulting in the pheromones of better paths being less prominent. To explore optimal paths that combine distance, threat coefficient, and cumulative reward, and to provide a good search foundation for the next generation of ants, an optimization method integrating distance, threat coefficient, and cumulative reward is proposed. After each generation of ants completes its path planning task, the three ants with the best combination of distance, threat coefficient, and cumulative reward are selected. Their local paths with the best combination of distance, threat coefficient, and cumulative reward are integrated, and the paths are reorganized into potentially better paths. The formula for the trade-off between path distance, threat coefficient, and reward is as follows:

[0095] (17);

[0096] in, and These are weighting parameters used to balance the contribution ratios of the "distance term, cumulative benefit term, and threat term." It is the cumulative reward that the ant gains as it travels along the path. It is a threat item; the greater the threat, the smaller this item. This is a distance term; the smaller the distance, the larger this term becomes. Represents the path.

[0097] After the optimal path is planned, the planned curve needs to be optimized. A Bézier curve is defined by two endpoints and a set of intermediate control points, used to fit the curve. A second-order Bézier curve is a linear combination of two first-order Bézier curves. A third-order Bézier curve is obtained by a linear combination of two second-order Bézier curves. To ensure that the optimized path covers both the starting and ending points and satisfies the smoothness, continuity, and optimality of the path under the maximum curvature constraint, a third-order Bézier curve is used for smoothing.

[0098] (18);

[0099] (19);

[0100] in, Let be the order of the Bézier curve. It is the first One path control point, It is a parameter. It is the expression for a Bézier curve. It is a third-order Bézier curve. It is the starting point of the path. It is the end point of the path. These are intermediate control points (used to adjust the curve shape, ensuring path smoothness and curvature constraints), as... It can continuously change from 0 to 1 to generate connection starting points. and the end point And a path that satisfies smoothness.

[0101] Figure 5 A UAV path planning system suitable for three-dimensional terrain is shown, comprising:

[0102] The modeling initialization unit 501 is configured to: model the terrain of the planning space based on the acquired terrain data and the digital elevation model to obtain the planning space model; construct an objective function with path length and threat minimization; and obtain the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor.

[0103] The state transition formula improvement unit 502 is configured to: construct a heuristic function model based on the Q-Learning algorithm, incorporate the initial Q value, pheromone weight, heuristic information scaling factor and Q value scaling factor, and obtain the improved state transition formula;

[0104] The pheromone update unit 503 is configured to: based on the objective function, identify ants whose path length is less than the average path length of the current iteration as elite ants, update the pheromone concentration of the path according to the pheromone increment equation, and dynamically adjust the pheromone volatility value according to the adaptive pheromone fluctuation factor equation, in combination with the initial pheromone volatility value and the current iteration number.

[0105] The path search recording unit 504 is configured to: based on the improved state transition formula and pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative benefit and number of node visits.

[0106] Q-value update unit 505 is configured to: calculate the maximum cumulative return frequency based on cumulative return and node access frequency, and update the Q-value between nodes according to the Q-value update formula by combining the learning rate and discount factor.

[0107] The path fusion unit 506 is configured to: after each generation of ants completes the path planning task, select the three ants with the best combination of path length, threat coefficient and cumulative benefit based on the objective function and the updated inter-node Q value, and merge the local paths of the three ants to obtain the fused path.

[0108] The path smoothing unit 507 is configured to: smooth the fused path using a third-order Bézier curve to generate the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

[0109] It is understood that the aforementioned units can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of the present invention. The aforementioned units are based on logical functional division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of the present invention, the system may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0110] According to another embodiment of the present invention, the system of this embodiment can be constructed by running a computer program (including program code) capable of performing the steps involved in the corresponding method of the present invention on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the aforementioned computing device through the computer-readable recording medium, and run therein.

[0111] Figure 6 A computer device is shown, which includes a processor 601, a communication interface 602, and a computer-readable storage medium 603. The processor 601, communication interface 602, and computer-readable storage medium 603 can be connected via a bus or other means.

[0112] The communication interface 602 is used to receive and send data. The computer-readable storage medium 603 can be stored in the memory of the electronic device. The computer-readable storage medium 603 is used to store computer programs, which include program instructions. The processor 601 is used to execute the program instructions stored in the computer-readable storage medium 603.

[0113] The processor 601 is the computing and control core of an electronic device. It is suitable for implementing one or more instructions, specifically for loading and executing one or more instructions to achieve the corresponding method flow or corresponding function.

[0114] Processor 601 is configured to perform the following procedure:

[0115] Based on the acquired terrain data, the terrain of the planning space is modeled using a digital elevation model to obtain a planning space model. An objective function with the path length and threat minimization is constructed, and the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor are obtained.

[0116] Based on the Q-Learning algorithm, a heuristic function model is constructed, incorporating the initial Q value, pheromone weight, heuristic information scaling factor, and Q value scaling factor, to obtain an improved state transition formula;

[0117] Based on the objective function, ants whose path length is less than the average path length of the current iteration are identified as elite ants. The pheromone concentration of the path is updated according to the pheromone increment equation. At the same time, the pheromone volatility value is dynamically adjusted according to the adaptive pheromone fluctuation factor equation, taking into account the initial pheromone volatility value and the current iteration number.

[0118] Based on the improved state transition formula and pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative reward and node visit count.

[0119] Based on the cumulative return and the number of node visits, the maximum cumulative return frequency is calculated. Then, the Q-values ​​between nodes are updated according to the Q-value update formula, taking into account the learning rate and discount factor.

[0120] After each generation of ants completes the path planning task, based on the objective function and the updated Q-value between nodes, the three ants with the best combination of path length, threat coefficient and cumulative benefit are selected, and the local paths of the three ants are merged to obtain the fused path.

[0121] A third-order Bézier curve is used to smooth the fusion path, generating the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

[0122] This invention also provides a computer-readable storage medium, which is a memory device in an electronic device for storing programs and data. It is understood that the computer-readable storage medium here may include both built-in storage media in the electronic device and extended storage media supported by the electronic device. The computer-readable storage medium provides storage space for storing the processing system of the electronic device.

[0123] Furthermore, this storage space also contains one or more instructions suitable for loading and execution by the processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory; alternatively, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0124] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to perform the following process:

[0125] Based on the acquired terrain data, the terrain of the planning space is modeled using a digital elevation model to obtain a planning space model. An objective function with the path length and threat minimization is constructed, and the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor are obtained.

[0126] Based on the Q-Learning algorithm, a heuristic function model is constructed, incorporating the initial Q value, pheromone weight, heuristic information scaling factor, and Q value scaling factor, to obtain an improved state transition formula;

[0127] Based on the objective function, ants whose path length is less than the average path length of the current iteration are identified as elite ants. The pheromone concentration of the path is updated according to the pheromone increment equation. At the same time, the pheromone volatility value is dynamically adjusted according to the adaptive pheromone fluctuation factor equation, taking into account the initial pheromone volatility value and the current iteration number.

[0128] Based on the improved state transition formula and pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative reward and node visit count.

[0129] Based on the cumulative return and the number of node visits, the maximum cumulative return frequency is calculated. Then, the Q-values ​​between nodes are updated according to the Q-value update formula, taking into account the learning rate and discount factor.

[0130] After each generation of ants completes the path planning task, based on the objective function and the updated Q-value between nodes, the three ants with the best combination of path length, threat coefficient and cumulative benefit are selected, and the local paths of the three ants are merged to obtain the fused path.

[0131] A third-order Bézier curve is used to smooth the fusion path, generating the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

[0132] The present invention also provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the following process:

[0133] Based on the acquired terrain data, the terrain of the planning space is modeled using a digital elevation model to obtain a planning space model. An objective function with the path length and threat minimization is constructed, and the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor are obtained.

[0134] Based on the Q-Learning algorithm, a heuristic function model is constructed, incorporating the initial Q value, pheromone weight, heuristic information scaling factor, and Q value scaling factor, to obtain an improved state transition formula;

[0135] Based on the objective function, ants whose path length is less than the average path length of the current iteration are identified as elite ants. The pheromone concentration of the path is updated according to the pheromone increment equation. At the same time, the pheromone volatility value is dynamically adjusted according to the adaptive pheromone fluctuation factor equation, taking into account the initial pheromone volatility value and the current iteration number.

[0136] Based on the improved state transition formula and pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative reward and node visit count.

[0137] Based on the cumulative return and the number of node visits, the maximum cumulative return frequency is calculated. Then, the Q-values ​​between nodes are updated according to the Q-value update formula, taking into account the learning rate and discount factor.

[0138] After each generation of ants completes the path planning task, based on the objective function and the updated Q-value between nodes, the three ants with the best combination of path length, threat coefficient and cumulative benefit are selected, and the local paths of the three ants are merged to obtain the fused path.

[0139] A third-order Bézier curve is used to smooth the fusion path, generating the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

[0140] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0141] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, digital cable) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0142] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A UAV path planning method suitable for three-dimensional terrain, characterized in that, The process includes the following: Based on the acquired terrain data, the terrain of the planning space is modeled using a digital elevation model to obtain a planning space model. An objective function with the path length and threat minimization is constructed, and the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor are obtained. Based on the Q-Learning algorithm, a heuristic function model is constructed, incorporating the initial Q value, pheromone weight, heuristic information scaling factor, and Q value scaling factor, to obtain an improved state transition formula; Based on the objective function, ants whose path length is less than the average path length of the current iteration are identified as elite ants. The pheromone concentration of the path is updated according to the pheromone increment equation. At the same time, the pheromone volatility value is dynamically adjusted according to the adaptive pheromone fluctuation factor equation, taking into account the initial pheromone volatility value and the current iteration number. Based on the improved state transition formula and the pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative benefit and node visit count. Based on the cumulative return and the number of node visits, the maximum cumulative return frequency is calculated. Then, the Q-values ​​between nodes are updated according to the Q-value update formula, taking into account the learning rate and discount factor. After each generation of ants completes the path planning task, based on the objective function and the updated inter-node Q value, the three ants with the best combination of path length, threat coefficient and cumulative benefit are selected, and the local paths of the three ants are merged to obtain the fused path. A third-order Bézier curve is used to smooth the fusion path, generating the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

2. The UAV path planning method applicable to three-dimensional terrain as described in claim 1, characterized in that, The heuristic function model is as follows: ； in, For the current node To the next node Euclidean distance, As endpoints To the next node Euclidean distance, The degree of expectation at the search path nodes.

3. The UAV path planning method applicable to three-dimensional terrain as described in claim 2, characterized in that, The improved state transition formula is: ； in, It's an ant. From the current node To the next node The probability transition equation, It is the pheromone weight. It is a heuristic weight. Ants Able to search the current node Reach the next set of nodes Place, It is the pheromone concentration value at the search path node. It is the degree of expectation at the nodes of the search path. It is a value scaling factor; The current node To the next node cumulative value; Represents a moment.

4. The UAV path planning method applicable to three-dimensional terrain as described in claim 1, characterized in that, The pheromone increment equation is: ； in, It's an ant. In the The pheromones left behind by the iteration It is the pheromone constant. It's an ant. Length of the search path, This is the average path length during this iteration. It is the shortest path length in this iteration. It is an adaptive weight value. This represents the total number of ants.

5. The UAV path planning method applicable to three-dimensional terrain as described in claim 4, characterized in that, Adaptive weight values ,include: 。 6. The UAV path planning method applicable to three-dimensional terrain as described in claim 1, characterized in that, The adaptive pheromone fluctuation factor equation is: ； in, This represents the introduced normal distribution factor. Represents the initial pheromone evaporation value. Represents the current iteration number. It is the threshold for the number of iterations. This represents the scaling parameter of the normal distribution. This represents the location parameter of a normal distribution.

7. The UAV path planning method applicable to three-dimensional terrain as described in claim 1, characterized in that, The three ants with the best combination of path length, threat level, and cumulative reward are selected using a trade-off factor: ； in, and These are weight parameters. It is the cumulative reward that the ant gains as it travels along the path. It is a threat. It is a distance term. This represents the terrain threat assessment function. Fitness value indicating flight distance.

8. A UAV path planning system suitable for three-dimensional terrain, characterized in that, include: The modeling initialization unit is configured to: model the terrain of the planning space based on the acquired terrain data and the digital elevation model to obtain the planning space model; construct an objective function with path length and threat minimization; and obtain the initial Q value, pheromone weight, heuristic information scaling factor, Q value scaling factor, initial pheromone evaporation value, learning rate, and discount factor. The state transition formula improvement unit is configured to: construct a heuristic function model based on the Q-Learning algorithm, incorporate the initial Q value, pheromone weight, heuristic information scaling factor and Q value scaling factor, and obtain the improved state transition formula; The pheromone update unit is configured to: based on the objective function, identify ants whose path length is less than the average path length of the current iteration as elite ants, update the pheromone concentration of the path according to the pheromone increment equation, and dynamically adjust the pheromone volatility value according to the adaptive pheromone fluctuation factor equation, in combination with the initial pheromone volatility value and the current iteration number. The path search recording unit is configured to: based on the improved state transition formula and the pheromone concentration, the ant selects the next node from the set of allowed nodes that satisfy the collision and turning angle constraints, and records the path length, threat coefficient, cumulative benefit and number of node visits; The Q-value update unit is configured to: calculate the maximum cumulative return frequency based on the cumulative return and the number of node visits, and update the Q-values ​​between nodes according to the Q-value update formula by combining the learning rate and the discount factor; The path fusion unit is configured to: based on the objective function and the updated inter-node Q-value, select three ants with the shortest path length, the lowest threat coefficient, and the best combination of cumulative benefits, extract local paths, and use trade-off factors to integrate them through the path distance and benefit trade-off formula to obtain the fused path. The path smoothing unit is configured to smooth the fused path using a third-order Bézier curve to generate the optimal three-dimensional path within the planning space model that satisfies collision and turning angle constraints.

9. A computer device, characterized in that, include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the UAV path planning method for three-dimensional terrain as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1 to 7 for UAV path planning in three-dimensional terrain.

Images