Reinforced learning enhanced continuous domain ant colony optimization method for three-dimensional path planning of unmanned aerial vehicle
By enhancing the continuous domain ant colony optimization method through reinforcement learning, and combining various cooperative operations and adaptive strategies, the problems of adaptive decision-making and multi-constraint handling in UAV 3D path planning are solved, generating efficient and safe flight paths, and achieving rapid convergence and improved stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2026-02-06
- Publication Date
- 2026-05-12
AI Technical Summary
Existing UAV 3D path planning algorithms lack adaptive decision-making capabilities in complex environments and cannot effectively handle various practical constraints. Furthermore, existing reinforcement learning ant colony optimization algorithms lack multi-strategy collaborative mechanisms and cannot generate efficient and reliable path plans.
We employ reinforcement learning to enhance continuous domain ant colony optimization. By designing four collaborative operations—quantum behavior exploration, multi-elite guided development, Bézier curve smoothing, and information entropy minimization adjustment—and combining them with a Q-learning adaptive strategy, we optimize the path planning model and construct a comprehensive cost function that considers path length, collision avoidance, height constraints, and smoothness.
It generates high-quality, smooth, and safe flight paths, and converges quickly to the optimal solution, achieving an 8.3% improvement over other algorithms. It also demonstrates good stability and fast convergence speed in complex environments.
Smart Images
Figure CN122018525A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of UAV path planning technology, specifically relating to a reinforcement learning-enhanced continuous domain ant colony optimization method for UAV three-dimensional path planning. Background Technology
[0002] Unmanned aerial vehicles (UAVs) are widely used in precision agriculture, search and rescue, infrastructure inspection, and cargo transportation as important tools for exploring complex environments. In these applications, autonomous navigation is one of the core requirements of UAV systems. Path planning, a key technology for autonomous navigation, aims to calculate the optimal path from start to finish for the UAV in complex environments. Especially in complex terrain environments with dense obstacles, efficient and reliable path planning algorithms are needed to achieve autonomous flight of UAVs.
[0003] Based on existing research findings, the relevant technical solutions mainly focus on graph search algorithms. [1] Metaheuristic Algorithms [2] In the same direction. Ant colony optimization algorithm, as an important metaheuristic algorithm, can handle continuous coordinate optimization problems in path planning through its continuous domain extension form.
[0004] In recent years, researchers have attempted to combine reinforcement learning with ant colony optimization algorithms to improve performance, such as the reinforcement learning ant colony optimization algorithm proposed by Zhang et al. [3] However, it adopts a fixed search strategy, only performs adaptive optimization for a single parameter, lacks a multi-strategy collaborative mechanism, and lacks comprehensive consideration of various practical constraints in the path planning scenario, so it cannot be effectively applied to UAV 3D path planning.
[0005] Therefore, it is of great significance to study UAV path planning algorithms with adaptive decision-making capabilities in complex three-dimensional environments.
[0006] [1] Ding Ningning; Chen Lei; Niu Xiaodong; Chen Haotian; Zhang Yuxue; Liu Guodong. A path planning method for unmanned aerial vehicles based on fusion A-star potential field algorithm [P]. Zhejiang Province: CN120029308A, 2025-05-23.
[0007] [2] Peng Shuyan. A method for UAV 3D path planning based on neuronal chaotic wolf pack algorithm [P]. Jiangsu Province: CN120315451A, 2025-07-15.
[0008] [3]Zhang W, Wang C, Lin W, et al. Continuous-domain ant colony optimization algorithm based on reinforcement learning[J]. InternationalJournal of Wavelets, Multiresolution and Information Processing, 2021, 19(03): 2050084. Summary of the Invention
[0009] To address the problems existing in the background technology, the purpose of this invention is to provide a reinforcement learning-enhanced continuous domain ant colony optimization method for 3D path planning of unmanned aerial vehicles (UAVs). This method integrates reinforcement learning mechanisms into the continuous domain ant colony optimization algorithm framework, enabling each search agent to adaptively select optimization strategies as an independent intelligent agent. By designing four collaborative operations—quantum behavior exploration, multi-elite guided development, Bézier curve smoothing, and information entropy minimization adjustment—it comprehensively solves the problems of exploration-development balance, path feasibility, and trajectory regularity, overcoming the limitations of existing methods with their single strategies.
[0010] To achieve the above objectives, the technical solution of the present invention is as follows:
[0011] A reinforcement learning-based continuous domain ant colony optimization method for UAV 3D path planning includes the following steps:
[0012] Step 1: Establish a mathematical model for UAV path planning and construct a comprehensive cost function that includes path length, collision avoidance, altitude constraints, and smoothness.
[0013] Step 2: Initialize the reinforcement learning-enhanced continuous domain ant colony optimization framework, configure an independent Q-table for each individual, and define the state space and action space. The specific process is as follows:
[0014] Set the population size to Random initialization If there are 10 individuals, then the i-th individual is... , ,individual The path scheme represented is denoted as The coordinates of its n waypoints are represented as follows: ,in Provide the three-dimensional coordinates of the j-th waypoint; configure an independent Q-table for each individual. All Q-tables are initialized to zero matrices; the state space is defined. These correspond to the four collaborative operations designed later; define the action space. The actions This indicates that the m-th cooperative operation will be selected. ;
[0015] Calculate the cost for each individual using the cost function defined in step 1. Find the global optimal solution Set the maximum number of iterations. Initialize the current iteration count ;
[0016] Step 3: Design four cooperative operations, namely quantum behavior exploration operation, multi-elite guided development operation, Bézier curve smoothing operation, and information entropy minimization adjustment operation;
[0017] Specifically, the quantum behavior exploration operation is as follows:
[0018] The step size is generated using the Lévy flight mechanism, representing the breadth of the exploration range:
[0019] ,
[0020] in, All of them are random variables that follow a corresponding normal distribution. Lévy index;
[0021] The standard deviation is expressed by the following formula:
[0022] ,
[0023] It is a gamma function;
[0024] Generate the d-th dimension component of the new solution.
[0025] ,
[0026] Where L is the feature length parameter. The Lévy index, ,for Uniform random numbers over an interval It is a symbolic function; To select the guiding solution in the first The value of the dimension; For the first The standard deviation of dimension d, where d represents a coordinate component of a waypoint in the path; logarithmic factor Simulate quantum tunneling effect;
[0027] The multi-elite guided development operation is specifically as follows:
[0028] Define elite set ,in For the number of elite solutions, assign a ranking-based weight to each elite solution: , ,
[0029] Constructing a new solution using the Gaussian kernel function:
[0030] ,
[0031] The new solution represents the first... Dimensional components from mean to Standard deviation is Sampling from a normal distribution,
[0032] Introducing the optimal solution bias:
[0033]
[0034] in, The bias strength coefficient, For the current optimal individual in the th... The value of the dimension;
[0035] The Bézier curve smoothing operation is as follows:
[0036] For three consecutive waypoints Constructing a quadratic Bézier curve:
[0037] ,
[0038] in, For curve parameters,
[0039] A weighted mixing strategy is used to process the quadratic Bézier curve to avoid over-smoothing.
[0040] ,
[0041] in, For smoothing weighting coefficients;
[0042] The specific operation of minimizing information entropy is as follows:
[0043] Calculate the entropy of the direction change of the path:
[0044] ,
[0045] To normalize to a probability distribution, ,
[0046] in, To take into account changes from all angles, To sum dummy variables , Let the turning angle be at the j-th waypoint. Let the angle of ascent be the angle of ascent for the j-th waypoint;
[0047] The ideal position of each waypoint is obtained by minimizing the entropy of directional changes.
[0048]
[0049] Waypoint updates employ an adaptive step size strategy.
[0050]
[0051] Among them, the adjustment coefficient , This represents the current iteration number. This represents the maximum number of iterations.
[0052] Step 4: Perform Q-learning adaptive strategy selection, through... - A greedy strategy selects cooperative actions for each individual;
[0053] Step 5: Evaluate the new solution and update the Q-table, define an immediate reward based on the cost improvement, and update the Q-value using an adaptive learning rate;
[0054] Step 6: Update the population and the global optimal solution, and determine whether the maximum number of iterations has been reached. If not, return to step 4 to continue iterating; otherwise, proceed to step 7.
[0055] Step 7: Using the globally optimal solution As the final UAV 3D path planning solution, the optimal path includes each waypoint , .
[0056] Furthermore, the specific process of step 1 is as follows:
[0057] For any path P i From the starting point S to the ending point T, there are n waypoints, where i is the path number. Then path P... i The spatial coordinates of the j-th waypoint on the [plane name] are P. i,j , , Let J be the horizontal coordinate of the j-th waypoint on the i-th path. Let J be the vertical coordinate of the j-th waypoint on the i-th path. Let J be the altitude coordinates of the j-th waypoint on the i-th path;
[0058] Then path length cost for,
[0059] ,
[0060] in, Indicates adjacent waypoints and The Euclidean distance between them;
[0061] Collision Costs for,
[0062] ,
[0063] in, Let K be the number of obstacles. For collision evaluation parameters;
[0064] High cost for,
[0065] ,
[0066] Let j be the altitude of the j-th waypoint. The ideal flight altitude is set according to mission requirements. This is a penalty factor for low-altitude flight, used to avoid flying too low; This is a penalty coefficient for high-altitude flight, used to limit the increase in energy consumption caused by excessively high-altitude flight; This is the minimum safe altitude at the j-th waypoint, which is usually the terrain altitude plus a safety margin; This represents the maximum permissible altitude at the j-th waypoint, subject to airspace restrictions.
[0067] Smoothing cost for,
[0068] ,
[0069] and The penalty coefficient for smoothness cost. Let the turning angle be at the j-th waypoint. Let the climb angle be the (j+1)th waypoint. Let the angle of ascent be the angle of ascent for the j-th waypoint;
[0070] Therefore, the total cost function for, ; These are the weighting coefficients for different costs.
[0071] Furthermore, collision assessment parameters ,
[0072] and These are the altitude coordinates of two adjacent waypoints. This is the horizontal distance of the path segment obtained by connecting the center of the cylinder to two adjacent waypoints; As a safety threshold, D is the radius of the drone, and L is the danger distance. In order to wrap the first The minimum radius of the cylinder containing the obstacle. In order to wrap the first The minimum cylinder height for an obstacle.
[0073] Furthermore, the climb angle at the j-th waypoint The calculation formula is as follows:
[0074] ,
[0075] Among them, molecules The denominator is the altitude difference between two adjacent waypoints, and the horizontal distance of the path segment is the denominator.
[0076] Turning angle between adjacent path segments:
[0077] ,
[0078] in, Represents the vector dot product. Represents the magnitude of a vector. For path segment Horizontal projection,
[0079]
[0080] for Unit vector in the positive direction of the axis. This represents the cross product of vectors.
[0081] Furthermore, in step 3, , , , .
[0082] Furthermore, in step 3, the number of elite solutions... Mean and standard deviation The calculation formula is:
[0083] ,
[0084] ,
[0085] in, For the first The elite solution is in the... Dimension value, For the r-th elite solution, the ranking-based weights and concentration parameters are... .
[0086] Furthermore, in step 3, special handling is applied to boundary waypoints during the Bézier curve smoothing operation: the first waypoint Construct triples ,in The starting point; the last waypoint Construct triples ,in The endpoint.
[0087] Furthermore, the specific process of step 4 is as follows:
[0088] For the current iteration Update the adaptive learning rate.
[0089] ,
[0090] in, The initial learning rate, This is the final learning rate;
[0091] For each individual Based on the current state and Q table ,use - Greedy strategy for selecting actions:
[0092] ,
[0093] in, For the exploration rate parameter.
[0094] Furthermore, the specific process of step 5 is as follows:
[0095] Cost of calculating the new solution ;
[0096] Define an immediate reward function based on cost improvement:
[0097] ,
[0098] in, For the first In the next iteration, the individual Location, The updated position;
[0099] Update the Q-table according to the Q-learning update rules:
[0100] ,
[0101] in, For adaptive learning rate, Discount factor;
[0102] like Then accept the new solution. Otherwise, keep the current position.
[0103] Furthermore, the specific process of step 6 is as follows:
[0104] For each individual If the cost of the new solution is better, that is Then update the individual location. ;
[0105] Update the global optimal solution.
[0106] ,
[0107] in, The operator returns the cost function. Individual index that obtains the minimum value ;
[0108] Record the optimal cost value of the current iteration. ;
[0109] At the same time, update the number of iterations. ;
[0110] like Return to step 4 to continue iterative optimization; if The iteration ends, and we proceed to step 7.
[0111] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0112] In terms of system modeling, the path planning method of this invention constructs a comprehensive cost function including path length, collision avoidance, height constraints, and smoothness, providing explicit collision evaluation conditions and a hierarchical height penalty mechanism. Regarding algorithm design, each search agent acts as an independent intelligent agent, adaptively selecting an optimization strategy through Q-learning. Four cooperative operations are designed to comprehensively address the issues of exploration-development balance, path feasibility, and trajectory regularity. Furthermore, considering the influence of flight dynamics constraints and complex three-dimensional multi-obstacle environments, the designed reinforcement learning-enhanced continuous domain ant colony optimization algorithm generates high-quality, smooth, and safe flight paths, achieving optimal solutions in all four test cases. Compared to the second-best algorithm, it shows an average performance improvement of 8.3%, exhibiting fast convergence speed and good stability. Attached Figure Description
[0113] Figure 1 This is a flowchart of the optimization algorithm of the present invention.
[0114] Figure 2 This is a comparison of the convergence curves of different algorithms in four test cases.
[0115] Figure 3 A comparison chart of 3D path planning for different algorithms in four test cases.
[0116] Figure 4 A top-down comparison diagram of the paths of different algorithms in four test cases.
[0117] Figure 5 Box plots comparing different algorithms across four test cases. Detailed Implementation
[0118] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings.
[0119] This invention discloses a reinforcement learning-based continuous domain ant colony optimization method for 3D path planning of unmanned aerial vehicles (UAVs). The flowchart of this method is as follows: Figure 1 As shown, the specific steps are as follows:
[0120] Step 1: Path From the starting point To the finish line ,Include There are 1 waypoint, and the spatial coordinates of the j-th waypoint on the path are P. i,j , ,in .
[0121] The path length cost is expressed as:
[0122] ,
[0123] in, Indicates adjacent waypoints and The Euclidean distance between them.
[0124] Can wrap the first The center of the smallest cylinder of the obstacle In practice, d is calculated kij The coordinates C of the cylinder's center will be used. k ,radius ,high Let the radius of the drone be... Danger distance is Safety threshold The collision cost is:
[0125] ,
[0126] Where K is the total number of obstacles,
[0127] ,
[0128] and These are the altitude coordinates of two adjacent waypoints. This is the horizontal distance from the center of the cylinder to the path segment.
[0129] High costs introduce tiered penalties:
[0130] ,
[0131] in, waypoints height, For the ideal height, and This is the penalty coefficient.
[0132] Calculate path segment Horizontal projection:
[0133] ,
[0134] in, for Unit vector in the positive direction of the axis. This represents the cross product of vectors.
[0135] Calculate the turning angle between adjacent path segments:
[0136] ,
[0137] in, Represents the vector dot product. Represents the magnitude of a vector.
[0138] Calculate the climb angle:
[0139] ,
[0140] Among them, molecules The difference is the height, and the denominator is the horizontal distance.
[0141] Smoothing cost:
[0142] ,
[0143] in, For the first Turns at waypoints For the climb angle, and This represents the corresponding penalty coefficient.
[0144] Total cost function:
[0145] ,
[0146] in, , These are the weighting coefficients for each cost component.
[0147] Step 2: Set the population size to Randomly initialize within the search space There are individuals, and the i-th individual is... The coordinates of its n waypoints can be represented as Ant colony algorithm individual Represents a complete path scheme, i.e., an individual The represented path scheme can be denoted as: .
[0148] Configure an independent Q table for each individual All Q-tables are initialized to zero matrices. The state space is defined. These correspond to four types of cooperative operations. Define the action space. The actions This indicates that the m-th cooperative operation will be selected. .
[0149] Calculate the cost for each individual Find the global optimal solution Set the maximum number of iterations. Initialize the current iteration count .
[0150] Step 3: Design four collaborative operation strategies.
[0151] Quantum Behavior Exploration Operation:
[0152] Generate Lévy step size:
[0153] ,
[0154] in, Let be a random variable that follows a normal distribution. The Lévy index,
[0155] Formula for calculating standard deviation parameter:
[0156] ,
[0157] in, This is a gamma function.
[0158] The first step in generating a new solution Dimensional components:
[0159] ,
[0160] in, For the feature length parameter; ,for Uniformly distributed random numbers over an interval; It is a symbolic function; To select the guiding solution in the first The value of dimension d represents a coordinate component of a waypoint in the path; For the first Standard deviation of dimension; log factor Simulate the quantum tunneling effect.
[0161] Multi-elite guided development operation:
[0162] Define elite set ,in For the number of elite solutions, assign a ranking-based weight to each elite solution: , .
[0163] Constructing a new solution using the Gaussian kernel function:
[0164] ,
[0165] The new solution represents the first... Dimensional components from mean to Standard deviation is Sampling is performed within a normal distribution.
[0166] The formulas for calculating the weighted mean and standard deviation are:
[0167] ,
[0168] ,
[0169] in, For the first The elite solution is in the... Dimension value, concentration parameter .
[0170] Introducing the optimal solution bias:
[0171] ,
[0172] Among them, the bias intensity coefficient , For the current optimal individual in the th... The value of the dimension.
[0173] Bézier curve smoothing operation:
[0174] For three consecutive waypoints ( Construct a quadratic Bézier curve:
[0175] ,
[0176] in, These are curve parameters.
[0177] A weighted mixing strategy is used to avoid oversmoothing.
[0178] ,
[0179] in, These are the smoothing weighting coefficients.
[0180] Special handling for boundary waypoints: the first waypoint Constructing triples ,in The starting point; the last waypoint Constructing triples ,in The endpoint.
[0181] Information entropy minimization adjustment operation:
[0182] Definition of changes from a comprehensive perspective:
[0183] ,
[0184] in, For the first Horizontal turning angle at each waypoint Let be the climb angle for the j-th waypoint.
[0185] Normalized to a probability distribution:
[0186] ,
[0187] Calculate the entropy of the direction change of the path:
[0188] ,
[0189] The ideal position of each waypoint is obtained by minimizing the entropy of directional changes:
[0190] ,
[0191] Waypoint updates employ an adaptive step size strategy:
[0192]
[0193] Among them, the adjustment coefficient , This represents the current iteration number. This represents the maximum number of iterations.
[0194] Step 4: For the current iteration Update the adaptive learning rate:
[0195] ,
[0196] in, The initial learning rate, This is the final learning rate.
[0197] For each individual ( Based on the current state and Q table ,use - Greedy strategy for selecting actions:
[0198] ,
[0199] in, For the exploration rate parameter.
[0200] Step 5: Calculate the cost of the new solution .
[0201] Define an immediate reward function based on cost improvement:
[0202] ,
[0203] in, For the first In the next iteration, the individual Location, This is the updated position.
[0204] Update the Q-table according to the Q-learning update rules:
[0205] ,
[0206] in, The adaptive learning rate in step 4. This is the discount factor.
[0207] like Then accept the new solution. Otherwise, keep the current position.
[0208] Step 6: For each individual If the cost of the new solution is better, that is Then update the individual location. .
[0209] Update the global optimal solution:
[0210] ,
[0211] in, The operator returns the cost function. Individual index that obtains the minimum value .
[0212] Record the optimal cost value of the current iteration. .
[0213] At the same time, update the number of iterations. .like Return to step 4 to continue iterative optimization; if Terminate the iteration and proceed to step 7;
[0214] Step 7: Return the global optimal solution This serves as the final 3D path planning solution for unmanned aerial vehicles.
[0215] The optimal path includes each waypoint , .in, For the first The three-dimensional coordinates of each waypoint.
[0216] Example 1
[0217] Using this method for UAV path planning, in (Unit: meters) Four test cases with different complexities are constructed in three-dimensional space. The parameters of the four test cases are as follows, with obstacles represented by O. k [Center coordinates (x)] k y k Height H k radius R k ]:
[0218] Case 1: Starting point S(100,100,150), ending point T(900,700,150), containing 5 obstacles with obstacle parameters O1[300,250,400,80], O2[700,550,430,80], O3[500,400,400,100], O4[400,600,430,70], O5[600,200,480,60];
[0219] Case 2: Starting point S(150,150,140), ending point T(850,550,160), containing 6 obstacles with obstacle parameters O1[650,600,310,85], O2[250,500,350,70], O3[350,300,390,90], O4[750,300,420,75], O5[550,400,470,110], O6[730,620,310,90];
[0220] Case 3: Starting point S(120,120,150), ending point T(880,680,150), containing 7 obstacles with obstacle parameters O1[250,200,400,80], O2[700,550,435,85], O3[575,425,400,80], O4[400,600,390,75], O5[400,350,470,95], O6[650,250,480,75], O7[270,650,435,85];
[0221] Case 4: Starting point S(180,100,145), ending point T(820,700,155), containing 7 obstacles with obstacle parameters O1[500,250,325,80], O2[700,550,430,75], O3[250,500,400,65], O4[750,350,420,70], O5[300,250,480,85], O6[500,450,450,80], O7[350,280,470,75].
[0222] The reinforcement learning-enhanced continuum ant colony optimization method proposed in this invention is simplified and named RLACO. R Algorithms. For comparison, six advanced representative algorithms were selected: RLGWO, RLWOA, ACOSRA, and others. R The algorithms used are MSHHO, CFSSA, and PSO. Each algorithm is run independently 20 times for each case.
[0223] Figure 2 RLACO is displayed. R Excellent convergence performance was achieved in all four test cases. In a simple environment, RLACO... R It converges quickly and demonstrates excellent ability to escape local optima. While other algorithms show decreased convergence speed and increased tendency to reach local optima as complexity increases, RLACO... R Maintain rapid and stable convergence to the lowest cost value.
[0224] Figure 3 and Figure 4The flight path is displayed from both 3D and top-down perspectives. (RLACO) R The generated path has the shortest length while maintaining a safe distance from obstacles, and the turns are smooth and natural, consistent with the drone's dynamic characteristics. In contrast, PSO exhibits a noticeable meandering motion, while RLGWO, RLWOA, and ACOSRA... R They tend to find local optima. MSHHO and CFSSA have limited optimization capabilities in complex environments.
[0225] Figure 5 Using box plots to illustrate RLACO R It not only finds high-quality solutions but also maintains stable performance, with the bin position significantly lower than other algorithms and a smaller bin height. RLACO R It achieved the best results across all cases, with the optimal value being an average improvement of 8.3% compared to the second-best algorithm. In terms of average performance, RLACO... R RLACO maintains the lowest average cost across 20 runs, demonstrating good robustness. In complex environments where the average of other algorithms deviates significantly from the optimal value, RLACO... R Maintaining relatively small fluctuations. Regarding standard deviation, RLACO... R Maintain a relatively low value across all cases.
[0226] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.
Claims
1. A reinforcement learning-based continuous domain ant colony optimization method for 3D path planning of unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Step 1: Establish a mathematical model for UAV path planning and construct a comprehensive cost function that includes path length, collision avoidance, altitude constraints, and smoothness. Step 2: Initialize the reinforcement learning-enhanced continuous domain ant colony optimization framework, configure an independent Q-table for each individual, and define the state space and action space. The specific process is as follows: Set the population size to Random initialization If there are 10 individuals, then the i-th individual is... , ,individual The path scheme represented is denoted as The coordinates of its n waypoints are represented as follows: ,in Provide the three-dimensional coordinates of the j-th waypoint; configure an independent Q-table for each individual. All Q-tables are initialized to zero matrices; the state space is defined. These correspond to the four collaborative operations designed later; define the action space. The actions This indicates that the m-th cooperative operation will be selected. ; Calculate the cost for each individual using the cost function defined in step 1. Find the global optimal solution ; Set the maximum number of iterations. Initialize the current iteration count ; Step 3: Design four cooperative operations, namely quantum behavior exploration operation, multi-elite guided development operation, Bézier curve smoothing operation, and information entropy minimization adjustment operation; Specifically, the quantum behavior exploration operation is as follows: The step size is generated using the Lévy flight mechanism, representing the breadth of the exploration range: ， in, All of them are random variables that follow a corresponding normal distribution. Lévy index; The standard deviation is expressed by the following formula: ， It is a gamma function; Generate the d-th dimension component of the new solution. ， Where L is the feature length parameter. The Lévy index, ,for Uniform random numbers over an interval It is a symbolic function; To select the guiding solution in the first... The value of the dimension; For the first The standard deviation of dimension d, where d represents a coordinate component of a waypoint in the path; logarithmic factor Simulate quantum tunneling effect; The multi-elite guided development operation is specifically as follows: Define elite set ,in For the number of elite solutions, assign a ranking-based weight to each elite solution: , , Constructing a new solution using the Gaussian kernel function: ， The new solution represents the first... Dimensional components from mean to Standard deviation is Sampling from a normal distribution, Introducing the optimal solution bias: ， in, The bias intensity coefficient, For the current optimal individual in the th... The value of the dimension; The Bézier curve smoothing operation is as follows: For three consecutive waypoints Constructing a quadratic Bézier curve: ， in, For curve parameters, A weighted mixing strategy is used to process the quadratic Bézier curve to avoid over-smoothing. ， in, For smoothing weighting coefficients; The specific operation of minimizing information entropy is as follows: Calculate the entropy of the direction change of the path: ， To normalize to a probability distribution, ; To take into account changes from all angles, , Let the turning angle be at the j-th waypoint. Let the angle of ascent be the angle of ascent for the j-th waypoint; The ideal position of each waypoint is obtained by minimizing the entropy of directional changes. ， Waypoint updates employ an adaptive step size strategy. ， Among them, the adjustment coefficient , This represents the current iteration number. This represents the maximum number of iterations. Step 4: Perform Q-learning adaptive strategy selection, through... - A greedy strategy selects cooperative actions for each individual; Step 5: Evaluate the new solution and update the Q-table, define an immediate reward based on the cost improvement, and update the Q-value using an adaptive learning rate; Step 6: Update the population and the global optimal solution, and determine whether the maximum number of iterations has been reached. If not, return to step 4 to continue iterating; otherwise, proceed to step 7. Step 7: Using the globally optimal solution As the final UAV 3D path planning solution, the optimal path includes Waypoints , .
2. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 1, characterized in that, The specific process of step 1 is as follows: For any path P i From the starting point S to the ending point T, there are n waypoints, where i is the path number. Then path P... i The spatial coordinates of the j-th waypoint on the [plane name] are P. i,j , , Let J be the horizontal coordinate of the j-th waypoint on the i-th path. Let J be the vertical coordinate of the j-th waypoint on the i-th path. Let J be the altitude coordinates of the j-th waypoint on the i-th path; Then path length cost for, ， in, Indicates adjacent waypoints and The Euclidean distance between them; Collision costs for, ， in, Let K be the number of obstacles. For collision assessment parameters; High cost for, ， Let j be the altitude of the j-th waypoint. The ideal flight altitude is set according to mission requirements. This is the penalty coefficient for low-altitude flight. This is the penalty coefficient for high-altitude flight. Let the minimum safe altitude be at the j-th waypoint. This represents the maximum permissible altitude at the j-th waypoint; Smoothing cost for, ， and The penalty coefficient for smoothness cost. Let the turning angle be at the j-th waypoint. Let be the climb angle for the (j+1)th waypoint, and let be the climb angle for the j-th waypoint; Therefore, the total cost function for, ; These are the weighting coefficients for different costs.
3. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 2, characterized in that, The collision evaluation parameters are: ， and These are the altitude coordinates of two adjacent waypoints. This is the horizontal distance of the path segment obtained by connecting the center of the cylinder to two adjacent waypoints; As a safety threshold, D is the radius of the drone, and L is the danger distance. In order to wrap the first The minimum radius of the cylinder containing the obstacle. In order to wrap the first The minimum cylinder height for an obstacle.
4. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 2, characterized in that, Angle of ascent at waypoint j The calculation formula is as follows: ， Among them, molecules The denominator is the altitude difference between two adjacent waypoints, and the horizontal distance of the path segment is the denominator. Turning angle between adjacent path segments: ， in, Represents the vector dot product. Represents the magnitude of a vector. For path segment Horizontal projection, ， for Unit vector in the positive direction of the axis. This represents the cross product of vectors.
5. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 1, characterized in that, In step 3, , , , .
6. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 1, characterized in that, In step 3, the number of elite solutions Mean and standard deviation The calculation formula is: ， ， in, For the first The elite solution is in the... Dimension value, For the r-th elite solution, the ranking-based weights and concentration parameters are... .
7. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 1, characterized in that, In step 3, special handling of boundary waypoints during the Bézier curve smoothing operation: the first waypoint Construct triples ,in The starting point; the last waypoint Construct triples ,in The endpoint.
8. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 1, characterized in that, The specific process of step 4 is as follows: For the current iteration Update the adaptive learning rate. ， in, The initial learning rate, This is the final learning rate; For each individual Based on the current state and Q table ,use - Greedy strategy for selecting actions: ， in, For the exploration rate parameter.
9. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 1, characterized in that, The specific process of step 5 is as follows: Cost of calculating the new solution ; Define an immediate reward function based on cost improvement: ， in, For the first In the next iteration, the individual Location, The updated position; Update the Q-table according to the Q-learning update rules: ， in, For adaptive learning rate, Discount factor; like Then accept the new solution. Otherwise, keep the current position.
10. The reinforcement learning-based continuous domain ant colony optimization method as described in claim 1, characterized in that, The specific process of step 6 is as follows: For each individual If the cost of the new solution is better, that is Then update the individual location. ; Update the global optimal solution. ， in, The operator returns the cost function. Individual index that obtains the minimum value ; Record the optimal cost value of the current iteration. ; At the same time, update the number of iterations. ; like Return to step 4 to continue iterative optimization; if The iteration ends, and we proceed to step 7.