Unmanned aerial vehicle swarm coordination method based on fuzzy ant colony algorithm and proximal policy optimization combination

Abstract

The unmanned aerial vehicle group cooperative method based on the combination of fuzzy ant colony algorithm and proximal policy optimization comprises the following steps: 1) adaptive clustering is used to assign task areas to unmanned aerial vehicles, and the task range is reduced; 2) fuzzy rules and membership functions are formulated, and the fuzzy ant colony algorithm is used to solve the first-level task planning between different subgroups in the target group, so that the cost minimization of each unmanned aerial vehicle in executing the subgroup task is ensured; 3) the proximal policy optimization is combined with the particle swarm optimization method, and is used for the task planning optimization in the target subgroup, so that the planning accuracy and efficiency are improved. The present application can improve the intelligent control and efficiency of unmanned aerial vehicles in the task execution process through the cooperative task planning of the fusion of time domain and space domain features, and further enhance the task execution efficiency of the unmanned aerial vehicle group.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent decision optimization, and in particular provides a UAV group coordination method based on the combination of fuzzy ant colony algorithm and proximal policy optimization. BACKGROUND

[0002] The rapid development of unmanned aerial vehicle (UAV) technology has enabled its widespread application in military, logistics, environmental monitoring, and other fields. In complex task environments, the capabilities of a single UAV are limited, making it difficult to efficiently complete tasks. Therefore, studying the cooperative task planning of UAV groups has become a key to improving task efficiency and reliability. When multiple UAVs collaborate to perform tasks, it is necessary to allocate tasks and plan paths reasonably to ensure efficient task completion. Traditional methods often have low efficiency when dealing with complex environments and multiple tasks, making it difficult to meet real-time requirements. During the execution of tasks by UAV groups, dynamic changes in the environment may occur, such as sudden obstacles and weather changes. How to enable UAV groups to have adaptive capabilities to respond to environmental changes is an important challenge in cooperative task planning.

[0003] In recent years, with the rapid development of artificial intelligence, communication technology, and UAV manufacturing technology, multi-UAV systems and swarm technology have gradually become a research hotspot for various countries. For example, the "Quail" project led by the U.S. Air Force successfully demonstrated the ability of more than 100 UAVs to collaborate in reconnaissance and target strikes in a battlefield environment (SUAS, 2016). China has also made significant progress in UAV swarm technology, with the China Electronics Technology Group demonstrating the feasibility of coordinated flight of hundreds of UAVs for military and civilian applications (Wei Yunfeng et al., New Reading, 2017). In addition, the Dutch military (defensie.nl, 2018), one of the applications of the European "HORNET" project, is exploring intelligent decision-making algorithms for using UAV groups to perform complex tasks. Multi-UAV systems are constantly driving the development of swarm intelligence technology by drawing on the wisdom of group-living organisms in nature.

[0004] Proximal Policy Optimization (PPO) is a deep reinforcement learning algorithm that has fast convergence speed and high stability. In UAV task planning, PPO can learn the optimal policy through interaction with the environment, enabling autonomous decision-making and path planning for UAVs. For example, Jiao Weidong (Aviation Computing Technology, 2024) et al. used the PPO algorithm for UAV attitude control, achieving good results.

[0005] Ant Colony Optimization (ACO) is a swarm intelligence optimization algorithm that simulates the foraging behavior of ants and is suitable for solving path planning problems. In UAV mission planning, the ACO can be used to find the optimal flight path, avoid obstacles, and reduce the probability of being discovered. For example, Shao Changxu (Ordnance Automation, 2018) et al. proposed a UAV route planning method based on the ACO and verified its effectiveness. To this end, Zhang Lin (Northwestern Polytechnical University, 2020) introduced fuzzy logic into the ACO to form a fuzzy ant colony algorithm to improve the algorithm's global search capability and convergence speed. In UAV mission planning, the fuzzy ant colony algorithm can more effectively handle uncertainty and complexity, and improve the efficiency and reliability of mission planning. In a dynamic environment, task allocation and path planning need to consider multiple factors such as task priority, time constraints, and environmental changes at the same time, which makes the complexity of problem solving increase dramatically.

[0006] Most existing evaluation methods are difficult to obtain the global optimal solution in the complex environment of large-scale drone groups. They may have problems such as slow convergence and easy falling into local optimality, and may cause performance degradation in some interference environments. Summary of the Invention

[0007] In order to overcome the above-mentioned shortcomings of the existing technology and improve the intelligent control and efficiency of drones during mission execution, the present invention proposes a drone group collaborative task method based on the combination of fuzzy ant colony algorithm and proximal strategy optimization.

[0008] When drones perform ground missions, efficient execution of multi-drone collaborative tasks is achieved by optimizing pre-mission collaborative path planning. First, the clustering results are optimized through an adaptive clustering method to determine the optimal task area division scheme. The present invention uses a fuzzy ant colony algorithm to solve the first-level task planning between different subgroups in the target group to ensure that the cost of each drone is minimized when performing subgroup tasks. Secondly, the proximal strategy optimization is combined with the particle swarm optimization method to optimize the task planning within the target subgroup, thereby improving planning accuracy and efficiency. Finally, through the integration of time domain and airspace characteristics in collaborative task planning, the overall collaborative combat capability and task execution efficiency of the drone group are further improved.

[0009] In order to solve the technical problem, the present invention adopts the following technical solution:

[0010] The first aspect of the present invention relates to a drone swarm collaboration method based on a combination of a fuzzy ant colony algorithm and proximal strategy optimization, comprising:

[0011] S1, using an adaptive optimization method combining hierarchical clustering and DBSCAN clustering to adjust the task area allocation according to the location of the UAV group and the target;

[0012] S2, introducing a fuzzy ant colony algorithm based on the adaptive clustering allocation result, combining fuzzy rules to dynamically adjust pheromone intensity, assigning task target points to individual intelligent unmanned aerial vehicles and optimizing their path planning;

[0013] S3, preliminary path planning based on fuzzy ant colony, using proximal policy optimization algorithm to dynamically adjust the path selection of individual unmanned aerial vehicles, and combining particle swarm optimization algorithm to globally search and optimize the path, thereby generating the optimal mission path of the unmanned aerial vehicle group.

[0014] The step S1 comprises:

[0015] The coordinate matrix of the task target points and the coordinate matrix of the unmanned aerial vehicle starting point are taken as inputs, and a Euclidean distance matrix D is constructed based on the task target point set. Hierarchical clustering and DBSCAN clustering are applied to the target points respectively to generate different clustering results. Then, the comprehensive quality score is calculated according to the clustering results, including the contour coefficient, Calinski-Harabasz index and Davies-Bouldin index. The clustering scheme with the highest score is selected as the basis for the allocation of unmanned aerial vehicle mission areas, achieving the optimization of task allocation.

[0016] Further, the step S1 comprises the following specific steps:

[0017] S1.1 uses the Ward method to gradually merge the independent clusters of the task target points initialized to minimize the square error increment between clusters until the number of unmanned aerial vehicles is reached, completing hierarchical clustering. The Ward method merging formula is:

[0018]

[0019]

[0020] where (x i ,y i ) and (x j ,y j ) are the centers of clusters C i and C j , respectively, and the initial family is each task target point.

[0021] S1.2 uses the DBSCAN method to cluster the task target points, where the elements of the Euclidean distance matrix D are calculated by the following formula:

[0022]

[0023] where (x p ,y p ) and (x q ,y q) are the coordinates of target point p and target point q; by setting the neighborhood radius and the minimum number of samples, the high-density region is divided into clusters, and the sparse region is marked as a noise point; the noise point is assigned to the nearest base station according to the Euclidean distance from the base station, ensuring that all target points are effectively covered;

[0024] S1.3 Quality evaluation of S1.1 and S1.2 clustering schemes by contour coefficient, clustering density and clustering tightness, the comprehensive score formula is as follows:

[0025]

[0026]

[0027] Wherein, a(i) represents the average distance of sample i and other points in the same cluster, b(i) represents the average distance of sample i and other points in the nearest cluster, μ is the mean value of the whole data set, is the mean value of the jth cluster C j , n is the number of data points, k is the number of clusters, s i represents the average dispersion within cluster i, d ij represents the distance between cluster i and cluster j; finally, the clustering scheme with the highest comprehensive score is selected as the optimal basis for the allocation of the unmanned aerial vehicle task area.

[0028] Wherein, step S2 comprises:

[0029] In step S1, the task area is divided into multiple smaller task areas and allocated to the unmanned aerial vehicle group; each unmanned aerial vehicle selects a path according to the pheromone concentration and target point distance on the path; in path planning, the pheromone update strength is dynamically adjusted by introducing fuzzy rules, considering target distance, obstacle density, energy consumption and task urgency factors, and optimizing path selection; fuzzy reasoning converts input variables into accurate values of adjusting pheromones, and shorter paths obtain more pheromones, thereby increasing the probability of being selected; after updating for the maximum number of iterations, the pheromones gradually converge, and the global optimal path is finally found.

[0030] Further, step S2 comprises the following specific steps:

[0031] S2.1 Assign the task area allocated by S1 to the adjacent unmanned aerial vehicle group, and regard each task cluster as an ant colony, wherein the number of samples in the cluster corresponds to the number of unmanned aerial vehicles;

[0032] S2.2 Each unmanned aerial vehicle starts from its starting base, randomly selects the next target node based on the pheromone concentration and the distance between target points on the path, and gradually constructs the path; the probability calculation formula of path selection is:

[0033]

[0034] where T ij denotes the pheromone concentration from node i to node j, d ij denotes the distance between nodes;

[0035] S2.3 fuzzy rules are introduced in path selection, considering target distance, obstacle density, energy consumption and task urgency, and the pheromone concentration is dynamically adjusted through fuzzy reasoning to optimize the path selection process;

[0036] S2.4 after each iteration, the pheromone on the path is updated, and more pheromone is accumulated on the shorter path to increase the selection probability; within the maximum number of iterations, the pheromone is constantly optimized, and the path gradually converges to the global optimum, completing the efficient allocation and planning of UAV tasks.

[0037] Further, in step S2.3, the fuzzy reasoning process is as follows:

[0038] S2.3.1 the target distance, obstacle density, energy consumption and task urgency are converted into fuzzy set membership degrees through membership functions, as follows:

[0039] The target distance is divided into three fuzzy sets: "Near", "Medium" and "Far", and the membership function is defined as follows:

[0040]

[0041] The obstacle density ρ is divided into three fuzzy sets: "Low", "Medium" and "High", and the membership function is defined as follows:

[0042] μ Low (ρ)=-e -5ρ (11)

[0043]

[0044] μ High (ρ)=1-e -5ρ (13)

[0045] The energy consumption E is divided into three fuzzy sets: "Low", "Medium" and "High", and the membership function is defined as follows:

[0046]

[0047] The task urgency U is divided into three fuzzy sets: "Low", "Medium" and "High", and the membership function is defined as follows:

[0048] μ Low (U)=-e-5U (17)

[0049]

[0050] μ High (U)=1-e -5U (19)

[0051] S2.3.2 According to the formulated fuzzy rules, fuzzy reasoning is carried out in combination with membership degrees, and the fuzzy rule table is as follows:

[0052]

[0053]

[0054] Among them, "+", "o" and "-" respectively represent increasing, moderate and decreasing pheromone update amount; Overall, when the target distance is short, the obstacle density is low, the energy consumption is low and the task urgency is high, the pheromone is increased, and the current path is preferentially selected; When the target distance is far, the obstacle density is high, the energy consumption is high and the task urgency is low, the pheromone is reduced, and other paths are encouraged to explore; When there is a contradiction between variables (such as task urgency but high obstacle density), the pheromone update amount is dynamically adjusted according to the task priority and resource situation, so as to realize intelligent optimization of path selection;

[0055] S2.3.3 The fuzzy reasoning result is de-fuzzied by the barycentric method, which is converted into an accurate numerical value, which is used to dynamically adjust the pheromone concentration on the path; The pheromone on the path is updated according to the following formula:

[0056]

[0057] T ij (t+1)=(1-ρ)·T ij (t)+ΔT ij (21)

[0058] Among them, A k is the set of ants in the ant colony, L k is the path length of ant k, Q is the total weight of the task, and p is the volatilization coefficient, which is used to control the decay rate of pheromone;

[0059] Among them, step S3 comprises:

[0060] Based on step S2, the path planning is modeled as a Markov decision process, the strategy network is optimized by a proximal policy optimization method, the path length, the energy consumption and the task priority are taken as a reward function, and an optimal path strategy capable of balancing time and energy constraints is generated; subsequently, the global search ability of a particle swarm algorithm is combined to further optimize the path, the deficiency of local optimization is made up, and finally more efficient and more intelligent UAV path planning is realized.

[0061] Further, step S3 includes the following specific steps:

[0062] S3.1 modeling the path planning problem as a Markov decision process, wherein the state represents the current path information of the UAV, including the current position, the visited target points and the remaining task target points; the action corresponds to the next flight node selected by the UAV; an explicit reward function is constructed by the path length, the energy consumption and the task priority, to evaluate the quality of each step path selection, while balancing the time and energy constraints; by sampling the path trajectory, the cumulative reward is calculated, and the strategy is constantly adjusted, so that the UAV gradually generates a path planning strategy capable of maximizing the reward function, thereby realizing more efficient path optimization and ensuring task completion;

[0063] S3.2 further optimizing the path by using the global search ability of the particle swarm optimization based on the path generated in step S3.1; finally, the global optimal path that minimizes the fitness function is found, and the global optimization of the UAV path planning is realized.

[0064] The second aspect of the application relates to a UAV group coordination device based on the combination of fuzzy ant colony algorithm and proximal policy optimization, comprising a memory and one or more processors, the memory stores executable code, and the one or more processors execute the executable code to realize the UAV group coordination method based on the combination of fuzzy ant colony algorithm and proximal policy optimization.

[0065] The third aspect of the application relates to a computer readable storage medium having a program stored thereon, which is executed by a processor to realize the UAV group coordination method based on the combination of fuzzy ant colony algorithm and proximal policy optimization.

[0066] The working principle of the application is:

[0067] The whole UAV group cooperative task planning process can be divided into four stages of "task area division → task initial allocation and preliminary path planning → local dynamic adjustment → global optimization", and each stage has corresponding algorithms and methods to realize. Hierarchical clustering and DBSCAN: adaptively divide the task area into multiple reasonable sub-regions by considering density and hierarchy at the same time, reduce the problem of repeated flight or conflict when the UAV is dispatched disorderly in a large range. Fuzzy ant colony algorithm: through the fusion of bionics and fuzzy theory, the relatively optimal initial scheme is obtained in the local (sub-region), and multiple constraints and environmental uncertainties are considered. Near-end strategy optimization: through the "trial and error + feedback" mechanism of reinforcement learning, the flight path of the current UAV is perceived and corrected in real time, and adaptive adjustment is provided for local emergencies. Particle swarm optimization: use swarm intelligence algorithm to iteratively update the overall layout and route of the UAV in a larger search space, avoid fragmentation or suboptimization caused by local dynamic decision-making, and ensure better overall efficiency of the UAV group.

[0068] The advantages of the present application are:

[0069] The UAV group task planning method combining "hierarchical clustering + DBSCAN adaptive partitioning", "fuzzy ant colony algorithm preliminary path planning" and "PPO + PSO dynamic and global optimization" proposed based on the patent has the following advantages:

[0070] 1. Through the combination of hierarchical clustering and DBSCAN, dense areas, sparse distribution and noise points can be adaptively identified, complex scenarios such as uneven distribution of task points or sudden increase and decrease of task points can be effectively dealt with; the partitioning is more reasonable, the search space for subsequent task allocation and path planning is effectively reduced, and repeated flight and conflict are reduced.

[0071] 2. Fuzzy logic is introduced based on the traditional ant colony algorithm, and comprehensive evaluation is performed through the "fuzzy ant colony algorithm"; the initial allocation and path scheme generated by the fuzzy ant colony algorithm is more robust than the pure ant colony algorithm and performs better in uncertain or multi-constraint environments.

[0072] 3. Near-end strategy optimization (PPO) can perceive and adapt to environmental changes in real time during task execution, and realize rapid fine tuning of the local path of the UAV; particle swarm optimization (PSO) periodically or triggeringly re-optimizes the overall path or task allocation from a global perspective, effectively avoiding local optimization and ensuring overall efficiency.

[0073] 4. The "real-time online learning ability" of reinforcement learning (PPO) is combined with the "global search" ability of swarm intelligence algorithms (PSO, ant colony), which can not only cope with instantaneous disturbances and emergencies, but also ensure the optimal long-term planning goal; the training process converges more smoothly by limiting the difference between new and old strategies, and the deployment is safer and more reliable.

[0074] 5. Each sub-module (clustering method, task allocation algorithm, local / global optimization algorithm) has certain replaceability: the clustering method (K-means + +, etc.) can be replaced according to the scene, the fuzzy rules or parameters of the fuzzy ant colony algorithm can be adjusted, and other reinforcement learning algorithms and optimization algorithms can be combined; suitable for various unmanned aerial vehicle group sizes and multiple task scenarios, such as large-scale target patrol, disaster monitoring, logistics distribution and military reconnaissance, etc.

[0075] 6. Through adaptive partitioning, intelligent allocation and local-global dual optimization, mutual interference and redundant flight between unmanned aerial vehicles are significantly reduced; at the same time, better balance can be achieved in terms of low energy consumption, low latency and high security. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 is a flowchart of an unmanned aerial vehicle group coordination method based on the combination of fuzzy ant colony algorithm and proximal policy optimization.

[0077] Figure 2 is a flowchart of unmanned aerial vehicle group task allocation based on an adaptive clustering method.

[0078] Figure 3 is a flowchart of unmanned aerial vehicle task path allocation based on fuzzy ant colony.

[0079] Figure 4 is a flowchart of unmanned aerial vehicle task path optimization based on proximal policy optimization. DETAILED DESCRIPTION

[0080] The technical solutions of the present application will be further described below with reference to the accompanying drawings.

[0081] Example 1

[0082] Reference Figures 1-4 The unmanned aerial vehicle group coordination method based on the combination of fuzzy ant colony algorithm and proximal policy optimization includes the following steps:

[0083] S1, an adaptive optimization method combining hierarchical clustering and DBSCAN clustering is used to adjust the task area allocation according to the positions of the unmanned aerial vehicle group and the target;

[0084] In view of the problem that traditional single clustering algorithm is difficult to adapt to various scenes due to the complex distribution of task target points in the division of unmanned aerial vehicle task areas, a method combining hierarchical clustering and DBSCAN clustering is proposed. The method optimizes the clustering results by calculating a comprehensive score index to determine the optimal task area division scheme.

[0085] Specifically, the coordinate matrix of the task target point and the coordinate matrix of the unmanned aerial vehicle starting point are taken as inputs, and the task target point set is hierarchically clustered and DBSCAN clustered respectively to obtain different clustering results. Then, for each clustering result, the comprehensive score of its clustering quality is calculated to select the optimal clustering result as the basis for unmanned aerial vehicle task area allocation. Finally, the optimal target point cluster is associated with the unmanned aerial vehicle, and by analyzing the distance between clusters and the attributes of the unmanned aerial vehicle, a task allocation matrix is constructed to complete the task area division.

[0086] This method significantly improves the efficiency and rationality of task allocation in complex scenarios by introducing a clustering result optimization strategy, providing a high-quality basic area division scheme for subsequent unmanned aerial vehicle path planning and task execution. The specific process of step S1 is as follows:

[0087] S1.1, hierarchical clustering analysis

[0088] In a certain task scenario, the radar system monitors a number of target points, forming a target point set Target = {Target1, Target2, …, Target N}. These target points can be stationary targets on the ground or dynamic monitoring areas. In order to achieve effective task allocation and path planning of unmanned aerial vehicles, the relative positions between target points need to be determined first.

[0089] In this case, the Euclidean distance between target points needs to be calculated first to obtain the distance matrix D. The construction of distance matrix D can intuitively describe the spatial distribution relationship between target points, providing basic data support for subsequent clustering analysis and path planning. The formula for calculating the distance d ij is as follows:

[0090]

[0091] Where (x j ,y j ) represents the coordinates of target point j, and d ij represents the distance between the i-th target and the j-th target.

[0092] Next, each task target point is regarded as an independent cluster, and the number of initial clusters is N, i.e. the number of target points. At the same time, the center coordinates (x c ,y c ) of each cluster are recorded, and the center of each cluster is the coordinate of its corresponding target point at the beginning. Then Ward method is used to gradually merge clusters until the preset target cluster number k ≤ N is reached. Ward method is a clustering method based on variance minimization, which selects the merged clusters by minimizing the sum of squared deviations within the merged clusters, ensuring that each merged cluster has the best compactness. The specific merging process is as follows:

[0093]

[0094]

[0095]

[0096] where, and are the centroids (means) of clusters C i and C j respectively. Formula 2 describes the calculation of the distance between clusters based on the Ward method, formula 3 describes the calculation formula of the centroid, formula 4 describes the criterion for selecting the two clusters with the smallest distance to merge, and formula 4 describes the process of merging two clusters. Through this step-by-step merging method, the required target cluster number can be obtained, and the target points within the cluster are as close as possible.

[0097] S1.2, DBSCAN clustering analysis

[0098] For any target point, its ε-neighborhood is defined as follows:

[0099] N ε (p) = {q | d(p, q) ≤ ε} (6)

[0100] where d(p, q) represents the distance between points p and q (usually using Euclidean distance), and its calculation formula is:

[0101]

[0102] where (x p , y p ) and (x q , y p ) are the coordinates of target point p and target point q.

[0103] Set the minimum sample size MinPts. If the number of points contained in the ε-neighborhood of point p |N ε (p) | ≥ MinPts, then point p is called a core target point; if point p is located in the ε-neighborhood of a core target point, but the number of points in its neighborhood is less than MinPts, then point p is called a boundary target point; if point p is neither a core target point nor located in the neighborhood of any core target point, then point p is called a noise target point.

[0104] Starting from any unvisited core target point p, mark it as visited and expand the cluster from it. First, take the ε-neighborhood N ε (p) of point p.All points in (p) are added to the current cluster if a point in the neighborhood is a core point, then continue to add points in its neighborhood to the current cluster until no new point can be added. For a point p, if its neighborhood N ε There are other core points in (p) (i.e. the number of points in the neighborhood is greater than or equal to MinPts, then recursively add the points in the neighborhood of these core points to the current cluster until all points are visited and assigned to a certain cluster.

[0105] This process ensures that all core points and points in their neighborhoods are correctly grouped into the same cluster, thereby achieving effective clustering of target points.

[0106] S1.3, Comprehensive analysis of clustering results

[0107] In order to select the optimal clustering result, the results of hierarchical clustering and DBSCAN clustering need to be scored. The comprehensive indicators of scoring include the following three: Silhouette Coefficient, Calinski-Harabasz (CH) index and Davies-Bouldin (DB) index.

[0108] The Silhouette Coefficient is used to measure the difference between the similarity of a sample to the samples in its own cluster and the dissimilarity to the samples in other clusters. Its value is between [-1, 1], the larger the value, the better the clustering effect. The calculation formula of Silhouette Coefficient is:

[0109]

[0110] Where a(i) represents the average distance of sample i to other points in the same cluster, and b(i) represents the average distance of sample i to other points in the nearest cluster.

[0111] The Calinski-Harabasz (CH) index measures the compactness and separation of the clustering clusters, the larger the value, the better the clustering effect. The calculation formula of CH index is:

[0112]

[0113] Where, is the trace of the inter-class distance difference matrix, is the trace of the intra-class dispersion matrix, μ is the mean of the entire data set, is the mean of the jth cluster C j , n is the number of data points, and k is the number of clusters.

[0114] The Davies-Bouldin (DB) index is used to measure the similarity between clusters, the smaller the value, the better the clustering effect. The calculation formula of DB index is:

[0115]

[0116] where s i represents the average dispersion within cluster i, d ij represents the distance between cluster i and cluster j.

[0117] Finally, the overall score of the clustering result can be calculated according to the above three indicators, and the calculation formula is:

[0118] Overall score = silhouette coefficient + CH index - DB index (11) By scoring the clustering results of hierarchical clustering and DBSCAN, the method with the highest overall score is selected as the final clustering scheme for this task area division. This process ensures the best balance between compactness, separation, and overall effectiveness of the clustering results, so as to provide a high-quality division scheme for the reasonable allocation of unmanned aerial vehicle tasks.

[0119] S2, initial allocation of unmanned aerial vehicle task path based on fuzzy ant colony

[0120] Learning from the behavior of ants in nature to find food paths, the path planning problem of unmanned aerial vehicles can be transformed into the process of finding the optimal path. Fuzzy ant colony optimization (FACO) combines traditional ant colony optimization and fuzzy logic, and introduces a fuzzy control mechanism to improve the robustness and adaptability of the algorithm, especially in complex and dynamic environments.

[0121] In step S1, the larger task area has been divided into several smaller task areas, and adjacent unmanned aerial vehicle groups have been allocated their responsible space, forming a grid-based task layout. In this case, each task cluster can be regarded as an ant colony, where the number of samples in the cluster corresponds to the number of ants in the colony, and each ant represents an unmanned aerial vehicle. Each unmanned aerial vehicle starts from its starting base position, randomly selects a neighboring node as the target node, and then continues to select the next node from that node. The selection probability of the path depends on the pheromone concentration on the current path and the heuristic information (such as the distance between nodes and the importance of tasks, etc.). The probability calculation formula for path selection is:

[0122]

[0123] where T ij represents the pheromone concentration from node i to node j, η ij = 1 / d ij is the heuristic factor, representing the distance between nodes; α and β are the weights of pheromone concentration and heuristic factor, respectively.

[0124] Through this mechanism, the ant colony algorithm simulates the process of a UAV finding the optimal path in a complex environment. The pheromone concentration plays a key role in path selection, while heuristic factors such as distance and task importance help the UAV make more rational decisions. This approach effectively balances global search and local exploration, providing a reliable solution for efficient path planning for UAV missions.

[0125] During the path selection process, the update strength of pheromones is adjusted by introducing fuzzy rules, allowing ants to intelligently avoid obstacles and choose better paths when searching for paths. Based on four key influencing factors: target distance, obstacle density, energy consumption, and task urgency, the following fuzzy rules are developed in combination with actual situations:

[0126] (1) When the target distance is far, the obstacle density is high, and the task urgency is low, the update amount of pheromones should be "reduced" to encourage ants to explore other paths.

[0127] (2) When the target distance is near, the obstacle density is low, and the task urgency is high, the update amount of pheromones should be "increased" to preferentially select the current best path.

[0128] (3) In the case of high energy consumption and high task urgency, paths with higher energy efficiency should be prioritized to reduce energy consumption.

[0129] (4) When both the obstacle density and task urgency are high, a trade-off between risk and efficiency is needed, and a complex but faster path may be chosen.

[0130] The formulation of these fuzzy rules allows the UAV to dynamically adjust its path selection strategy according to different environmental conditions, enabling more intelligent and efficient path planning. Specifically, the target distance d is divided into three fuzzy sets: "near" (Near), "medium" (Medium), and "far" (Far), with the corresponding membership functions defined as follows:

[0131]

[0132] where equation (13) uses an inverse s-type function, making the membership degree quickly tend to 1 in the near distance range; equation (14) uses a Gaussian function, with the highest membership degree in the medium distance range; and equation (15) uses an S-type function, making the membership degree quickly tend to 1 in the large distance range.

[0133] The obstacle density ρ is divided into three fuzzy sets: "low density" (Low Density), "medium density" (Medium Density), and "high density" (High Density), with the membership functions as follows:

[0134] μ Low (ρ)=-e -5ρ (16)

[0135]

[0136] μ High (ρ)=1-e -5ρ (18)

[0137] Wherein formula (16) adopts inverse exponential function, the membership is higher in low density range; Formula (17) adopts triangular membership function, ρ1, ρ2, ρ3 respectively take 0.3, 0.5, 0.7, and the peak value is located in medium density; Formula (18) adopts exponential function, and the membership increases rapidly in high density range.

[0138] Divide the energy consumption E into three fuzzy sets: "low consumption" (Low Consumption), "medium consumption" (Medium Consumption), "high consumption" (High Consumption), and the membership functions are as follows:

[0139]

[0140] Wherein formula (19) adopts inverse s function; Formula (20) adopts bell-shaped membership function, and realizes smooth transition to the middle region; Formula (21) adopts S function.

[0141] Divide the task urgency U into three fuzzy sets: "low urgency" (Low Urgency), "medium urgency" (Medium Urgency), "high urgency" (High Urgency), and the specific membership functions are as follows:

[0142] μ Low (U)=-e -5U (22)

[0143]

[0144] μ High (U)=1-e -5U (24)

[0145] Wherein formula (22) adopts inverse exponential function; Formula (23) adopts bell-shaped membership function, and reflects high membership of intermediate value; Formula (24) adopts exponential function.

[0146] Using these membership functions, the radar's actual input variables (such as target distance and obstacle density) are converted into corresponding fuzzy set memberships. These memberships are then combined with established fuzzy rules for fuzzy inference. Finally, using the centroid defuzzification method, the inference results are converted into precise numerical values, which are then used to dynamically adjust the ants' pheromone update strategies.

[0147] Under the preset maximum number of iterations k, the pheromone on the path will be updated after each iteration. Shorter paths will accumulate more pheromones, thereby increasing the probability of being selected by other ants and gradually finding the optimal path. The pheromone update formula is:

[0148]

[0149] Among them, A k represents the set of ants in the ant colony, ΔT ij represents the pheromone increment left by ant k on path i→j. The calculation method of incremental pheromone is:

[0150]

[0151] Among them, L k is the length of the path taken by ant k, and Q is the total weight of the task.

[0152] After each pheromone update, the pheromone will gradually evaporate, simulating the natural dissipation process of pheromones in reality. The pheromone volatilization formula is:

[0153] T ij (t+1)=(1-ρ)·T ij (t)+ΔT ij (27)

[0154] Among them, ρ is the volatility coefficient, which controls the decay rate of the pheromone.

[0155] When the set maximum number of iterations is reached, the ants stop searching for paths and select the path with the highest pheromone concentration as the preliminary optimal path.

[0156] S3. Reassignment of UAV mission paths based on proximal strategy optimization

[0157] S3.1 Path Optimization Based on PPO

[0158] Based on the fuzzy ant colony algorithm, a preliminary optimized path was obtained. The path planning problem was further modeled as a Markov decision process (MDP) to enhance the intelligence of path optimization. Specifically, the state represents the current path of the drone, and the action corresponds to the selection of the next flight node. To more comprehensively measure path quality, a reward function is defined as a weighted combination of path length, energy consumption, and task priority factors.

[0159] In this model, the PPO (Proximal Policy Optimization) algorithm is used to further optimize the path selection strategy, aiming to minimize the loss function. The objective function of PPO is in the form of:

[0160]

[0161] where r t (θ) represents the ratio of the current policy to the old policy; is the advantage function, which measures the goodness of the current action; ∈ is the clipping range to prevent the policy update from being too large.

[0162] By setting the number of iterations, PPO continuously optimizes the policy network and gradually finds the path planning strategy that can maximize the reward function. After training, the strategy generated by PPO can predict the optimal path, thereby further optimizing the flight path of the UAV based on the combination of the fuzzy ant colony algorithm, and improving the intelligence and overall efficiency of path selection.

[0163] S3.2 Path optimization based on PSO

[0164] The preliminary work has been done by training the policy network with PPO (Proximal Policy Optimization) to optimize path selection, so that the UAV can more effectively balance constraints such as time and energy consumption in path planning. To further refine path optimization, the particle swarm optimization (PSO) algorithm can be combined to perform global search and adjustment on the path, thereby finding a more optimal path solution.

[0165] In this stage, the idea of particle swarm optimization is to represent each particle as a possible path solution, with the dimension of the particle corresponding to each flight node in the path. The position of the particle represents a possible solution of the path, usually represented by the coordinates of each node, while the velocity of the particle represents the amount of change in the path.

[0166] In each iteration, the particle updates its velocity and position based on its historical best position and the global best position among all particles. The update formula is as follows:

[0167] v i (t+1)=ω·v i (t)+c1·r1·(p best -x i (t))+c2·r2·(g best -x i (t)) (17)

[0168] This formula is used to calculate the velocity of the particle at the next time, which includes three parts: the inertia term, the cognitive term and the social term. The inertia term ω·v i(t) to ensure the particle continues moving in the current direction; cognitive term c1 · r1 · (p best -x i (t)) to guide the particle to move towards its own historical best position; social term c2 · r2 · (g best -x i (t)) to guide the particle to move towards the global best position. Where v i (t) is the velocity of particle i at time t, x i (t) is the position (i.e. path) of particle i at time t, p best is the historical best position of particle i, g best is the global best position (i.e. the best path among all particles), c1, c2 are learning factors to control the learning degree of the particle to its own historical best position and the global best position, r1, r2 are random numbers to ensure the randomness of the algorithm, and ω is the inertia weight to control the self-adaptation degree of the particle in the search. In this way, the particle finds a balance between the global and local best positions and gradually approaches the optimal solution.

[0169] Then, according to the updated velocity, the position of the particle is also updated, and the update formula is:

[0170] x i (t+1) = x i (t) + v i (t+1) (29)

[0171] This formula is used to update the position of the particle, i.e. to determine the new position of the particle at the next time according to the current position and the newly calculated velocity. This makes each particle constantly move in the solution space and explore new path solutions.

[0172] In order to measure the quality of each path, a fitness function is set, which usually considers factors such as path length, flight time and energy consumption. The form of the fitness function is:

[0173] F(x i ) = a · L(x i ) + β · E(x i ) + γ · T(x i ) (30)

[0174] Where L(x i ) is the path length, E(x i ) is the energy consumption of the path, T(x i ) is the task completion time, and a, β, γ are weight coefficients to balance the factors.

[0175] In the whole optimization process, firstly, the PPO optimization strategy network is used to determine an effective path. On this basis, the particle swarm optimization (PSO) is applied to further iteratively search, and finally the path capable of minimizing the fitness function is found, that is, the global optimal path, so as to realize the overall optimization of the UAV path planning.

[0176] Embodiment 2

[0177] This embodiment relates to a UAV group coordination device based on a fuzzy ant colony algorithm combined with a proximal policy optimization, comprising a memory and one or more processors, the memory has stored executable codes, and the one or more processors execute the executable codes to implement the UAV group coordination method based on the fuzzy ant colony algorithm combined with the proximal policy optimization of embodiment 1.

[0178] Embodiment 3

[0179] This embodiment relates to a computer readable storage medium, which has stored a program, and the program is executed by a processor to implement the UAV group coordination method based on the fuzzy ant colony algorithm combined with the proximal policy optimization of embodiment 1.

[0180] The embodiments of the present specification are only enumerations of implementation forms of the inventive concept, and are only for the purpose of description. The protection scope of the present application should not be regarded as being limited to the specific forms described in the embodiments, and also includes equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A method for UAV swarm cooperation based on fuzzy ant colony algorithm combined with proximal policy optimization, characterized in that, The application relates to an adaptive optimization method combining hierarchical clustering and DBSCAN clustering, which adjusts the task area distribution according to the positions of the unmanned aerial vehicle group and the target; S2, based on the adaptive clustering distribution result, a fuzzy ant colony algorithm is introduced, the pheromone intensity is dynamically adjusted in combination with a fuzzy rule, the unmanned aerial vehicle individual intelligent task target point is distributed, and the path planning is optimized; the pheromone intensity is updated according to the following formula: S3, based on the preliminary path planning of the fuzzy ant colony, a proximal policy optimization algorithm is adopted to dynamically adjust the path selection of the unmanned aerial vehicle individual, and a particle swarm optimization algorithm is adopted to globally search and optimize the path, so that the optimal task path of the unmanned aerial vehicle group is generated; specifically, S3.1 models the path planning problem as a Markov decision process, wherein the state represents the current path information of the unmanned aerial vehicle, including the current position, the visited target point and the remaining task target point; the action corresponds to the next flight node selected by the unmanned aerial vehicle; an explicit reward function is constructed by using the path length, energy consumption and task priority to evaluate the quality of each step path selection, and the time and energy constraints are balanced; the cumulative reward is calculated by sampling the path trajectory, and the strategy is continuously adjusted, so that the unmanned aerial vehicle gradually generates a path planning strategy capable of maximizing the reward function, thereby realizing more efficient path optimization and ensuring task completion; the objective function of the path planning strategy is represented as: S3.2 further optimizes the path based on the path generated in step S3.1 by using the global search capability of the particle swarm optimization; finally, the global optimal path minimizing the fitness function is found, and the global optimization of the unmanned aerial vehicle path planning is realized; the fitness function is in the form of: T ij (t+1) = (1 - p) - T ij (t) + AT ij (21) wherein A k is a set of ants in the ant colony, L k is the path length of ant k, Q is the total weight of the task, and p is a volatility coefficient for controlling the decay rate of pheromone. Step S1 includes: The coordinate matrix of the task target point and the coordinate matrix of the unmanned aerial vehicle starting point are taken as inputs, and a Euclidean distance matrix D is constructed based on the task target point set; hierarchical clustering and DBSCAN clustering are applied to the target points respectively to generate different clustering results; then, the comprehensive quality score is calculated according to the clustering results, including the contour coefficient, the Calinski-Harabasz index and the Davies-Bouldin index, the clustering scheme with the highest score is selected from among them as the basis for the unmanned aerial vehicle task area distribution, and the optimization of the task distribution is realized. wherein r t (θ) represents the ratio of the current policy to the old policy; is the advantage function, which measures the goodness of the current action; ∈ is the clipping range to prevent the policy update from being too large; Step S1 includes the following specific steps: F(x i ) = a · L(x i ) + b · E(x i ) + g · T(x i ) (30) where L(x i ) is the path length, E(x i ) is the energy consumption of the path, T(x i ) is the task completion time, and α, β, γ are weight coefficients for balancing the factors.

2. The UAV swarm coordination method based on the combination of the fuzzy ant colony algorithm and the proximal policy optimization according to claim 1, characterized in that, S1.1 adopts the Ward method to gradually merge the independent clusters of the task target points initialized to minimize the square error increment between the clusters, until the number of unmanned aerial vehicles is reached, the hierarchical clustering is completed, and the Ward method merging formula is: S1.2 utilizes the DBSCAN method to cluster the task target points, wherein the elements of the Euclidean distance matrix D are calculated by the following formula:

3. The UAV swarm coordination method based on fuzzy ant colony algorithm combined with proximal policy optimization of claim 2, characterized in that, S1.3 the quality of the clustering schemes of S1.1 and S1.2 is evaluated by the contour coefficient, the clustering density and the clustering tightness, and the comprehensive score formula is as follows: Step S2 includes: where (x i ,y i ) and (x j ,y j ) are the centers of clusters C i and C j , respectively, and the initial family is the set of each task goal point; ​ where (x p ,y p ) and (x q ,y q ) are the coordinates of target point p and target point q; by setting the neighborhood radius and the minimum sample number, the high-density region is divided into clusters, and the sparse region is marked as a noise point; the noise point is assigned to the nearest base station according to the Euclidean distance from the base station, ensuring that all target points are effectively covered; ​ Wherein a(i) represents the average distance of sample i and other points in the same cluster, b(i) represents the average distance of sample i and other points in the nearest cluster, μ is the mean of the entire data set, is the mean of the jth cluster C j , n is the number of data points, k is the number of clusters, s i represents the average dispersion within cluster i, d ij represents the distance between cluster i and cluster j; finally, the highest comprehensive score of the clustering scheme is selected as the optimal basis for the allocation of the unmanned aerial vehicle task area.

4. The UAV swarm coordination method based on the fuzzy ant colony algorithm combined with the proximal policy optimization of claim 1, characterized in that, ​ In step S1, the task area is divided into multiple smaller task areas and assigned to the drone swarm; each drone departs from the starting base and selects a path according to the pheromone concentration on the path and the distance to the target point; in the path planning, the fuzzy rule is introduced, and the target distance, obstacle density, energy consumption and task urgency are comprehensively considered to dynamically adjust the updating strength of the pheromone and optimize the path selection; the fuzzy reasoning converts the input variables into accurate values of adjusting the pheromone, and the shorter path obtains more pheromone, thereby increasing the probability of being selected; after updating for the maximum number of iterations, the pheromone gradually converges, and the global optimal path is finally found.

5. The method of claim 4, wherein the method is characterized by, Step S2 includes the following specific steps: S2.1 Assign the task area assigned in step S1 to the adjacent drone swarm, and regard each task cluster as an ant colony, wherein the number of samples in the cluster corresponds to the number of drones; S2.2 Each drone departs from its starting base and randomly selects the next target node based on the pheromone concentration on the path and the distance between target points to gradually build a path; the probability calculation formula for path selection is: where T ij denotes the pheromone concentration from node i to node j, d ij denotes the distance between nodes; S2.3 In the path selection, fuzzy rules are introduced, and the target distance, obstacle density, energy consumption and task urgency are comprehensively considered to dynamically adjust the pheromone concentration through fuzzy reasoning and optimize the path selection process; S2.4 After each iteration, the pheromone on the path is updated, and the shorter path accumulates more pheromone, increasing the probability of being selected; within the maximum number of iterations, the pheromone is continuously optimized, and the path gradually converges to the global optimum, completing the efficient allocation and planning of the drone task.

6. The method of claim 5, wherein the method is characterized by, In step S2.3, the specific process of fuzzy reasoning is as follows: S2.3.1 Convert the target distance, obstacle density, energy consumption and task urgency into fuzzy set membership degrees through membership functions, as follows: The target distance is divided into three fuzzy sets "Near", "Medium" and "Far", and the membership functions are defined as follows: The obstacle density p is divided into three fuzzy sets "Low", "Medium" and "High", and the membership functions are defined as follows: μ Low (ρ) = -e -5ρ (11) μ High (ρ) = 1 - e -5ρ (13) The energy consumption E is divided into three fuzzy sets "Low", "Medium" and "High", and the membership functions are defined as follows: The task urgency U is divided into three fuzzy sets "Low", "Medium" and "High", and the membership functions are defined as follows: μ Low (U) = -e -5U (17) μ High (U) = 1 - e -5U (19) S2.3.2 According to the formulated fuzzy rules, fuzzy reasoning is performed in combination with the membership degrees, and the fuzzy rule table is as follows: Where "+", "o" and "-" represent increasing, moderate and decreasing pheromone updating amount respectively; in general, when the target distance is short, the obstacle density is low, the energy consumption is low and the task urgency is high, the pheromone is increased and the current path is preferentially selected; when the target distance is far, the obstacle density is high, the energy consumption is high and the task urgency is low, the pheromone is decreased and exploration of other paths is encouraged; when there is a contradiction between variables, the pheromone updating amount is dynamically adjusted according to the task priority and resource situation to achieve intelligent optimization of path selection; S2.3.

3. De-fuzz the fuzzy inference result by the barycentric method, and convert it into an accurate numerical value, which is used to dynamically adjust the pheromone concentration on the path.

7. A UAV swarm coordination device based on the combination of fuzzy ant colony algorithm and proximal policy optimization, characterized in that, The computer readable storage medium stores a program, and the program is executed by the processor to implement the method for UAV group cooperation based on the fuzzy ant colony algorithm combined with the proximal policy optimization in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program is executed by the processor to implement the method for UAV group cooperation based on the fuzzy ant colony algorithm combined with the proximal policy optimization in any one of claims 1-6.

Images