UAV swarm control and navigation method based on MAPPO

Abstract

The present invention discloses a MAPPO-based UAV cluster control and navigation method, including extracting basic characteristic data of UAVs, analyzing dynamic characteristics and standardizing them, calculating comprehensive capability index of UAVs, and constructing cluster heterogeneity matrix; calculating communication quality index according to real-time communication data, predicting link reliability data, constructing dynamic communication graph, and generating optimal communication topology structure; calculating task allocation fitness matrix based on cluster heterogeneity matrix and optimal communication topology structure, constructing hierarchical strategy model and hierarchical value function; performing multi-dimensional analysis on communication data, generating information importance index, calculating transmission priority data, generating bandwidth allocation matrix, and constructing adaptive routing strategy; calculating strategy uncertainty index, constructing robustness constraint condition, optimizing strategy model and realizing distributed execution. The present invention improves the collaborative control efficiency and system reliability of heterogeneous UAV clusters.

Description

Technical Field

[0001] The present invention belongs to the field of cluster navigation, and in particular to a MAPPO-based unmanned aerial vehicle cluster control and navigation method. Background Art

[0002] Drone swarm systems are playing an increasingly important role in drone performances, forest inspections, emergency rescue, and urban delivery due to their advantages such as efficiency, robustness, and scalability. Especially in complex dynamic environments, the use of multiple heterogeneous drones working together can effectively integrate the advantages of different platforms and significantly improve task execution efficiency and system reliability. With the rapid development of artificial intelligence technology, multi-agent collaborative control methods based on reinforcement learning have provided new ideas and possibilities for solving the autonomous decision-making and collaborative control problems of drone swarms. Among them, the proximal policy optimization (MAPPO) has shown unique advantages in dealing with large-scale heterogeneous swarm control problems due to its good convergence and policy stability.

[0003] At present, there has been a lot of research work in the field of UAV swarm control and navigation. In terms of communication topology optimization, the minimum spanning tree algorithm based on graph theory and the distributed consensus algorithm are mainly used to build the communication network, but there is a lack of in-depth consideration of heterogeneous characteristics; in terms of task allocation, commonly used methods include the Hungarian algorithm, genetic algorithm and ant colony algorithm, etc. These methods mainly focus on the optimality of static task allocation and have limited online adjustment capabilities in dynamic environments; in terms of collaborative control, traditional methods are mostly based on artificial potential field method, behavioral rule method and consistency theory, etc. Although basic formation control and obstacle avoidance functions are achieved, they are insufficient in dealing with complex environments and multi-objective optimization problems. In addition, when the existing reinforcement learning methods are applied to actual UAV swarm systems, they often use simplified communication models and homogeneous agent assumptions, which are difficult to meet the needs of actual applications.

[0004] However, the existing technologies still have the following key technical difficulties in dealing with heterogeneous UAV cluster control problems: first, the lack of effective heterogeneity characterization methods makes it difficult to accurately quantify the performance differences and capability complementarity between different UAV platforms, resulting in inefficient resource allocation and task planning; second, the existing communication topology optimization methods fail to fully consider the dynamic change characteristics of link reliability, and the bandwidth allocation strategy is too static to adapt to the changes in communication needs in complex environments; third, in the traditional hierarchical control architecture, the task allocation layer and the execution control layer are often optimized independently, lacking an effective coordination mechanism, resulting in inconsistencies between local decisions and global goals; finally, in the process of strategy optimization, the existing methods are difficult to balance the relationship between exploration and utilization, and lack an effective robustness guarantee mechanism, which is prone to problems such as strategy collapse and performance oscillation. These technical difficulties seriously restrict the performance and reliability of heterogeneous UAV cluster systems in practical applications. Summary of the invention

[0005] The purpose of the invention is to provide a MAPPO-based UAV cluster control and navigation method to solve the above-mentioned problems existing in the prior art.

[0006] The technical solution is a MAPPO-based UAV cluster control and navigation method, which includes the following steps:

[0007] S1. Obtain the basic characteristic data of the UAV, generate a standardized dynamic parameter matrix after dynamic characteristic analysis and standardization processing; calculate the comprehensive capability index of the UAV based on the standardized dynamic parameter matrix; construct a cluster heterogeneity matrix that characterizes the degree of difference between UAVs based on the comprehensive capability index of the UAV; the basic characteristic data include the maximum speed, maximum acceleration, energy capacity, mission load type and communication range of the UAV;

[0008] S2. Obtain real-time communication data and calculate the communication quality index between each pair of drones; based on the communication quality index, predict the link reliability data; build a dynamic communication graph based on the cluster heterogeneity matrix and link reliability data; optimize the dynamic communication graph and generate the optimal communication topology structure that meets the connectivity constraints and bandwidth constraints;

[0009] S3. Based on the cluster heterogeneity matrix and the optimal communication topology, calculate the task allocation fitness matrix; based on the task allocation fitness matrix, build a hierarchical strategy model including task allocation strategy and execution strategy; based on the hierarchical strategy model, build a hierarchical value function to evaluate the global synergy effect and local execution effect;

[0010] S4. Based on the hierarchical value function, the real-time communication data is analyzed for timeliness, task relevance and communication cost, and the information importance index is generated; based on the information importance index and the communication quality index, the transmission priority data is calculated; based on the transmission priority data, a bandwidth allocation matrix is ​​generated; based on the bandwidth allocation matrix and the optimal communication topology structure, an adaptive routing strategy is constructed;

[0011] S5. Based on the adaptive routing strategy, the entropy value of the strategy output is calculated to generate an uncertainty index; based on the uncertainty index, a robustness constraint condition is constructed; under the robustness constraint condition, the hierarchical strategy model is optimized to obtain an optimized strategy model; based on the optimal communication topology structure, the distributed execution of the optimized strategy model is realized.

[0012] Beneficial effects: The present invention establishes an accurate heterogeneity measurement system, ensures efficient and reliable communication of the cluster system, improves the overall performance of the system, and enhances the environmental adaptability of the system; it not only overcomes the limitations of traditional methods in dealing with heterogeneity, communication optimization, and decision-making coordination, but also improves the task execution capability and system stability of heterogeneous UAV clusters in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of the present invention.

[0014] Figure 2 This is a flow chart of step S1 of the present invention.

[0015] Figure 3 This is a flow chart of step S2 of the present invention.

[0016] Figure 4 This is a flow chart of step S3 of the present invention.

[0017] Figure 5 This is a flow chart of step S4 of the present invention.

[0018] Figure 6 This is a flow chart of step S5 of the present invention. DETAILED DESCRIPTION

[0019] like Figure 1 As shown, this application proposes a MAPPO-based UAV cluster control and navigation method, comprising the following steps:

[0020] S1. Obtain the basic characteristic data of the UAV, generate a standardized dynamic parameter matrix after dynamic characteristic analysis and standardization processing; calculate the comprehensive capability index of the UAV based on the standardized dynamic parameter matrix; construct a cluster heterogeneity matrix that characterizes the degree of difference between UAVs based on the comprehensive capability index of the UAV; the basic characteristic data include the maximum speed, maximum acceleration, energy capacity, mission load type and communication range of the UAV;

[0021] S2. Obtain real-time communication data and calculate the communication quality index between each pair of drones; based on the communication quality index, predict the link reliability data; build a dynamic communication graph based on the cluster heterogeneity matrix and link reliability data; optimize the dynamic communication graph and generate the optimal communication topology structure that meets the connectivity constraints and bandwidth constraints;

[0022] S3. Based on the cluster heterogeneity matrix and the optimal communication topology, calculate the task allocation fitness matrix; based on the task allocation fitness matrix, build a hierarchical strategy model including task allocation strategy and execution strategy; based on the hierarchical strategy model, build a hierarchical value function to evaluate the global synergy effect and local execution effect;

[0023] S4. Based on the hierarchical value function, the real-time communication data is analyzed for timeliness, task relevance and communication cost, and the information importance index is generated; based on the information importance index and the communication quality index, the transmission priority data is calculated; based on the transmission priority data, a bandwidth allocation matrix is ​​generated; based on the bandwidth allocation matrix and the optimal communication topology structure, an adaptive routing strategy is constructed;

[0024] S5. Based on the adaptive routing strategy, the entropy value of the strategy output is calculated to generate an uncertainty index; based on the uncertainty index, a robustness constraint condition is constructed; under the robustness constraint condition, the hierarchical strategy model is optimized to obtain an optimized strategy model; based on the optimal communication topology structure, the distributed execution of the optimized strategy model is realized.

[0025] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0026] S11, obtaining basic characteristic data of the UAV, reorganizing the basic characteristic data according to a preset characteristic vector format, and generating an initial characteristic vector including a maximum velocity component, a maximum acceleration component, an energy capacity component, a mission load type component, and a communication range component;

[0027] S12, using a preset linear mapping weight matrix to perform a linear transformation on the initial feature vector to obtain a linear transformation vector; adding the linear transformation vector to a preset bias vector to generate a standardized dynamic parameter matrix;

[0028] S13, based on the preset capability evaluation weight vector, weighted summing up the parameter components in the standardized dynamic parameter matrix, and calculating the comprehensive capability index of the UAV that characterizes the comprehensive performance of the UAV;

[0029] S14. Calculate the binary norm of the difference between the comprehensive capability indicators of any two drones, and divide the binary norm by the maximum value of the comprehensive capability indicators of the two drones to obtain a heterogeneity value that characterizes the heterogeneity of the drones; organize the heterogeneity values ​​between all pairs of drones into a matrix form to generate a cluster heterogeneity matrix.

[0030] In one embodiment of the present application, the basic characteristic data of the drone are read, including the maximum speed vi, the maximum acceleration ai, the energy capacity ei, the mission load type li, the communication range ri, etc., to construct the initial characteristic vector Fi = [vi, ai, ei, li, ri]. The initial characteristic vector is converted into the standardized dynamic parameter matrix Di by using an improved linearization mapping method. The mapping function is: Di = M(Fi) = WF·Fi + bF; where WF is the weight matrix and bF is the bias vector, which is updated in real time by an adaptive optimization algorithm. Based on the standardized dynamic parameter matrix, the comprehensive capability index Ci of the drone is calculated: Ci =∑(wj·dij); where wj is the weight coefficient of each capability and dij is the jth parameter in Di. A cluster heterogeneity matrix H is constructed, whose element hij represents the degree of heterogeneity between the i-th and j-th drones: hij=||Ci-Cj||2 / max(Ci, Cj).

[0031] This embodiment achieves accurate quantification and dynamic characterization of the internal differences of the cluster by constructing a multi-level UAV heterogeneity evaluation system. First, by extracting and standardizing the key parameters of the UAV from the basic feature database, a standardized measurement benchmark that can be quantified and compared is established; then, the static parameters are converted into dynamic performance indicators using dynamic characteristic analysis, which more accurately reflects the performance differences of the UAV in actual tasks; finally, by constructing a cluster heterogeneity matrix, it can not only characterize the degree of difference between any two UAVs, but also reflect the heterogeneous distribution characteristics of the entire cluster. This embodiment enables the system to adaptively perform task allocation and collaborative control according to the heterogeneous characteristics of the UAV, thereby improving the adaptability of the cluster system to heterogeneity and the resource utilization efficiency. In particular, by introducing the two-norm calculation and relative difference measurement, the scale invariance and comparability of the heterogeneity evaluation are ensured, providing a reliable decision-making basis for subsequent task planning and control optimization.

[0032] According to one aspect of the present application, step S12 is further:

[0033] S121, using a sliding window method to perform time-series sampling on each component in the initial feature vector to generate a multi-scale feature tensor containing features of different time scales; calculating the mean and standard deviation of the feature components at each time scale in the multi-scale feature tensor to generate a feature statistical matrix;

[0034] S122, based on the mean and standard deviation in the feature statistical matrix, normalize the multi-scale feature tensor to obtain a normalized feature tensor; use a principal component analysis method to reduce the dimension of the normalized feature tensor to generate a reduced-dimensional feature vector;

[0035] S123, reading a preset mapping weight matrix from a dynamics parameter database; based on the normalized feature tensor, using an adaptive moment matching algorithm to calculate high-order statistics of feature distribution to generate a distribution feature vector; based on the distribution feature vector, updating parameter values ​​in the mapping weight matrix to obtain an updated weight matrix;

[0036] S124, performing a matrix multiplication operation on the dimension-reduced feature vector and the updated weight matrix to generate a linear transformation vector; reading a preset bias vector from a dynamic parameter database, adding the bias vector to the linear transformation vector to obtain a feature mapping matrix;

[0037] S125, calculating the covariance matrix of the feature mapping matrix in different feature dimensions to generate a feature correlation matrix; based on the feature correlation matrix, using an adaptive feature decoupling algorithm to eliminate redundant correlations between features to generate a decoupled feature matrix;

[0038] S126. Dynamically calibrate the decoupling characteristic matrix using a recursive least squares algorithm to generate a calibration characteristic matrix; perform a nonlinear transformation on the calibration characteristic matrix using a hyperbolic tangent function to obtain a standardized kinetic parameter matrix.

[0039] In one embodiment of the present application, the adaptive moments matching algorithm: M(W) = γ1·||μ_s - μ_t|| 2 + γ2·||Σ_s - Σ_t|| 2 + γ3·||S_s - S_t|| 3 + γ4·||K_s - K_t|| 4 ; Where M(W) is the feature distribution matching loss function; μ_s and μ_t are the mean vectors of the source domain and the target domain respectively; Σ_s and Σ_t are the covariance matrices of the source domain and the target domain respectively; S_s and S_t are the skewness matrices of the source domain and the target domain respectively; K_s and K_t are the kurtosis matrices of the source domain and the target domain respectively; γ1, γ2, γ3, and γ4 are adaptive weight coefficients optimized by gradient descent; ||·|| represents the Frobenius norm; the weight update equation is W = W - η·ΞM(W) / ΞW, where η is the learning rate and Ξ is the partial derivative. The adaptive weight coefficient is used to dynamically adjust the matching importance of each order moment; the Frobenius norm is used to measure the matrix difference, which improves the computational efficiency.

[0040] Adaptive feature decoupling algorithm: D(X) = ||X T ·X-diag(X T ·X)||F+α·Tr(X T·L·X)+β·||X||{2,1}; where D(X) is the feature decoupling objective function; X is the decoupled feature matrix; diag(·) extracts the matrix diagonal elements; L is the Laplacian matrix of the feature correlation matrix L = D - W, W is the correlation coefficient matrix, and D is the degree matrix; ||·||F is the Frobenius norm; ||·||{2,1} is the L2,1 norm, which is used for feature sparsification; α and β are trade-off coefficients; feature update equation: X(t+1) = X(t)-η·(I+γ·S(t)) -1 ·▽D(X(t)), where S(t) is the adaptive regularization matrix and γ is the regularization coefficient. Adaptive regularization and sparse constraints are introduced to achieve dynamic decoupling between features.

[0041] This embodiment achieves accurate characterization and effective mapping of heterogeneous UAV dynamic characteristics by constructing a multi-level feature processing and dynamic parameter standardization process. First, the sliding window method is used for multi-scale time series sampling to capture the dynamic characteristics at different time scales; then, through feature statistics and normalization processing, the influence of dimensional differences is eliminated; then, the adaptive moments matching algorithm is used to optimize the mapping weights to improve the accuracy of feature conversion; finally, through feature decoupling and dynamic calibration, the independence and reliability of parameter characterization are ensured. In particular, by introducing the recursive least squares algorithm and the nonlinear transformation of the hyperbolic tangent function, the system can adaptively adjust the parameter mapping relationship and effectively deal with the time-varying and nonlinear nature of the dynamic characteristics. This embodiment not only improves the accuracy and robustness of feature characterization, but also achieves more accurate heterogeneous characterization through dynamic optimization, providing a reliable data basis for subsequent decision optimization.

[0042] According to one aspect of the present application, step S14 is further:

[0043] S141, arranging the comprehensive capability indicators of the UAVs in the order of the UAV numbers to generate a capability indicator sequence; smoothing the capability indicator sequence using a sliding mean method to obtain a smoothed capability indicator;

[0044] S142, calculating the probability density distribution of the smoothed capability index to generate a capability distribution vector; based on the capability distribution vector, using a kernel density estimation method to calculate the capability difference between any two UAVs to obtain an initial difference matrix;

[0045] S143, based on the initial difference matrix, calculating the second norm of the difference in smoothed capability indicators of any two UAVs to generate a capability difference matrix; reading normalization parameters from a preset parameter database, and normalizing the capability difference matrix based on the normalization parameters to obtain a normalized difference matrix;

[0046] S144, performing weighted fusion on the initial difference matrix and the normalized difference matrix to generate a fused difference matrix; performing adaptive threshold segmentation on each element in the fused difference matrix to obtain a segmented difference matrix;

[0047] S145, based on the segmentation difference matrix, at each matrix element position, the corresponding difference value is divided by the maximum value of the smoothing capability index of the corresponding two UAVs to generate a heterogeneity numerical matrix; using the exponential weighted average method to perform time series smoothing on the heterogeneity numerical matrix to obtain a smoothed heterogeneity matrix;

[0048] S146. Adopt an adaptive symmetry processing algorithm to optimize the symmetry of the smooth heterogeneity matrix, ensure the symmetry of the matrix, and generate a symmetric heterogeneity matrix; use the symmetric heterogeneity matrix as the final cluster heterogeneity matrix.

[0049] In one embodiment of the present application, the kernel density estimation and adaptive threshold segmentation algorithm: H(x) = 1 / n·Σ(1 / h·K((x-xi) / h)) + λ·R(T(x)); where H(x) is the kernel density estimation function; n is the number of samples; h is the bandwidth parameter; K(·) is the Gaussian kernel function K(u) = (1 / sqrt(2π))·exp(-u 2 / 2); xi is the ith sample point; λ is the regularization coefficient; R(T(x)) is the adaptive threshold function T(x) = μ(x) + α·σ(x), μ(x) is the local mean, σ(x) is the local standard deviation, and α is the adaptive coefficient; the threshold update equation is T(t+1) = T(t) - β·ΞJ / ΞT, J is the segmentation error function, and β is the update step size.

[0050] This embodiment realizes the accurate quantification and dynamic characterization of the heterogeneous characteristics of the UAV cluster by constructing a heterogeneity calculation framework based on probability statistics and adaptive fusion. First, the capability index is smoothed by the sliding mean method to reduce the impact of random fluctuations; then the capability difference distribution is calculated by the kernel density estimation method to accurately characterize the statistical characteristics of the heterogeneity degree; then the stability and reliability of the heterogeneity quantification are improved by adaptive threshold segmentation and time series smoothing; finally, the symmetric processing is used to ensure the consistency of the heterogeneity matrix. In particular, by introducing the capability distribution vector and multi-level fusion mechanism, the system can evaluate the degree of difference between UAVs from multiple dimensions, avoiding the one-sidedness that may be caused by a single indicator. This embodiment not only improves the accuracy and reliability of heterogeneity characterization, but also realizes a more stable heterogeneity evaluation through dynamic optimization, providing a reliable decision-making basis for the collaborative control of the cluster system.

[0051] like Figure 3 As shown, according to one aspect of the present application, step S2 is further:

[0052] S21, collecting real-time communication data including signal strength, packet loss rate and delay time from the communication module; multiplying the signal strength of the real-time communication data by a first preset weight, multiplying the packet loss rate by the second preset weight after taking the inverse, and multiplying the delay time by the third preset weight after taking the inverse, and adding them to obtain a communication quality index characterizing the communication performance;

[0053] S22, reading the link reliability data of the previous moment from the historical database; multiplying the communication quality index of the current moment by a preset dynamic smoothing coefficient, and adding the complement of the link reliability data of the previous moment multiplied by the preset smoothing coefficient to obtain the updated link reliability data;

[0054] S23, calculating the weighted combination of the updated link reliability data and the corresponding elements of the cluster heterogeneity matrix, generating edge weight data representing the weight of the communication link; and constructing a dynamic communication graph based on the edge weight data;

[0055] S24. Based on the dynamic communication graph, and under the conditions of satisfying preset connectivity constraints and bandwidth constraints, a spanning tree is calculated that maximizes the sum of edge weight data to obtain an optimal communication topology.

[0056] In one embodiment of the present application, real-time communication data is read, including signal strength si, packet loss rate pi, and delay ti, and the communication quality index Qij is calculated: Qij = α·si + β·(1-pi) + γ·(1 / ti); wherein α, β, and γ are weight coefficients. Based on the historical communication quality indicators, the link reliability Rij is predicted using the improved exponential smoothing method: Rij(t+1) = λ·Qij(t) + (1-λ)·Rij(t); wherein λ is the smoothing coefficient, which is adjusted dynamically. A dynamic communication graph G(t) is constructed, and the edge weight wij is calculated based on the link reliability and heterogeneity matrix: wij = Rij·(1 - η·hij); wherein η is the heterogeneity influence coefficient. Solve the improved maximum spanning tree problem to obtain the optimal communication topology structure T*: T* = argmax∑wij, (i, j)∈Tsubject to: connectivity and bandwidth constraints.

[0057] This embodiment improves the communication efficiency and reliability of heterogeneous drone clusters by establishing a dynamic and adaptive communication topology optimization mechanism. First, a multi-dimensional communication quality evaluation system is constructed based on real-time communication data, including key indicators such as signal strength, packet loss rate and delay time; then, a link reliability prediction model is used to achieve a forward-looking evaluation of the communication link status; finally, based on the cluster heterogeneity matrix and link reliability data, the optimal communication topology that meets connectivity and bandwidth constraints is optimized and constructed. In particular, by introducing a dynamic weight adjustment mechanism, the system can adaptively adjust the topology structure according to real-time changes in communication quality, effectively avoiding communication bottlenecks and network congestion. This embodiment not only improves the robustness and reliability of the network, but also achieves more efficient resource allocation by considering heterogeneous characteristics, so that the entire cluster system can still maintain stable communication performance in a complex dynamic environment.

[0058] According to one aspect of the present application, step S22 is further:

[0059] S221, reading the link reliability data of the previous N moments from the historical database; using the wavelet decomposition algorithm to perform multi-scale analysis on the link reliability data, extracting the trend component and the fluctuation component, and generating a multi-scale feature matrix; based on the multi-scale feature matrix, using the adaptive threshold detection algorithm to identify abnormal values, and obtain cleaned historical data; wherein N is a natural number greater than 1;

[0060] S222, using a Kalman filter to suppress noise on the communication quality indicator to generate a filtering quality indicator; constructing a short-term predictor based on an ARIMA model, performing a prediction analysis on the filtering quality indicator, and obtaining a predicted quality indicator;

[0061] S223. Based on the prediction quality index, the statistical characteristics of the cleaned historical data are calculated, including the mean, variance and autocorrelation coefficient, to generate a statistical characteristic vector; based on the statistical characteristic vector, the dynamic smoothing coefficient is estimated using the Bayesian reasoning method to obtain a smoothing coefficient matrix;

[0062] S224, extracting the main features of the smoothing coefficient matrix by using the principal component analysis method to generate a characteristic coefficient vector; optimizing and adjusting the characteristic coefficient vector by using a fuzzy logic controller to obtain an optimized smoothing coefficient;

[0063] S225. Based on the prediction quality index and the cleaned historical data, an integrated learning model is constructed to generate an integrated prediction result; a confidence interval of the integrated prediction result is calculated to obtain a credibility vector;

[0064] S226. Based on the credibility vector, dynamically adjust the optimized smoothing coefficient to generate a final smoothing coefficient; perform weighted combination of the integrated prediction result and the communication quality index at the current moment through the final smoothing coefficient to obtain updated link reliability data.

[0065] In one embodiment of the present application, wavelet decomposition and ensemble learning prediction algorithm: P(t) = w1·Φ1(D1(t)) + w2·Φ2(D2(t)) + w3·Φ3(D3(t)) + w4·Φ4(A(t)); wherein P(t) is the final prediction value; D1(t), D2(t), and D3(t) are detail coefficients of different scales obtained by wavelet decomposition; A(t) is an approximation coefficient; Φk(·) is the kth basic predictor, including three neural network structures: LSTM, GRU, and TCN; wk is a dynamic integration weight, calculated by the attention mechanism wk = softmax(v T ·tanh(W·hk + b)), hk is the hidden state vector of each predictor; the variance of the prediction error is estimated as σ 2 (t) = Σwk·(Φk(t) - P(t)) 2 , which is used to calculate the confidence of the prediction.

[0066] The specific process of identifying outliers is: A(x) = Σ|〈x, ψj, k〉| + λ·||d||1 + μ·TV(x); where A(x) is the anomaly detection function; 〈x, ψj, k〉 is the wavelet coefficient of signal x at scale j and position k; ψj, k are wavelet basis functions; d is the detail coefficient vector; TV(x) is the total variation regularization term TV(x) = Σ|x(i+1) - xi|; λ and μ are trade-off coefficients; outlier judgment threshold: T(t) = μ(t) + κ·σ(t)·(1 + ω·exp(-ρ·v(t))), where μ(t) and σ(t) are the local mean and standard deviation respectively, v(t) is the local coefficient of variation, κ, ω, ρ are adaptive parameters; online update is performed based on the cumulative sum (CUSUM) algorithm.

[0067] This embodiment achieves accurate evaluation and reliable prediction of the communication link status by establishing a link reliability prediction mechanism based on multi-scale analysis and integrated learning. First, a wavelet decomposition algorithm is used to extract multi-scale features from historical data, effectively separating the trend component and the fluctuation component; then, the Kalman filter is used to suppress noise on the communication quality index, thereby improving the reliability of the measurement data; then, the Bayesian reasoning method is used to adaptively estimate the dynamic smoothing coefficient, thereby achieving the optimal fusion of historical data and real-time data; finally, the outputs of multiple predictors are integrated through an integrated learning model, thereby improving the accuracy and stability of the prediction results. In particular, by introducing an adaptive fuzzy logic controller to optimize and adjust the smoothing coefficient, the system can adaptively adjust the prediction strategy according to the dynamic changes of the communication environment, thereby effectively improving the accuracy and robustness of the prediction. This embodiment not only overcomes the limitations of traditional prediction methods in processing non-stationary time series data, but also achieves more reliable link status prediction through multi-level adaptive optimization.

[0068] According to one aspect of the present application, step S23 is further:

[0069] S231, obtaining link reliability data; performing time series filtering on the link reliability data using an exponential smoothing method to obtain smoothed reliability data; based on the smoothed reliability data, calculating the rate of change within adjacent time windows to generate a reliability trend matrix.

[0070] S232, read the cluster heterogeneity matrix; use the spectral clustering method to analyze the cluster heterogeneity matrix, identify drone subgroups with similar heterogeneous characteristics, and generate a subgroup partition matrix; according to the subgroup partition matrix, calculate the heterogeneity weight coefficients between different subgroups to obtain a heterogeneous weight vector.

[0071] S233, obtaining smoothed reliability data and heterogeneous weight vectors; multiplying each element of the smoothed reliability data with the corresponding heterogeneous weight vector element to generate an initial weight matrix; applying an adaptive normalization algorithm to the initial weight matrix to obtain a normalized weight matrix.

[0072] S234, receiving a reliability trend matrix and a normalized weight matrix; dynamically adjusting the normalized weight matrix based on the change trend in the reliability trend matrix to generate an adjusted weight matrix; using the adjusted weight matrix as edge weight data characterizing the communication link weight.

[0073] S235, obtaining edge weight data; constructing an initial undirected graph structure, mapping the edge weight data to edge weights of the graph, and obtaining an initial communication graph; using a topology optimization algorithm to prune redundant edges from the initial communication graph to generate a simplified communication graph.

[0074] S236, receiving a simplified communication graph; calculating the degree centrality and betweenness centrality of each node in the simplified communication graph to generate a node importance vector; performing a secondary adjustment on the edge weights of the simplified communication graph according to the node importance vector, and finally generating a dynamic communication graph.

[0075] In one embodiment of the present application, spectral clustering and topology optimization algorithm: Q(G) = Tr(X T ·L·X) + μ·||W||1 + η·Σ(di - d*) 2 ; where Q(G) is the network topology optimization objective function; L is the standardized Laplacian matrix L = I - D -1 / 2· W.D. -1 / 2 ; X is the spectral embedding matrix; W is the adjacency weight matrix, I is the identity matrix; ||W||1 is the L1 regularization term, which is used to control the network sparsity; di is the node degree; d is the target degree; μ and η are the trade-off coefficients; the optimization is solved by the alternating direction multiplier method (ADMM): X = argmin{Tr(X T ·L·X)},st X T X = I.

[0076] The specific process of redundant edge pruning is: R(G) = -Σ(BCE(i, j)·log(p(i, j))) + α·Φ(G) + β·Ψ(G); where R(G) is the redundant edge evaluation function; BCE(i, j) is the betweenness centrality of edge (i, j); p(i, j) is the edge retention probability; Φ(G) is the network connectivity constraint Φ(G) = -log(λ2(L)), λ2(L) is the second smallest eigenvalue of the Laplace matrix; Ψ(G) is the network efficiency constraint Ψ(G) = Σ1 / d(i, j), d(i, j) is the shortest path between nodes; α and β are balance coefficients; edge pruning strategy: e* = argmax{R(G - e): e∈E}, where E is the edge set; adaptive edge pruning is achieved through iterative optimization.

[0077] This embodiment realizes the efficient communication structure optimization of heterogeneous drone clusters by constructing a dynamic communication graph based on timing analysis and topology optimization. First, the link reliability data is subjected to timing filtering and trend analysis to accurately grasp the dynamic change characteristics of communication quality; then, the drone subgroups with similar heterogeneous characteristics are identified through the spectral clustering method to realize the hierarchical organization of the communication structure; then, the communication links are dynamically weighted based on the heterogeneous weight vector to ensure the rationality of resource allocation; finally, the overall communication structure is optimized through node importance analysis and edge weight adjustment. In particular, by introducing the adaptive normalization algorithm and the topology optimization algorithm, the system can dynamically adjust the communication structure while ensuring network connectivity, effectively balancing communication efficiency and resource consumption. This embodiment not only improves the reliability and efficiency of the network, but also realizes a more reasonable resource allocation by considering heterogeneous characteristics, ensuring the stable communication of the cluster system in a complex environment.

[0078] According to one aspect of the present application, step S24 is further:

[0079] S241, receiving a dynamic communication graph; extracting node degree distribution features in the dynamic communication graph to generate a degree distribution vector; based on the degree distribution vector, using a community detection algorithm to identify densely connected subgroups in the network to obtain subgroup partitioning data.

[0080] S242, obtaining subgroup division data; calculating the connectivity within each subgroup and the connection strength between subgroups to generate a connectivity feature matrix; constructing a multi-level constraint condition that takes into account subgroup characteristics based on the connectivity feature matrix to obtain a hierarchical constraint set.

[0081] S243. Read preset bandwidth threshold data from the system parameter library; based on the bandwidth threshold data, perform bandwidth capacity analysis on each edge in the dynamic communication graph to generate a capacity constraint vector; combine the capacity constraint vector with the hierarchical constraint set to obtain a comprehensive constraint condition.

[0082] S244, obtaining comprehensive constraint conditions and a dynamic communication graph; initializing a minimum spanning tree using an improved Kruskal algorithm to generate an initial topology structure; and calculating bandwidth utilization of a critical path based on the initial topology structure to obtain a utilization index.

[0083] S245, receiving an initial topology structure and utilization index; constructing a topology optimization model based on a genetic algorithm, using the sum of edge weights as a fitness function, generating multiple candidate topologies, and obtaining a candidate solution set; performing feasibility verification on each solution in the candidate solution set to generate a feasible solution set.

[0084] S246. Obtain a set of feasible solutions; adopt a multi-objective evaluation method, comprehensively consider the sum of edge weights, network diameter and load balancing, sort the set of feasible solutions, and generate an evaluation result matrix; select the solution with the best comprehensive performance from the evaluation result matrix to obtain the optimal communication topology.

[0085] In one embodiment of the present application, the Kruskal algorithm and multi-objective evaluation method are improved: E(T) = α·Σw(e) - β·D(T) - γ·B(T) + λ·C(T); wherein E(T) is a topology evaluation function; w(e) is the weight of edge e; D(T) is a network diameter function D(T) = max{d(i, j)}, d(i, j) is the shortest path length between nodes i and j; B(T) is a load balancing function B(T) = Σ(Li - L*) 2 / n, Li is the load of node i, L* is the average load; C(T) is the connectivity constraint function C(T) = min{λmin(L(S))}, L(S) is the minimum eigenvalue of the Laplacian matrix of subgraph S; α, β, γ, λ are trade-off coefficients; the improved Kruskal algorithm introduces a heuristic edge selection strategy: p(e) = w(e)·exp(-θ·ΔD(e)), ΔD(e) is the network diameter increment caused by adding edge e, and θ is the temperature parameter.

[0086] The algorithm for constructing multi-level constraints is: M(C) = Σ[wi·Hi(Ci)] + λ·D(C) + μ·V(C); where M(C) is the multi-level constraint evaluation function; Hi(Ci) is the entropy function of the i-th layer constraint Hi(Ci) = -Σp(ci)·log(p(ci)); wi is the layer weight; D(C) is the inter-layer dependency measure D(C) = Σ||Ri,j - Ri,j*||, Ri,j is the actual inter-layer relationship matrix, Ri,j* is the expected inter-layer relationship matrix; V(C) is the constraint validity function V(C) = Σexp(-γ·vi), vi is the degree of violation of the i-th constraint; λ, μ, γ are adjustment parameters; constraint update equation: C(t+1) = C(t) - η·▽M(C(t))·(1 + ξ·S(t)), where S(t) is the constraint sensitivity matrix and ξ is the adaptive coefficient.

[0087] This embodiment realizes the optimal communication structure design of heterogeneous drone clusters by constructing a multi-level constrained communication topology optimization framework. First, community detection is performed based on degree distribution characteristics to identify densely connected subgroups in the network, providing a structured basis for topology optimization; then, through the construction of multi-level constraints, the constraints of multiple dimensions such as connectivity, bandwidth capacity and subgroup characteristics are comprehensively considered; then, the minimum spanning tree is initialized using the improved Kruskal algorithm, and global optimization is performed through the genetic algorithm to maximize the sum of edge weights while satisfying various constraints; finally, a multi-objective evaluation method is used to evaluate the performance of the topology scheme from multiple dimensions such as the sum of edge weights, network diameter and load balancing. In particular, by introducing hierarchical constraints and adaptive optimization strategies based on subgroup characteristics, the system can maintain local communication efficiency while ensuring global optimality, effectively solving the communication optimization problem in large-scale heterogeneous clusters. This embodiment not only improves the optimization effect of the communication structure, but also realizes a more balanced and efficient network structure design by considering multi-dimensional evaluation indicators.

[0088] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0089] S31, combining the cluster heterogeneity matrix with the optimal communication topology structure, calculating the fitness of each UAV to perform various tasks, and generating a task allocation fitness matrix;

[0090] S32, obtaining the current state, constructing a task allocation strategy model based on the current state, the task allocation fitness matrix and the cluster heterogeneity matrix, and generating a top-level strategy function; constructing an execution strategy model based on the current state and the standardized dynamic parameter matrix, and generating a bottom-level strategy function; combining the top-level strategy function and the bottom-level strategy function to form a hierarchical strategy model;

[0091] S33. Based on the hierarchical strategy model, calculate the expected cumulative return based on task allocation rewards to generate a top-level value function for evaluating the global synergy effect; based on the hierarchical strategy model, calculate the expected cumulative return based on execution rewards to generate a bottom-level value function for evaluating the local execution effect; combine the top-level value function and the bottom-level value function to form a hierarchical value function.

[0092] In one embodiment of the present application, based on the cluster heterogeneity matrix and the optimal communication topology, a task allocation matrix A is constructed: Aij represents the fitness of the i-th task assigned to the j-th drone. A hierarchical multi-agent proximal policy optimization (MAPPO) architecture is constructed: the top-level strategy πh is used for task allocation: πh(s) = P(a|s, H, T*); the bottom-level strategy πl is used for specific execution: πl(s) = P(a|s, Di); a two-layer value function is constructed: Vh(s) = E[Rh +γ·Vh(s')], which evaluates the global synergy effect; Vl(s) = E[Rl + γ·Vl(s')], which evaluates the local execution effect, where P(a|s, H, T*) represents the probability of taking action a in state s based on the history H and the optimal communication topology T*; P(a|s, Di) represents the probability of taking action a in state s based on the current task Di; E[ ] is the expected value operator; Rh represents the reward for global synergy effect; γ is the discount factor; Vh(s') is the top-level value function at the next state s'; Vl(s') is the bottom-level value function at the next state s'; Rl is the reward for local execution effect.

[0093] This embodiment realizes the collaborative optimization of task allocation and execution control by constructing a hierarchical strategy model based on heterogeneous perception. First, a task allocation fitness matrix is ​​constructed according to the cluster heterogeneity matrix and the optimal communication topology, and the ability matching of different drones to perform various tasks is accurately quantified; then, the task allocation and execution control are decoupled through the hierarchical strategy model, which reduces the complexity of the problem while maintaining decision-making flexibility; finally, the unified evaluation and optimization of global synergy effect and local execution effect are realized through the hierarchical value function. In particular, by introducing a hierarchical architecture, the system can consider the impact of heterogeneous characteristics at the task level, adapt to individual dynamic constraints at the execution level, and improve the accuracy of decision-making and the efficiency of execution. This embodiment not only improves the scalability and robustness of the system, but also realizes more accurate task allocation and more efficient execution control by considering heterogeneous characteristics.

[0094] According to one aspect of the present application, step S32 is further:

[0095] S321, based on the task allocation fitness matrix, using the hierarchical analysis method, calculate the priority relationship between tasks and generate a task priority vector; based on the task priority vector, perform weighted processing on the task allocation fitness matrix to obtain a weighted fitness matrix;

[0096] S322, read the current environment state data, perform feature fusion on the environment state data and the cluster heterogeneity matrix, and generate a state feature vector; use an adaptive feature selection algorithm to reduce the dimension of the state feature vector to obtain a reduced dimension feature vector;

[0097] S323, construct a multi-head attention network, use the weighted fitness matrix as the query matrix, use the reduced-dimensional feature vector as the key value matrix, and generate an attention score matrix; based on the attention score matrix, construct a task allocation strategy network to obtain a top-level strategy function;

[0098] S324, extracting time series features from the standardized kinetic parameter matrix using a recursive neural network to generate a kinetic feature sequence; applying an adaptive gating mechanism to the kinetic feature sequence to obtain a gated feature sequence;

[0099] S325, using a deep deterministic policy gradient algorithm, inputting the gated feature sequence and the dimension-reduced feature vector into the action generation network to generate an action policy function; and using the action policy function as the underlying policy function;

[0100] S326. Construct a hierarchical connection structure, fuse the output of the top-level strategy function with the intermediate layer features of the bottom-level strategy function through the attention mechanism, and generate a fused strategy representation; based on the fused strategy representation, construct a complete strategy network system to obtain a hierarchical strategy model.

[0101] In one embodiment of the present application, a multi-head attention network and a deep deterministic policy gradient algorithm: A(Q, K, V) = Σh{softmax(Q·Wh Q ·(K·Wh K ) T / sqrt(dk))·(V·Wh V )}; π(s) = μ(s) + σ(s)·ε, ε~N(0,I); where A(Q,K,V) is the multi-head attention output; Q, K, V are query, key value, and value matrices respectively; Wh Q Wh K Wh V is the parameter matrix of the h-th attention head; dk is the key value dimension; π(s) is the random policy function; μ(s) is the deterministic action mean; σ(s) is the action standard deviation; policy gradient update: ▽θJ = E[▽θlogπθ(a|s)·(Q(s,a) - V(s))] +λ·▽θH(πθ), where H(πθ) is the policy entropy and λ is the entropy regularization coefficient.

[0102] Adaptive feature selection algorithm: F(X) = ||Y - WX|| 2 + α·||W||1 + β·Tr(XT ·L·X) +θ·Ω(W); where F(X) is the feature selection objective function; Y is the target output; W is the feature weight matrix; L is the Laplacian matrix of the feature similarity matrix; Ω(W) is the structured sparse regularization term Ω(W) = Σ||wi||2, wi is the weight vector of the i-th feature; α, β, θ are trade-off coefficients; feature importance calculation: s(i) = ||wi||2 / max{||wj||2}; feature selection criterion: X* = {xi: s(i) > τ(t)}, where τ(t) is the adaptive threshold τ(t) = μ·exp(-λ·σ(s)), σ(s) is the standard deviation of feature importance.

[0103] Adaptive gating mechanism: G(h, x) = σ(Wg·[h, x] + bg) Θtanh(Wh·h + Wx·x + b); where G(h, x) is the gated output; h is the hidden state vector; x is the input vector; σ(·) is the Sigmoid function; Θ is the Hadamard product; Wg, Wh, Wx are weight matrices; bg, b are bias vectors; Gating update: z(t) = σ(Wz·[h(t-1),x(t)]), r(t) = σ(Wr·[h(t-1),x(t)]), h(t) = (1-z(t)) Θh(t-1) + z(t) ΘG(r(t) Θh(t-1), x(t)); Weight adaptive adjustment: W(t+1) = W(t) - η·▽L(t)·(1 + ρ·||▽L(t-1)||), where L is the loss function and ρ is the momentum coefficient.

[0104] This embodiment realizes intelligent task allocation and execution control of heterogeneous drone clusters by constructing a hierarchical strategy modeling mechanism based on deep learning. First, the task priority system is constructed using the hierarchical analysis method to scientifically quantify the importance of different tasks; then, the intelligent modeling of the task allocation strategy is realized through the multi-head attention network, which improves the accuracy of the allocation decision; then, the recursive neural network and adaptive gating mechanism are used to extract dynamic features to achieve accurate modeling of the execution strategy; finally, the effective fusion of strategies is achieved through the hierarchical connection structure and attention mechanism. In particular, by introducing the strategy fusion method of the deep deterministic policy gradient algorithm and the attention mechanism, the system can realize the coordinated optimization of task allocation and execution control while considering heterogeneous features, effectively improving the accuracy of decision-making and the efficiency of execution. This embodiment not only overcomes the limitations of traditional methods in dealing with high-dimensional state space and complex decision logic, but also realizes more intelligent and efficient control decisions through multi-level strategy optimization.

[0105] like Figure 5As shown, according to one aspect of the present application, step S4 is further:

[0106] S41. Based on the hierarchical value function, construct an importance calculation function; calculate the timeliness value of the real-time communication data, the relevance value to the current task, and the communication cost value required for transmission; substitute the timeliness value, the relevance value, and the communication cost value into the importance calculation function to generate an information importance index;

[0107] S42, constructing a priority calculation function based on the hierarchical value function; substituting the information importance index and the communication quality index into the priority calculation function to generate transmission priority data representing the transmission priority order;

[0108] S43, based on the transmission priority data, calculating the communication bandwidth allocation ratio between the drones and generating a bandwidth allocation matrix;

[0109] S44. Based on the bandwidth allocation matrix and the optimal communication topology, construct an adaptive routing strategy for determining the data transmission path; and store the adaptive routing strategy in a routing strategy database.

[0110] In one embodiment of the present application, the importance index I of each type of information is calculated: I = f (information timeliness, task relevance, communication cost); based on the importance index and communication quality, the transmission priority P is calculated: P = g(I, Q) to ensure that key information is transmitted first; according to the transmission priority, the bandwidth allocation matrix B is dynamically adjusted: Bij represents the bandwidth allocation ratio from i to j; based on bandwidth allocation and link reliability, an adaptive routing strategy R is designed: R = h(B, T*).

[0111] This embodiment achieves efficient data exchange and resource allocation of heterogeneous drone clusters by establishing a multi-dimensional information transmission optimization mechanism. First, the importance of information is evaluated from three dimensions: timeliness, task relevance, and communication cost, and a scientific information value measurement system is established; then, the transmission priority is dynamically adjusted based on the importance of information and communication quality, and differentiated allocation of network resources is achieved; finally, the timely and reliable transmission of important information is ensured through the coordinated optimization of the bandwidth allocation matrix and the adaptive routing strategy. In particular, by introducing the adaptive routing mechanism, the system can dynamically adjust the transmission path according to the network status and data characteristics, effectively avoiding network congestion and information loss. This embodiment not only improves the utilization efficiency of the network, but also achieves more reasonable resource allocation by considering the value of information, ensuring the quality of information interaction of the cluster system in a complex environment.

[0112] According to one aspect of the present application, step S41 is further:

[0113] S411. Obtain the communication data to be transmitted from the UAV communication buffer area; extract the type label of the communication data based on the data packet header information, generate a timestamp and a target address, and generate a data feature vector; use Shannon entropy to calculate the amount of information of the communication data to obtain an information entropy value.

[0114] S412, obtaining a data feature vector; reading the current system time from the time management module, calculating the survival time of the communication data, and generating an initial timeliness value; normalizing the initial timeliness value based on an exponential decay model to obtain a timeliness value.

[0115] S413, obtaining current task status data from the task scheduling module; using a semantic similarity algorithm to calculate the matching degree between the data feature vector and the task status data to generate an initial correlation; and using a context-aware weighted method to correct the initial correlation to obtain a correlation value.

[0116] S414, receiving an information entropy value; obtaining current network load data from a network status monitoring module; based on the information entropy value and the network load data, calculating the bandwidth resources and energy consumption required to transmit the communication data, and generating a resource consumption vector; mapping the resource consumption vector through a cost function to obtain a communication cost value.

[0117] S415, obtaining a timeliness value, a relevance value and a communication cost value; constructing an adaptive fuzzy reasoning system, taking the three values ​​as input variables, and generating fuzzy importance; using a dynamic defuzzification method to clarify the fuzzy importance and obtain an initial importance value.

[0118] S416, receiving an initial importance value; reading recent importance history data from a history database; using a recurrent neural network to perform time series analysis on the importance history data and the initial importance value to generate an importance prediction value; performing weighted fusion of the importance prediction value and the initial importance value to obtain a final information importance index.

[0119] In one embodiment of the present application, information entropy calculation and fuzzy reasoning algorithm: I(X) = -Σp(xi)·log(p(xi)) + β·T(t)·exp(-Δt) + ω·R(x|c); wherein I(X) is the comprehensive information importance; p(xi) is the probability distribution of information symbols; T(t) is the timeliness function, and t is the difference between the data generation time and the current time; R(x|c) is the task relevance function, and R(x|c) = x·c / (||x||·||c||) is calculated by cosine similarity; β and Δ are timeliness parameters; ω is the relevance weight; fuzzy reasoning adopts the Mamdani model: μout = max{min[μA(x), μB(y)]}, μA and μB are the membership functions of the input fuzzy set; defuzzification adopts the centroid method.

[0120] The exponential decay model is specifically: E(t) = exp(-λ(t)·Δt)·(1 + α·S(t) + β·U(t)); where E(t) is the timeliness scoring function; Δt is the data survival time; λ(t) is the adaptive decay rate λ(t) =λ0·(1 +ω·V(t)), V(t) is the data value assessment function, and λ0 is the initial decay rate; S(t) is the data state factor S(t) = (1 -exp(-γ·Q(t))) / (1 + exp(-γ·Q(t))), Q(t) is the data quality measure; U(t) is the urgency function U(t) = min(1, (D - t) / D), D is the deadline; α, β, γ, ω are adjustment parameters; normalized output: N(t) = E(t) / max{E(τ): τ∈[tT, t]}, T is the time window size.

[0121] Dynamic defuzzification method: O(μ) = ∫(x·μ(x)·w(x))dx / ∫(μ(x)·w(x))dx + λ·R(μ); where O(μ) is the defuzzification output; μ(x) is the membership function of the fuzzy set; w(x) is the dynamic weight function w(x) = 1+ α·|▽μ(x)|·exp(-β·||xx*|| 2 ), x* is the local optimal solution; R(μ) is the regularization term R(μ) = -∫μ(x)·log(μ(x))dx; λ is the balance coefficient; weight update: w(t+1) = w(t) + η·(ΞJ / Ξw)·(1 - exp(-ρ·|ΞJ / Ξw|)), where J is the performance indicator, η is the learning rate, and ρ is the adaptive coefficient.

[0122] This embodiment realizes the accurate quantification of the importance of heterogeneous drone cluster communication data by establishing a multi-dimensional information value evaluation system. First, based on the packet header information, the feature vector is extracted and the information entropy is calculated to accurately quantify the amount of information in the data; then the timeliness of the data is evaluated by the exponential decay model, which dynamically reflects the value decay of information over time; then the semantic similarity algorithm and the context-aware weighted method are used to evaluate the task relevance, ensuring the task orientation of information evaluation; finally, the comprehensive evaluation and dynamic prediction of information importance are realized through the adaptive fuzzy reasoning system and the recurrent neural network. In particular, by introducing the dynamic defuzzification method and the timing analysis mechanism, the system can adaptively adjust the evaluation weights and accurately grasp the dynamic change characteristics of information value. This embodiment not only improves the pertinence and efficiency of resource allocation, but also realizes a more comprehensive and accurate information value measurement by considering multiple evaluation dimensions, providing a reliable decision-making basis for subsequent transmission optimization.

[0123] According to one aspect of the present application, step S42 is further:

[0124] S421, obtaining an information importance index; performing a time series analysis on the information importance index using a sliding time window method to generate an importance change sequence; based on the importance change sequence, using a trend prediction algorithm to calculate the short-term importance change trend to obtain an importance trend vector.

[0125] S422, receiving a communication quality indicator; using an adaptive weighted average method to filter out noise from the communication quality indicator to generate a smoothed quality indicator; calculating a change gradient of the smoothed quality indicator to obtain a quality change matrix that characterizes the dynamic characteristics of the communication quality.

[0126] S423, obtaining the importance trend vector and the quality change matrix; constructing a dynamic correlation analysis model, calculating the time-varying correlation coefficient between the importance trend vector and the quality change matrix, and generating a related feature vector; using an adaptive threshold segmentation algorithm to segment the related feature vector to obtain segmented feature data.

[0127] S424, receiving segmented feature data; based on a fuzzy inference system, mapping the segmented feature data to an initial transmission priority value to generate an initial priority vector; using a competition mechanism to separate similar priority values ​​in the initial priority vector to obtain a differentiated priority vector.

[0128] S425. Obtain a priority vector; design a priority scheduling model based on queue theory, dynamically adjust the priority value in the priority vector according to the system load, and generate an adjusted priority vector; use a feedback control mechanism to optimize the stability of the adjusted priority vector to obtain a stable priority vector.

[0129] S426, receiving a stable priority vector; using a sequence-to-sequence mapping method, converting the stable priority vector into a transmission sequence with timing constraints, and generating transmission priority data representing the transmission priority order.

[0130] In one embodiment of the present application, the dynamic correlation analysis model is: C(t) = ρ·Σ[K(t-τ)·IR(τ)·Q(τ)] + (1-ρ)·||▽IR(t)||·||▽Q(t)||·cos(θ(t)); wherein C(t) is the dynamic correlation function; K(t-τ) is the exponential decay kernel function K(t-τ) = exp(-α|t-τ|); IR(τ) is the information importance time series; Q(τ) is the communication quality time series; ▽IR(t) and ▽Q(t) are the time gradients of importance and quality, respectively; θ(t) is the gradient vector angle; ρ is the historical correlation weight coefficient; priority calculation function P(t) = σ(W·C(t) + b), wherein σ(·) is the Sigmoid function, and W and b are adaptive parameters; dynamic update equation: W(t+1) = W(t) - η·ΞL / ΞW, L is the priority prediction error.

[0131] Priority scheduling model based on queue theory: Q(p) = Σ[wi·Li(p)·exp(ρi·Wi(p))] +α·B(p) + β·F(p); where Q(p) is the priority evaluation function; Li(p) is the length of the i-th level queue; Wi(p) is the average waiting time; ρi is the time sensitivity coefficient; B(p) is the load balancing metric B(p) = Σ(qi - q*) 2 / n, qi is the load of queue i; F(p) is the fairness index F(p) = -Σ(si / s*)·log(si / s*), si is the service ratio; wi, α, β are weight coefficients; scheduling decision: p* = argmin{Q(p): p∈P}, where P is the feasible priority set; dynamic priority adjustment is achieved through reinforcement learning.

[0132] This embodiment realizes intelligent scheduling and dynamic priority allocation of communication data by constructing a transmission priority evaluation mechanism based on dynamic correlation analysis. First, the information importance index is analyzed in time series and trend predicted to grasp the law of importance change; then, the communication quality index is filtered out by adaptive weighted average method to improve the reliability of quality evaluation; then, the dynamic correlation analysis model is used to evaluate the correlation characteristics of importance and quality change to realize the dynamic adaptability of priority evaluation; finally, the fuzzy reasoning system and competition mechanism are used to realize the precise division and dynamic adjustment of priority. In particular, by introducing queue theory and feedback control mechanism, the system can adaptively adjust the priority allocation strategy according to the load situation, effectively avoiding imbalance in resource allocation and transmission congestion. This embodiment not only improves the accuracy and efficiency of transmission scheduling, but also realizes more flexible and reliable priority management through multi-level adaptive optimization.

[0133] According to one aspect of the present application, step S43 is further:

[0134] S431. Receive transmission priority data; stratify the transmission priority data using a quantile statistics method to generate a priority stratification result; based on the priority stratification result, construct a transition probability matrix between priority levels to obtain an inter-layer transfer matrix.

[0135] S432. Obtain current channel status information from the network monitoring module; use a channel detection algorithm to evaluate the real-time quality of each communication link and generate a link quality vector; combine the link quality vector and use a channel capacity estimation model to calculate the maximum available bandwidth of each link to obtain an available bandwidth matrix.

[0136] S433. Obtain the priority stratification results and the available bandwidth matrix; construct a bandwidth demand prediction model based on queuing theory, analyze the transmission requirements of data of different priorities, and generate a demand prediction vector; calculate the minimum bandwidth guarantee value of each priority layer according to the demand prediction vector to obtain the bandwidth guarantee threshold.

[0137] S434, receiving bandwidth guarantee threshold and demand prediction vector; designing a resource allocation algorithm based on weighted proportional fairness, calculating the initial allocation ratio of each communication link under the premise of meeting the minimum bandwidth guarantee, and generating an initial allocation plan; using a congestion prediction model to evaluate the performance of the initial allocation plan and obtain a congestion risk indicator.

[0138] S435, obtaining an initial allocation plan and a congestion risk indicator; constructing a dynamic bandwidth adjustment model, optimizing the initial allocation plan according to the congestion risk indicator, and generating an adjusted allocation plan; using a resource utilization evaluation algorithm to verify the rationality of the adjusted allocation plan, and obtaining a plan evaluation result.

[0139] S436, receiving the adjustment allocation plan and the plan evaluation result; designing a stability optimization algorithm for bandwidth allocation based on feedback control theory, fine-tuning the adjustment allocation plan, and generating an optimized allocation plan; converting the optimized allocation plan into a standard format to obtain a bandwidth allocation matrix.

[0140] In one embodiment of the present application, the bandwidth demand prediction and resource allocation algorithm is: B(i, j) = μ·D(i, j)+ ν·exp(F(i, j) / F*) + ξ·(1-exp(-λ·U(i, j))); wherein B(i, j) is the bandwidth allocation of link (i, j); D(i, j) is the predicted value of data transmission demand, calculated by LSTM network; F(i, j) is the data flow rate; F is the reference flow rate; U(i, j) is the link utilization; μ, ν, ξ, λ are adaptive weight coefficients; resource allocation constraint: ΣB(i, j) ≤ Bmax, min{B(i, j)} ≥ Bmin; fairness index: J = -Σ(B(i, j) / B)·log(B(i, j) / B*), B* is the average bandwidth; dynamic adjustment is achieved by gradient projection method.

[0141] Weighted proportional fair resource allocation algorithm: W(r) = Σ[ui·log(ri + ε)] - α·V(r) - β·D(r); where W(r) is the resource allocation utility function; ri is the resource allocation amount of the i-th link; ui is the user weight; ε is the smoothing factor; V(r) is the variance penalty term V(r) = Σ(ri - r*) 2 / n, r is the average allocation; D(r) is the demand deviation measure D(r) = Σmax(0, di - ri), di is the demand; α and β are penalty coefficients; resource allocation constraints: Σri ≤ R, ri ≥rmin; optimization solution: r = argmax{W(r)}; adaptive update: r(t+1) = r(t) + η·▽W(r)·(1 +γ·E(t)), where E(t) is the congestion indicator, η is the step size, and γ is the adaptive coefficient.

[0142] This embodiment realizes intelligent communication resource management of heterogeneous drone clusters by constructing a bandwidth allocation mechanism based on queuing theory and resource optimization. First, the transmission priority data is stratified by the quantile statistics method, and a scientific priority level system is established; then, the real-time quality of the link and the available bandwidth are evaluated by the channel detection algorithm and the capacity estimation model to ensure the feasibility of resource allocation; then, a bandwidth demand prediction model is constructed based on queuing theory to accurately grasp the transmission requirements of data with different priorities; finally, the optimal allocation of bandwidth is realized through the weighted proportional fairness resource allocation algorithm and congestion prediction model. In particular, by introducing a dynamic bandwidth adjustment model and feedback control theory, the system can adaptively adjust the bandwidth allocation strategy according to the network status and transmission requirements, effectively improving the resource utilization efficiency. This embodiment not only ensures the transmission quality of high-priority data, but also realizes more reasonable and efficient resource allocation through multi-level optimization and adjustment, ensuring the communication performance and reliability of the cluster system.

[0143] like Figure 6 As shown, according to one aspect of the present application, step S5 is further:

[0144] S51, obtaining the output probability distribution of the hierarchical strategy model, calculating the entropy value of the probability distribution, and generating an uncertainty index representing the uncertainty of the strategy;

[0145] S52. Based on the uncertainty index, construct the robustness constraint condition for limiting the update step of the strategy;

[0146] S53. Under the premise of satisfying the robustness constraint condition, optimizing the expected cumulative reward of the hierarchical strategy model to obtain an optimized strategy model;

[0147] S54. Build a communication relationship based on the optimal communication topology structure; perform data transmission based on the communication relationship and the adaptive routing strategy, so that each UAV executes the decision action corresponding to the optimization strategy model according to the pre-stored local observation data and the data from the communication neighbors.

[0148] In one embodiment of the present application, the uncertainty index U of the strategy output is calculated: U = -∑π(a|s)·logπ(a|s); a robustness constraint condition based on the uncertainty index is constructed: ||π'(a|s) - π(a|s)||2 ≤ ε·(1-U); where ε is an adjustable constraint coefficient; the optimization problem considering robustness is solved: maxπ E[R|π] subject to: robustness constraint; based on the optimal communication topology, the distributed execution of the strategy is realized: each agent executes the optimized strategy based on local observations and neighbor information.

[0149] This embodiment improves the control stability and adaptability of heterogeneous drone clusters by constructing a robust optimization framework based on uncertainty perception. First, the uncertainty of the decision is quantified by calculating the entropy value of the policy output, and an evaluation mechanism for the reliability of the policy is established; then, robustness constraints are constructed based on uncertainty indicators to limit the risk of policy updates; finally, stable iteration and efficient deployment of the policy are achieved through the distributed execution of the optimization model. In particular, by introducing entropy measurement and robustness constraints, the system can control the uncertainty of policy updates while ensuring performance improvement, effectively avoiding decision shocks and performance crashes. This embodiment not only improves the robustness and reliability of the system, but also achieves more efficient policy deployment through distributed execution, so that the entire cluster system has stronger environmental adaptability and task completion capabilities.

[0150] According to one aspect of the present application, step S53 is further:

[0151] S531, obtaining robustness constraints; using Lagrangian relaxation method to convert the robustness constraints into penalty terms, generating a constraint penalty function; constructing an augmented Lagrangian function based on the constraint penalty function, and obtaining an optimization objective function.

[0152] S532, reading historical reward data from the strategy evaluation module; smoothing the historical reward data using an exponential moving average method to generate a smoothed reward sequence; performing baseline correction on the smoothed reward sequence using an adaptive deviation correction algorithm to obtain a corrected reward sequence.

[0153] S533, receiving the optimization objective function and the correction reward sequence; constructing a trust region constraint, limiting the maximum step length of the strategy update, and generating a trust region constraint condition; combining the trust region constraint condition with the optimization objective function to obtain a constrained optimization problem.

[0154] S534, obtaining a constrained optimization problem; using an adaptive momentum estimation algorithm to calculate the policy gradient and generate a gradient estimation vector; constructing a second-order approximate model based on the gradient estimation vector to obtain an approximate optimization model.

[0155] S535, receiving an approximate optimization model; solving the approximate optimization model using a conjugate gradient method to generate a policy update vector; applying an adaptive learning rate adjustment algorithm to the policy update vector to obtain an adjusted update vector.

[0156] S536, obtaining an adjustment update vector; applying the adjustment update vector to the current strategy model to generate an updated strategy model; and fine-tuning the updated strategy model using a residual network structure to obtain an optimized strategy model.

[0157] In one embodiment of the present application, the trust region constrained optimization algorithm is: L(θ, λ) = E[R(τ)] - α·KL(πθ_old||πθ) + β·H(πθ) + λ·(Δ - D(θ, θ_old)); where L(θ, λ) is the constraint optimization objective function; R(τ) is the trajectory return; KL(·||·) is the KL divergence of the policy distribution; H(πθ) is the policy entropy; D(θ, θ_old) is the parameter distance metric; α and β are trade-off coefficients; λ is the Lagrange multiplier; Δ is the trust region radius; policy update: θ_new = θ_old +sqrt(2Δ / g T ·F -1 ·g)·F -1 ·g, where g is the policy gradient, F is the Fisher information matrix, and θ_new is the updated trust region radius value; adaptive trust region adjustment: Δ_new = clip(ζ·Δ_old, Δ_min, Δ_max), ζ is the adaptation coefficient, where Δ_old is the current trust region radius value, Δ_min is the minimum trust region radius, Δ_max is the maximum trust region radius, and clip( ) is the truncation function.

[0158] Adaptive momentum estimation algorithm: G(θ) = β1·m(t-1) + (1-β1)·g(t) + λ·Δ(t); v(t) = β2·v(t-1) + (1-β2)·g(t) 2 ; where G(θ) is the gradient estimate; m(t) is the first-order momentum; v(t) is the second-order momentum; g(t) is the current gradient; Δ(t) is the gradient correction term Δ(t) = α·sign(g(t))·||g(t)||·exp(-ρ·||g(t-1) - g(t)||); β1 and β2 are momentum decay rates; λ, α, ρ are adaptive parameters; parameter update: θ(t+1) = θ(t)- η·G(θ) / (sqrt(v(t)) + ε)·(1 + ω·C(t)), where C(t) is the confidence coefficient, η is the basic learning rate, ε is the numerical stability constant, and ω is the adaptive weight.

[0159] This embodiment achieves stable optimization and performance improvement of heterogeneous UAV cluster control strategy by constructing a policy optimization framework based on trust region constraints. First, the Lagrangian relaxation method is used to transform the robustness constraint into a penalty term, and a complete optimization objective function is constructed; then, the reward signal is preprocessed by the exponential moving average method and the adaptive deviation correction algorithm to improve the stability of the optimization process; then, the trust region constraint is used to limit the strategy update step size to avoid performance collapse caused by over-optimization; finally, efficient strategy optimization is achieved through the second-order approximation model and the conjugate gradient method. In particular, by introducing the adaptive momentum estimation algorithm and the residual network structure, the system can maintain the stability of the strategy while ensuring the optimization efficiency, effectively balancing the relationship between exploration and utilization. This embodiment not only improves the stability and efficiency of the optimization process, but also achieves more reliable and efficient strategy updates through multi-level constraint optimization, ensuring the control performance of the cluster system in complex environments.

[0160] According to one aspect of the present application, a MAPPO-based drone cluster control and navigation method comprises the following steps:

[0161] Step 1: System initialization and data collection. When the system starts, it is first initialized, including the setting of drone hardware parameters, activation of the communication module, and calibration of the navigation system. Each drone collects environmental information through a variety of sensors (such as lidar, camera, inertial measurement unit). The collected environmental information is processed by the data preprocessing module, including denoising, normalization, and feature extraction. The denoising process uses a Kalman filter to reduce sensor noise and ensure data accuracy; normalization unifies the data scale for subsequent algorithm processing; feature extraction includes the extraction of key data such as obstacle position and relative speed, providing reliable input for subsequent decision-making.

[0162] Step 2: Centralized training. The centralized training phase is conducted in a high-fidelity simulation environment, using the multi-agent proximal policy optimization (MAPPO) algorithm to jointly train the drone cluster. The simulation environment ensures the generalization ability of the training strategy in practical applications by simulating dynamic environments such as complex terrain, obstacle distribution, and wind speed. During the training process, each drone acts as an intelligent agent and jointly optimizes its strategy in a centralized value network. In this process, a global information estimation mechanism is introduced, so that each drone can obtain information about the group center through the central server, including the mean position and mean speed of all drones, so as to consider the global situation when updating the strategy and enhance the collaborative ability of the cluster.

[0163] The specific training adopts the experience replay and policy gradient optimization methods. The state, action, reward and next state of each drone are stored in the experience replay buffer, and the policy network and value network are updated by small batch sampling. In order to improve the stability of training, the entropy regularization term is used to prevent the strategy from converging to the local optimum too early, and the sophistication of the strategy is gradually improved by periodically reducing the learning rate.

[0164] Step 3, Adaptive execution phase. In the actual execution phase, each UAV independently performs tasks according to the strategy obtained through centralized training. Since the UAV can only obtain local observation information during the execution process, it is necessary to obtain global information approximately through an adaptive estimation module. The adaptive estimation module is implemented based on a deep neural network (DNN). Its input is the local observation data of the current UAV, including the relative position, speed and environmental characteristics with neighboring UAVs, and its output is the estimated group center position and movement trend. The neural network is optimized through centralized learning during the training phase to ensure that the global state can be accurately estimated during the execution phase.

[0165] During execution, the drone uses the adaptively estimated group center information to adjust its own motion parameters through a proportional-integral-derivative (PID) controller. The input of the PID controller is the error between the current position of the drone and the estimated group center, and the output is the correction amount for the speed and direction of the drone, thereby achieving dynamic maintenance and navigation control of the cluster. In order to adapt to the dynamic changes of the environment, the controller's gain parameters (K_p, K_i, K_d) are adaptively adjusted through online learning to cope with different mission requirements and environmental conditions.

[0166] Step 4: Dynamic reward optimization. Throughout the execution process, the drone cluster needs to cope with multiple task requirements, such as target navigation, obstacle avoidance, and maintaining cluster compactness. Therefore, a dynamic reward optimization mechanism is introduced to dynamically adjust the reward weights of each sub-goal according to the task requirements. At each time step, the reward function is dynamically updated according to the current task priority. For example, when the drone approaches the target area, the weight of the navigation reward will increase, while in the obstacle-dense area, the weight of the obstacle avoidance will increase.

[0167] Dynamic reward optimization is implemented through an adaptive algorithm based on policy performance. Specifically, the system regularly evaluates the task completion of the current policy, such as target arrival rate, number of collisions, etc., and adjusts the weights of each reward item in real time based on the evaluation results to achieve the optimal balance between multiple objectives. This optimization process is carried out through the policy gradient method in reinforcement learning to ensure that the adjustment of reward weights can effectively promote policy improvement, and ultimately improve the task completion and collaboration efficiency of the entire cluster.

[0168] In one embodiment of the present application, during the centralized training phase, the data of all drones (including position, speed, acceleration, etc.) are synchronously collected and used to establish an accurate global information model. At this stage, the entire system aggregates and processes data through a central server, and the status information of each drone is transmitted to the central server through a communication network. The server calculates the position of the group center and its motion status based on the information of all drones.

[0169] Each drone obtains local observation data (such as distance and angle with neighboring drones) through its own sensors and sends these data to the central server. In order to achieve real-time performance, a data transmission architecture based on ROS (Robot Operating System) is adopted, and the data of all drones is transmitted to the central node through a publish / subscribe mode.

[0170] After receiving the data from all drones, the central server calculates the center position, average speed, and overall movement trend of the entire drone group. The position of the group center can be calculated using the following formula: Ct=1 / N∑ i=1 N Pi(t); where Ct is the group center position at time t, Pi(t) represents the position of the i-th drone at time t, and N is the total number of drones. The training process adopts a centralized training method, that is, during training, each drone can obtain the global information of other drones (such as the group center position, speed, etc.), so as to optimize the strategy based on the global state. This allows drones to better learn how to maintain the stability of the cluster in different environments.

[0171] During the execution phase, due to communication limitations between drones, it is impossible to obtain global state information in real time, so an adaptive estimation module is needed to approximate the global state of the group. The adaptive estimation module uses a deep neural network (DNN) to estimate the position of the group center. Specifically, the deep neural network consists of two fully connected layers, and the input is the local observation information of the drone, including: the distance and relative speed between neighboring drones; its own state information, including position, speed and direction. The output is an estimate of the group center position Ct*. The network is trained using supervised learning in the training phase. The training label is the precise group center position Ct calculated in the centralized training phase. The goal is to minimize the mean square error (MSE) between the network output and the true group center: L=1 / M∑ t=1 M (Ct-Ct*) 2 ; Where M represents the number of samples.

[0172] Before inputting into the neural network, the observation data is standardized to ensure that the data is within a reasonable range and to prevent the gradient from disappearing or exploding due to different data scales. In addition, historical data is used to extract time series features from the observation information to enhance the prediction ability of the neural network. During execution, the deep neural network is updated online through the observation data collected in real time by the drone to continuously improve the estimation accuracy of the group center. This process adopts the incremental learning method, that is, the model is continuously fine-tuned during execution to adapt it to the dynamically changing environment.

[0173] The adaptive adjustment mechanism is used to adjust the behavior of each drone based on the estimated group center information to ensure the overall consistency and stability of the cluster. The adaptive adjustment mechanism uses a proportional-integral-derivative (PID) control scheme to adjust the behavior of drones. The control goal is to minimize the distance error between each drone and the group center. The input of the PID controller is the deviation between the current position of the drone and the estimated group center, and the output is the correction amount for the speed and direction of the drone: u(t)=K p e(t)+ K i ∫ o t e(τ)dτ+ K d (de(t) / dt); where e(t) is the distance error between the drone and the group center at time t, e(τ) is the distance error between the drone and the group center at time τ, d represents the differential operator, and K p , K i and K d are proportional, integral and differential gain coefficients respectively. These gain coefficients are tuned through reinforcement learning methods to adapt to different environments and task requirements.

[0174] In order to strike a balance between navigation, obstacle avoidance, and cluster maintenance, the adaptive adjustment mechanism also integrates a multi-objective optimization strategy. When the drone approaches an obstacle, the weight of the obstacle avoidance goal will increase, thereby reducing the priority of keeping consistent with the center of the group and ensuring the safety of the drone. When there are no obstacles, the weight of cluster maintenance will increase to maintain the tight structure of the cluster. Each drone is adjusted not only based on the error with the center of the group, but also dynamically adjusted based on the relative state with neighboring drones. For example, when neighboring drones are too close, the adaptive adjustment mechanism introduces additional collision avoidance constraints and uses reverse repulsion to disperse to avoid collision. This relative state feedback is obtained through the local information perception module and executed through the incremental control strategy.

[0175] The adaptive estimation module and adjustment mechanism were fully tested in a high-fidelity simulation environment to verify the accuracy of the group center estimation and the effectiveness of the adjustment of drone behavior. The test showed that the drone cluster based on guided global information estimation can still maintain good overall consistency and task completion rate under the condition of limited communication range. In actual tests, drones were deployed in complex environments, such as urban construction areas and forests. By comparing the performance of the system with and without adaptive estimation, it was found that the drone cluster using this mechanism showed significant advantages in obstacle avoidance and navigation, with a smaller overall cluster dispersion and a shorter target arrival time.

[0176] This embodiment adopts the multi-agent proximal policy optimization (MAPPO) algorithm to obtain the optimal strategy through centralized training, and distributes the strategy to each drone during the execution phase. This centralized training and distributed execution framework has the following specific implementation methods: During the centralized training phase, the strategies of all drones are optimized in a simulation environment. The simulation environment is implemented based on high-fidelity simulators such as Gazebo or Unity, which are used to simulate the real physical environment of large-scale drone clusters. MAPPO uses a centralized value network to calculate the joint value function using the joint state and action of all drones. Training uses an experience replay buffer to collect the state, action, reward and next state of all drones, and updates the policy network and value network through small batch stochastic gradient descent. In order to ensure policy convergence, an entropy regularization method is used to prevent the strategy from converging to the local optimum too early, and a more stable training process is achieved by periodically reducing the learning rate.

[0177] During the execution phase, the centrally trained policy is distributed to each drone through the model deployment module. Each drone runs the policy independently and only uses local observation data to make decisions. To achieve this process, the drones are equipped with lightweight computing units, such as edge computing modules based on the NVIDIA Jetson series, to ensure that the policy network can run in real time. During execution, drones share necessary status information through frequency-limited communication protocols (such as low-frequency broadcasts) to achieve collaborative obstacle avoidance and cluster maintenance. This communication protocol is implemented by UDP, which ensures low latency and high reliability, while reducing communication overhead by compressing communication data.

[0178] During the centralized training phase, the system simulates actual communication constraints, such as setting communication radius and frequency limits, so that the strategy can adapt to the communication conditions in the execution phase during the training phase. Specifically, a distance-based communication trigger mechanism is adopted. When the distance between drones is less than the preset threshold, data exchange is triggered, thereby effectively reducing unnecessary communication load.

[0179] In the control and navigation of drone swarms, the mission objectives are usually diverse and complex. For example, in a navigation mission, it is necessary not only to achieve the arrival of the target point, but also to avoid obstacles and maintain the overall consistency of the cluster. To this end, this embodiment introduces a multi-objective-based dynamic reward mechanism to ensure the collaborative efficiency and task completion of drones in complex dynamic environments. The following is a specific implementation method: The reward function is designed as a staged, multi-objective dynamic form. At each time step, the reward function is dynamically adjusted according to the current mission requirements. For example, the reward for the navigation task is negatively correlated with the distance between the drone and the target point, while the reward for the obstacle avoidance task is positively correlated with the distance between the drone and the obstacle. In implementation, a conditional function is used to control the reward weights at different stages, for example: rt=w1f1(d target )+w2f2(d obstacle )+w3f3(d center ), where f1, f2, and f3 are the reward functions for reaching the target, avoiding obstacles, and maintaining the cluster, respectively; w1, w2, and w3 are dynamically adjusted weight factors; and d target is the distance between the UAV and the target point, d obstacle is the distance between the UAV and the obstacle, d center The distance between the drone and the cluster center is adjusted by a linear scheduler or an adaptive algorithm based on policy performance.

[0180] During the concentrated training phase, the reward weights of each goal are dynamically adjusted for different tasks. To this end, a weight optimization module is constructed to adjust the weights of each sub-goal based on the value of the policy gradient in reinforcement learning. In specific implementation, the weight optimization module increases or decreases the weights of related goals in real time by observing the completion status of each task (such as target arrival rate, number of collisions, etc.) to achieve overall performance improvement. At fixed training rounds, the module uses an optimization method similar to random search to find the best weight combination at the current stage, thereby effectively balancing the conflicts between multiple goals.

[0181] In the execution phase, in order to enhance the overall collaboration of the cluster, this embodiment constructs a distributed update mechanism for reward signals. Each drone calculates rewards based on its own observation information and feedback from neighboring drones, and this process is implemented through a decentralized signal convergence algorithm. Specifically, each drone will exchange state information with other drones within a preset communication radius and use this information to calculate local rewards. This process uses a distributed aggregation algorithm based on weighted average, so that each drone not only obtains its own reward signal, but also obtains reward feedback from its neighbors, thereby better performing cluster collaboration.

[0182] Through the above implementation method, this embodiment has significant advantages in improving the stability and collaboration efficiency of drone clusters, and is particularly suitable for large-scale drone mission scenarios that require complex navigation and cluster control. Based on the centralized training and distributed execution framework of MAPPO, as well as the introduction of global information estimation and adaptive adjustment mechanisms, this system can better cope with partial observation problems, reduce communication burdens, and ensure efficient collaboration in dynamic environments. Compared with the traditional Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, MAPPO adopts centralized training and distributed execution, so that each drone can obtain global information during the training phase and make autonomous decisions based on local observations during the execution phase, thereby improving the stability of the system and the collaboration efficiency among multiple agents. By introducing guided global information estimation and adaptive adjustment mechanisms, this embodiment can approximate the center of the group through an adaptive estimation module when drone communication is limited, thereby effectively solving partial observation problems and ensuring the collaboration capabilities of drones in complex environments. Traditional centralized control methods often lack sufficient flexibility and adaptability, and it is difficult to quickly respond to dynamically changing environments and mission requirements. This embodiment adopts an adaptive adjustment mechanism and a dynamic reward optimization mechanism. Through the PID controller and the dynamically adjusted reward weight, the drone can quickly adjust its behavior when the environment changes and the task requirements change, thereby improving its adaptability to complex dynamic environments. The traditional centralized control method relies on the central controller to obtain the status information of all drones in real time, which increases the communication burden and the computing pressure of the central node. This embodiment adopts a distributed execution strategy and a distance-based communication trigger mechanism, and only exchanges information when necessary, which reduces the communication burden, reduces the system delay and the risk of overload of the central node. Compared with traditional centralized control, this embodiment realizes decentralized distributed control. Each drone makes independent decisions based on local information without relying entirely on the central node, thereby improving the scalability of the system and its availability in large-scale cluster tasks. This embodiment uses a dynamic reward mechanism to dynamically adjust the reward weights of each target according to the task requirements. For example, effective weight adjustments are made between multi-target tasks such as navigation, obstacle avoidance and cluster maintenance to ensure that the drone achieves the best balance between different tasks. The flexibility and multi-task collaboration capabilities of the system are improved to ensure the efficient performance of the cluster in a complex environment.

[0183] The present invention realizes intelligent management of the entire process from heterogeneity characterization, communication optimization, task allocation to strategy execution by constructing a complete set of heterogeneous UAV cluster control and navigation methods. First, an accurate heterogeneity measurement system is established through dynamic characteristic analysis and standardized processing to provide a reliable basis for subsequent decision-making; secondly, through dynamic communication topology optimization and adaptive routing strategy, efficient and reliable communication of the cluster system is ensured; thirdly, through the hierarchical strategy model, the collaborative optimization of task allocation and execution control is realized, and the overall performance of the system is improved; finally, through uncertainty perception and robustness optimization, the environmental adaptability of the system is enhanced. In particular, these modules are organically integrated through the MAPPO algorithm framework to achieve multi-level optimization from local to global. The present invention not only overcomes the limitations of traditional methods in dealing with heterogeneity, communication optimization and decision-making coordination, but also improves the task execution capability and system stability of heterogeneous UAV clusters in complex environments.

[0184] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. The MAPPO-based UAV cluster control and navigation method is characterized by: The steps include: S1. Obtain the basic characteristic data of the UAV, generate a standardized dynamic parameter matrix through dynamic characteristic analysis and standardization processing; calculate the comprehensive capability index of the UAV based on the standardized dynamic parameter matrix; and construct a cluster heterogeneity matrix that characterizes the degree of difference between UAVs based on the comprehensive capability index of the UAV; The basic characteristic data include the maximum speed, maximum acceleration, energy capacity, mission payload type and communication range of the UAV; S2. Obtain real-time communication data and calculate the communication quality index between each pair of drones; based on the communication quality index, predict the link reliability data; Based on the cluster heterogeneity matrix and link reliability data, a dynamic communication graph is constructed; the dynamic communication graph is optimized and calculated to generate the optimal communication topology structure that meets the connectivity constraints and bandwidth constraints; S3, based on the cluster heterogeneity matrix and the optimal communication topology, calculate the task allocation fitness matrix; Based on the task allocation fitness matrix, a hierarchical strategy model including task allocation strategy and execution strategy is constructed; based on the hierarchical strategy model, a hierarchical value function for evaluating global synergy effect and local execution effect is constructed; S4. Based on the hierarchical value function, the real-time communication data is analyzed for timeliness, task relevance and communication cost to generate information importance index; Calculate transmission priority data based on information importance index and communication quality index; Based on the transmission priority data, a bandwidth allocation matrix is ​​generated; Construct an adaptive routing strategy based on the bandwidth allocation matrix and the optimal communication topology; S5. Based on the adaptive routing strategy, calculate the entropy value of the strategy output and generate an uncertainty index; Based on uncertainty indicators, robustness constraints are constructed; under the robustness constraints, the hierarchical strategy model is optimized to obtain the optimized strategy model; based on the optimal communication topology, the distributed execution of the optimized strategy model is realized; Step S1 is further as follows: S11, obtaining basic characteristic data of the UAV, reorganizing the basic characteristic data according to a preset characteristic vector format, and generating an initial characteristic vector including a maximum velocity component, a maximum acceleration component, an energy capacity component, a mission load type component, and a communication range component; S12, using a preset linear mapping weight matrix to perform a linear transformation on the initial feature vector to obtain a linear transformation vector; adding the linear transformation vector to a preset bias vector to generate a standardized dynamic parameter matrix; S13, based on the preset capability evaluation weight vector, weighted summing up the parameter components in the standardized dynamic parameter matrix, and calculating the comprehensive capability index of the UAV that characterizes the comprehensive performance of the UAV; S14, calculating the binary norm of the difference between the comprehensive capability indexes of any two drones, and dividing the binary norm by the maximum value of the comprehensive capability index of the two drones to obtain a heterogeneity value representing the heterogeneity degree of the drones; organizing the heterogeneity values ​​between all the drones into a matrix form to generate a cluster heterogeneity matrix; Step S12 is further as follows: S121, using a sliding window method to perform time series sampling on each component in the initial feature vector to generate a multi-scale feature tensor containing features of different time scales; calculating the mean and standard deviation of the feature components at each time scale in the multi-scale feature tensor to generate a feature statistical matrix; S122, normalizing the multi-scale feature tensor based on the mean and standard deviation in the feature statistical matrix to obtain a normalized feature tensor; The principal component analysis method is used to reduce the dimension of the normalized feature tensor and generate a reduced-dimensional feature vector; S123, reading a preset mapping weight matrix from a dynamics parameter database; based on the normalized feature tensor, using an adaptive moment matching algorithm to calculate high-order statistics of feature distribution to generate a distribution feature vector; based on the distribution feature vector, updating parameter values ​​in the mapping weight matrix to obtain an updated weight matrix; S124, performing a matrix multiplication operation on the dimension-reduced feature vector and the updated weight matrix to generate a linear transformation vector; reading a preset bias vector from a dynamic parameter database, adding the bias vector to the linear transformation vector to obtain a feature mapping matrix; S125, calculating the covariance matrix of the feature mapping matrix in different feature dimensions to generate a feature correlation matrix; Based on the feature correlation matrix, an adaptive feature decoupling algorithm is used to eliminate the redundant correlations between features and generate a decoupled feature matrix; S126, dynamically calibrating the decoupling characteristic matrix using a recursive least squares algorithm to generate a calibration characteristic matrix; performing a nonlinear transformation on the calibration characteristic matrix using a hyperbolic tangent function to obtain a standardized kinetic parameter matrix; Step S14 is further as follows: S141, arranging the comprehensive capability indicators of the UAVs in the order of the UAV numbers to generate a capability indicator sequence; smoothing the capability indicator sequence using a sliding mean method to obtain a smoothed capability indicator; S142, calculating the probability density distribution of the smoothed capability index to generate a capability distribution vector; based on the capability distribution vector, using a kernel density estimation method to calculate the capability difference between any two UAVs to obtain an initial difference matrix; S143, based on the initial difference matrix, calculating the second norm of the difference between the smoothed capability indicators of any two UAVs to generate a capability difference matrix; Reading normalization parameters from a preset parameter database, and normalizing the capability difference matrix based on the normalization parameters to obtain a normalized difference matrix; S144, performing weighted fusion on the initial difference matrix and the normalized difference matrix to generate a fused difference matrix; Perform adaptive threshold segmentation on each element in the fusion difference matrix to obtain a segmentation difference matrix; S145, based on the segmentation difference matrix, at each matrix element position, the corresponding difference value is divided by the maximum value of the smoothing capability index of the corresponding two UAVs to generate a heterogeneity numerical matrix; the exponential weighted average method is used to perform time series smoothing on the heterogeneity numerical matrix to obtain a smoothed heterogeneity matrix; S146. Adopting an adaptive symmetry processing algorithm to optimize the symmetry of the smooth heterogeneity matrix, ensuring the symmetry of the matrix, and generating a symmetric heterogeneity matrix; using the symmetric heterogeneity matrix as the final cluster heterogeneity matrix.

2. The MAPPO-based UAV cluster control and navigation method according to claim 1, characterized in that: Step S2 is further as follows: S21, collecting real-time communication data including signal strength, packet loss rate and delay time from the communication module; multiplying the signal strength of the real-time communication data by a first preset weight, multiplying the packet loss rate by the second preset weight after taking the inverse, and multiplying the delay time by the third preset weight after taking the inverse, and adding them to obtain a communication quality index characterizing the communication performance; S22, reading the link reliability data of the previous moment from the historical database; multiplying the communication quality index of the current moment by a preset dynamic smoothing coefficient, and adding the complement of the link reliability data of the previous moment multiplied by the preset smoothing coefficient to obtain the updated link reliability data; S23, calculating the weighted combination of the updated link reliability data and the corresponding elements of the cluster heterogeneity matrix to generate edge weight data representing the weight of the communication link; Based on edge weight data, a dynamic communication graph is constructed; S24. Based on the dynamic communication graph, and under the conditions of satisfying preset connectivity constraints and bandwidth constraints, a spanning tree is calculated that maximizes the sum of edge weight data to obtain an optimal communication topology.

3. The MAPPO-based UAV cluster control and navigation method according to claim 2 is characterized in that: Step S3 is further as follows: S31, combining the cluster heterogeneity matrix with the optimal communication topology structure, calculating the fitness of each UAV to perform various tasks, and generating a task allocation fitness matrix; S32, obtaining the current state, constructing a task allocation strategy model based on the current state, the task allocation fitness matrix and the cluster heterogeneity matrix, and generating a top-level strategy function; constructing an execution strategy model based on the current state and the standardized dynamic parameter matrix, and generating a bottom-level strategy function; combining the top-level strategy function and the bottom-level strategy function to form a hierarchical strategy model; S33. Based on the hierarchical strategy model, calculate the expected cumulative return based on task allocation rewards to generate a top-level value function for evaluating the global synergy effect; based on the hierarchical strategy model, calculate the expected cumulative return based on execution rewards to generate a bottom-level value function for evaluating the local execution effect; combine the top-level value function and the bottom-level value function to form a hierarchical value function.

4. The MAPPO-based UAV cluster control and navigation method according to claim 3 is characterized in that: Step S4 is further as follows: S41. Based on the hierarchical value function, construct an importance calculation function; calculate the timeliness value of the real-time communication data, the relevance value to the current task, and the communication cost value required for transmission; substitute the timeliness value, the relevance value, and the communication cost value into the importance calculation function to generate an information importance index; S42. Constructing a priority calculation function based on the hierarchical value function; Substituting the information importance index and the communication quality index into the priority calculation function to generate transmission priority data representing the transmission priority order; S43, based on the transmission priority data, calculating the communication bandwidth allocation ratio between the drones and generating a bandwidth allocation matrix; S44, constructing an adaptive routing strategy for determining a data transmission path based on a bandwidth allocation matrix and an optimal communication topology; The adaptive routing policy is stored in the routing policy database.

5. The MAPPO-based UAV cluster control and navigation method according to claim 4 is characterized in that: Step S5 is further as follows: S51, obtaining the output probability distribution of the hierarchical strategy model, calculating the entropy value of the probability distribution, and generating an uncertainty index representing the uncertainty of the strategy; S52. Based on the uncertainty index, construct the robustness constraint condition for limiting the update step of the strategy; S53. Under the premise of satisfying the robustness constraint condition, optimizing the expected cumulative reward of the hierarchical strategy model to obtain an optimized strategy model; S54, building a communication relationship based on the optimal communication topology; Based on the communication relationship and adaptive routing strategy, data transmission is carried out so that each UAV executes the decision action corresponding to the optimization strategy model according to the pre-stored local observation data and the data from the communication neighbors.

6. The MAPPO-based UAV cluster control and navigation method according to claim 5, characterized in that: Step S22 is further as follows: S221, read the link reliability data of the previous N moments from the historical database; use the wavelet decomposition algorithm to perform multi-scale analysis on the link reliability data, extract the trend component and the fluctuation component, and generate a multi-scale feature matrix; Based on the multi-scale feature matrix, an adaptive threshold detection algorithm is used to identify outliers and obtain cleaned historical data; where N is a natural number greater than 1; S222, using a Kalman filter to suppress noise on the communication quality indicator and generate a filtering quality indicator; constructing a short-term predictor based on an ARIMA model, performing a prediction analysis on the filtering quality indicator and obtaining a predicted quality indicator; S223. Based on the prediction quality index, the statistical characteristics of the cleaned historical data are calculated, including the mean, variance and autocorrelation coefficient, to generate a statistical characteristic vector; based on the statistical characteristic vector, the dynamic smoothing coefficient is estimated using the Bayesian reasoning method to obtain a smoothing coefficient matrix; S224, extracting the main features of the smoothing coefficient matrix by using the principal component analysis method to generate a characteristic coefficient vector; optimizing and adjusting the characteristic coefficient vector by using a fuzzy logic controller to obtain an optimized smoothing coefficient; S225. Based on the prediction quality index and the cleaned historical data, an integrated learning model is constructed to generate an integrated prediction result; a confidence interval of the integrated prediction result is calculated to obtain a credibility vector; S226. Based on the credibility vector, dynamically adjust the optimized smoothing coefficient to generate a final smoothing coefficient; perform weighted combination of the integrated prediction result and the communication quality index at the current moment through the final smoothing coefficient to obtain updated link reliability data.

7. The MAPPO-based UAV cluster control and navigation method according to claim 5, characterized in that: Step S32 is further as follows: S321, based on the task allocation fitness matrix, using the hierarchical analysis method, calculate the priority relationship between tasks and generate a task priority vector; based on the task priority vector, perform weighted processing on the task allocation fitness matrix to obtain a weighted fitness matrix; S322, read the current environment state data, perform feature fusion on the environment state data and the cluster heterogeneity matrix, and generate a state feature vector; use an adaptive feature selection algorithm to reduce the dimension of the state feature vector to obtain a reduced dimension feature vector; S323, construct a multi-head attention network, use the weighted fitness matrix as the query matrix, use the reduced dimension feature vector as the key value matrix, and generate an attention score matrix; Based on the attention score matrix, a task allocation strategy network is constructed to obtain the top-level strategy function. S324, extracting time series features from the standardized kinetic parameter matrix using a recursive neural network to generate a kinetic feature sequence; applying an adaptive gating mechanism to the kinetic feature sequence to obtain a gated feature sequence; S325, using a deep deterministic policy gradient algorithm, inputting the gated feature sequence and the dimension-reduced feature vector into the action generation network to generate an action policy function; and using the action policy function as the underlying policy function; S326, constructing a hierarchical connection structure, fusing the output of the top-level strategy function with the intermediate layer features of the bottom-level strategy function through an attention mechanism, and generating a fusion strategy representation; Based on the fusion strategy representation, a complete strategy network system is constructed to obtain a hierarchical strategy model.

Images