An unmanned aerial vehicle cluster cooperative planning method and system based on mean field game

Abstract

This invention discloses a method and system for cooperative planning of UAV swarms based on mean-field game theory, relating to the field of multi-agent learning technology. The method includes: acquiring the original state information of the UAV swarm, performing standardization, feature extraction, and dimension mapping to obtain a set of swarm state features; estimating the mean field of the swarm state feature set to obtain mean-field distribution information and swarm statistics; modeling the mean-field distribution information and swarm statistics using mean-field game theory to obtain a mean-field game model; solving the mean-field game model for optimal feedback to obtain an initial optimal feedback strategy; performing cooperative incentive compatibility correction on the initial optimal feedback strategy to obtain an incentive-corrected cooperative strategy; performing policy sharing feedback processing on the incentive-corrected cooperative strategy to obtain a shared feedback cooperative strategy; and combining real-time cooperative deviation information to perform adaptive closed-loop adjustment of the shared feedback cooperative strategy to obtain the final cooperative planning instruction.

Description

Technical Field

[0001] This invention relates to the field of multi-agent learning technology, and more specifically to a method and system for collaborative planning of unmanned aerial vehicle (UAV) swarms based on mean-field game theory. Background Technology

[0002] Against the backdrop of rapid development in technologies such as UAV swarm collaborative planning, multi-agent system control, and distributed intelligent decision-making, swarm sizes are constantly expanding and task types are becoming increasingly complex, placing higher demands on the system's autonomous decision-making capabilities and collaborative efficiency in dynamic environments. Traditional methods often employ centralized optimization or rule-based distributed control strategies, achieving collaborative planning through preset paths, fixed control parameters, or local information exchange. However, these methods typically rely on global information or strong assumptions, making them difficult to apply effectively in situations with limited communication, incomplete information, or dynamically changing environments, and are prone to problems such as high computational complexity, slow response, and insufficient system stability. Furthermore, as the complexity of multi-agent collaborative problems increases, aggregation game and two-level game models are gradually becoming important research directions. These models introduce aggregation variables to characterize group behavior, making individual decisions dependent on overall statistical information, thereby reducing the information dimensionality and supporting distributed solutions.

[0003] However, in practical engineering applications, aggregate variables are usually determined by the inner optimization problem, and it is difficult for individuals to directly obtain their precise expressions, leading to complex gradient calculations, difficulties in policy updates, and challenges in ensuring collaborative consistency. Furthermore, existing methods often focus on the equilibrium solution process, lacking systematic modeling of collaborative incentive mechanisms, policy consistency adjustment, and closed-loop stability, making it difficult to support the long-term stable operation of UAV swarms in complex dynamic environments.

[0004] Therefore, there is an urgent need for a collaborative planning method for UAV swarms that can achieve efficient distributed collaborative decision-making under information-constrained conditions, while taking into account solvability, stability, and engineering feasibility. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a method and system for collaborative planning of UAV swarms based on mean field game theory to overcome or at least partially solve the above problems. It aims to solve the technical problem of achieving consistency between individual decision-making and swarm collaborative goals of multiple UAVs under conditions of incomplete information and limited communication.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a method for collaborative planning of unmanned aerial vehicle (UAV) swarms based on mean-field game theory, comprising: Step 1: Obtain the original state information of the drone swarm, perform standardization processing, feature extraction and dimension mapping to obtain a set of swarm state features; Step 2: Perform mean field estimation on the set of population state characteristics to obtain mean field distribution information and population statistics; Step 3: Perform mean-field game modeling on the mean-field distribution information and population statistics to obtain the mean-field game model; Step 4: Solve the optimal feedback problem for the mean-field game model to obtain the initial optimal feedback strategy; Step 5: Perform cooperative incentive compatibility correction on the initial optimal feedback strategy to obtain the incentive-corrected cooperative strategy; Step 6: Perform policy sharing feedback processing on the incentive-modified cooperative strategy to obtain a shared feedback cooperative strategy; Step 7: Combine real-time collaborative deviation information to perform adaptive closed-loop adjustment on the shared feedback collaborative strategy to obtain the final collaborative planning instruction.

[0008] Preferably, the mean-field game model is constructed in the following manner: Based on the mean field distribution information and group statistics, a typical UAV's state evolution equation, stage cost function, and terminal cost function are defined. The stage cost function includes at least a control cost term, a mission tracking cost term, a formation coordination cost term, a safety collision avoidance cost term, and a coordination gain term. The terminal cost function is determined after the stage cost function based on the terminal target position, the expected terminal distribution, and the mission completion deviation, and serves as the boundary condition for the subsequent Hamilton-Jacobi-Bellman equation at the terminal moment. Define a population distribution evolution equation that describes the overall behavior evolution of the cluster; By coupling the state evolution equation, stage cost function, terminal cost function, and population distribution evolution equation, a mean-field game model composed of Hamilton-Jacobi-Bellman equation and Fock-Planck-Kolmogorov equation is obtained.

[0009] Preferably, the optimal feedback solution for the mean-field game model specifically includes: Given the mean field distribution information and the terminal cost function, the terminal cost function is used as the boundary condition of the Hamilton-Jacobi-Bellman equation at the terminal time. The Hamilton-Jacobi-Bellman equation is then solved to obtain the value function and its gradient. The optimal feedback control law is generated based on the value function gradient and the state evolution equation, and the optimal feedback control law is used as the control law expression of the initial optimal feedback strategy. The optimal feedback control law is substituted into the Fock-Planck-Kolmogorov equation to update the mean field distribution; Repeatedly perform value function solving, optimal feedback control law generation, and mean field distribution update until the distribution residual is less than a preset threshold, to obtain the initial optimal feedback strategy jointly characterized by the optimal feedback control law and its corresponding mean field distribution.

[0010] Preferably, the initial optimal feedback strategy is modified to be compatible with the incentive mechanism, specifically including: Based on the optimal value function corresponding to the initial optimal feedback strategy, a collaborative benefit function and a deviation penalty term are introduced to construct the modified optimal value function and the modified Hamiltonian. The deviation between the individual position and the expected position of the group, the deviation between the individual control quantity and the average control quantity of the group, and the deviation between the individual task contribution and the average task contribution of the group are calculated. The calculated deviation is used to correct the individual comprehensive cost function, subjecting individuals that deviate from the group's collaborative goal to a higher cost constraint. An incentive-corrected collaborative strategy is then generated based on the gradient of the corrected optimal value function or the corrected Hamiltonian. The individual comprehensive cost function includes a stage cost function and a collaborative incentive term, and is not just the terminal cost function. The incentive-corrected collaborative strategy is a control strategy derived from the corrected optimal value function, and is not the corrected optimal value function itself.

[0011] Preferably, the policy sharing feedback processing of the incentive-modified cooperative policy specifically includes: The incentive-modified cooperative strategy is compressed into a low-dimensional summary, and the group average direction, average velocity, task area density, local risk density and energy distribution summary are extracted to obtain the strategy sharing summary. The strategy sharing summary is used as a shared feedback quantity and sent to each UAV via broadcast or neighborhood communication. It is then embedded into the individual control law to form a shared feedback cooperative strategy with individual state, mean field distribution and strategy sharing summary as inputs.

[0012] Preferably, the shared feedback coordination strategy is adaptively adjusted in a closed-loop manner by incorporating real-time coordination deviation information, specifically including: Based on the shared feedback collaboration strategy, the group distribution deviation, task completion deviation, collision risk deviation, and strategy consistency deviation are obtained as real-time collaboration deviation information. An adaptive stage cost function is constructed based on the real-time collaborative deviation information, and the collaborative incentive strength and shared feedback weight are updated according to a preset adaptive update rate. A collaborative feedback control law is generated based on the updated collaborative incentive strength and shared feedback weight, and the stability constraint of the collaborative feedback control law is verified. When the preset stability condition is met, the final collaborative planning instruction is output.

[0013] Preferably, the original state information includes the UAV's number, location, speed, heading angle, remaining energy, mission payload, communication adjacency, obstacle distance, target area information, and environmental disturbance information.

[0014] Secondly, embodiments of the present invention provide a drone swarm collaborative planning system based on mean-field game theory, comprising: The information acquisition and processing module is used to acquire the raw state information of the UAV cluster, perform standardization processing, feature extraction and dimension mapping to obtain a set of group state features; The mean field estimation module is used to perform mean field estimation on the set of population state features to obtain mean field distribution information and population statistics. The game modeling module is used to perform mean-field game modeling on the mean-field distribution information and group statistics, and to construct a mean-field game model. The strategy solving module is used to solve the optimal feedback for the mean field game model and obtain the initial optimal feedback strategy. The incentive correction module is used to perform cooperative incentive compatibility correction on the initial optimal feedback strategy to obtain the incentive-corrected cooperative strategy. The shared feedback module is used to perform strategy sharing feedback processing on the incentive-corrected collaborative strategy to obtain a shared feedback collaborative strategy. An adaptive adjustment module is used to adaptively adjust the shared feedback coordination strategy by combining real-time coordination deviation information to obtain the final coordination planning instruction.

[0015] Thirdly, embodiments of the present invention provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for collaborative planning of unmanned aerial vehicle (UAV) swarms based on mean-field game theory.

[0016] Fourthly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a method for collaborative planning of unmanned aerial vehicle (UAV) swarms based on mean-field game theory.

[0017] As can be seen from the above technical solutions, compared with the prior art, this invention discloses a method and system for collaborative planning of UAV swarms based on mean-field game theory. This invention achieves distributed collaborative planning for large-scale UAV swarms without relying on global information, effectively reducing the computational complexity and communication burden of traditional centralized methods, and significantly improving the system's scalability and real-time response capabilities. By introducing mean-field game modeling, the high-dimensional coupling problem of multiple agents is transformed into a low-dimensional decision-making problem based on swarm distribution, enabling the system to maintain stable computational performance even when scaling up. Simultaneously, this invention establishes a consistent connection between individual decisions and swarm goals by constructing a two-layer game structure and a collaborative optimization mechanism, theoretically guaranteeing the existence and uniqueness of equilibrium solutions, thereby avoiding multiple solutions or oscillation problems and improving system stability. Through a collaborative incentive compatibility mechanism, individuals automatically converge towards the swarm's collaborative goal while pursuing their own gains, effectively resolving the conflict between individual rationality and overall optimality in traditional methods; through a strategy sharing feedback mechanism, efficient transmission and consistency adjustment of swarm information are achieved under limited communication conditions, improving the system's collaborative capabilities in scenarios with incomplete information. Furthermore, by introducing an adaptive adjustment mechanism, the system can dynamically adjust control parameters according to environmental changes, task requirements, and execution deviations, effectively suppressing external disturbances and uncertainties, and enhancing system robustness and stability. In summary, this invention can significantly improve the collaborative efficiency, planning accuracy, and operational stability of UAV swarms in complex dynamic tasks, demonstrating good engineering application value and promising prospects for widespread adoption. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a flowchart of a drone swarm collaborative planning method based on mean-field game theory provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a drone swarm collaborative planning system based on mean field game theory provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention discloses a method for collaborative planning of unmanned aerial vehicle (UAV) swarms based on mean-field game theory, such as... Figure 1 As shown, it includes: Step 1: Obtain the original state information of the drone cluster, and standardize the original state information to obtain standardized state information.

[0022] Specifically, the number, position, speed, heading angle, remaining energy, mission payload, communication adjacency, obstacle distance, target area information, and environmental disturbance information of each UAV are obtained and used as the raw state information. The raw state information is then processed by timestamp alignment, coordinate system transformation, outlier removal, noise filtering, and normalization to obtain standardized state information.

[0023] Standardized state information is subjected to state feature extraction and dimension mapping to obtain a set of group state features. Specifically, position features, velocity features, heading features, remaining energy features, task load features, neighborhood risk features, and communication reachability features are extracted from the standardized state information, and mapped according to a unified state space partitioning rule to obtain a set of group state features.

[0024] Step 2: Perform mean field estimation on the set of population state characteristics to obtain mean field distribution information and population statistics.

[0025] Specifically, the distribution density of the cluster in the state space is calculated by kernel density estimation, state grid statistics, or histogram estimation of the group state feature set to obtain the mean field distribution information; at the same time, the average position, average velocity, average control quantity, task density, risk density, and energy distribution statistics of the group are calculated based on the mean field distribution information to obtain the group statistics.

[0026] Step 3: Perform mean-field game modeling on the mean-field distribution information and population statistics to obtain the mean-field game model.

[0027] Specifically, based on mean field distribution information and swarm statistics, a typical UAV's state evolution equation, stage cost function, terminal cost function, and swarm distribution evolution equation are constructed. The stage cost function characterizes control energy consumption, mission tracking, formation coordination, collision avoidance, and cooperative benefits during the planning process. The terminal cost function characterizes the terminal target position, desired terminal distribution, and mission completion deviation, and serves as the boundary condition V(T,x) for the HJB equation at the terminal moment. Coupled with the above equations and cost functions, a mean field game model is obtained, consisting of the Hamilton-Jacobi-Bellman equation (HJB) and the Focke-Planck-Kolmogorov equation (FPK).

[0028] Step 4: Solve for the optimal feedback in the mean-field game model to obtain the initial optimal feedback strategy.

[0029] Specifically, given the mean field distribution information, the terminal cost function is used as the terminal boundary condition of the HJB optimality equation to solve the HJB optimality equation, obtaining the value function and its gradient. The optimal feedback control law is generated based on the gradient of the value function, and this optimal feedback control law is used as the specific control law expression of the initial optimal feedback strategy. The optimal feedback control law is then substituted into the FPK population distribution evolution equation to update the mean field distribution. The value function solution, optimal feedback control law generation, and distribution update are repeated until the distribution residual is less than a preset threshold, thus obtaining the initial optimal feedback strategy.

[0030] Step 5: Perform cooperative incentive compatibility correction on the initial optimal feedback strategy to obtain the incentive-corrected cooperative strategy.

[0031] Specifically, based on the optimal value function corresponding to the initial optimal feedback strategy, a collaborative reward function and a deviation penalty term are introduced to construct a modified optimal value function and a modified Hamiltonian. The deviations between the individual's position and the group's desired position, the individual's control quantity and the group's average control quantity, and the individual's task contribution and the group's average task contribution are calculated. Based on these deviations, the individual comprehensive cost function is modified, subjecting individuals deviating from the group's collaborative goal to higher cost constraints. An incentive-modified collaborative strategy is then generated based on the gradient of the modified optimal value function or the modified Hamiltonian. Note that the incentive-modified collaborative strategy is a control strategy derived from the modified optimal value function, not the modified optimal value function itself; the individual comprehensive cost function includes a stage cost function and a collaborative incentive term, not just the terminal cost function.

[0032] Step 6: Perform policy sharing feedback processing on the incentive-modified collaborative strategy to obtain the shared feedback collaborative strategy.

[0033] Specifically, the incentive-modified cooperative strategy is compressed into a low-dimensional summary, extracting summaries of the group's average direction, average velocity, task area density, local risk density, and energy distribution to obtain a shared strategy summary. This shared strategy summary is then transmitted to each UAV as a shared feedback quantity via broadcast or neighborhood communication and embedded into the individual control law. This enables each UAV to generate a shared feedback cooperative strategy with its individual state, average field distribution, and the shared strategy summary as input. Therefore, the shared feedback cooperative strategy is the feedback control strategy incorporating the shared strategy summary.

[0034] Step 7: Adaptively adjust the shared feedback collaboration strategy by combining real-time collaboration deviation information to obtain the final collaboration planning instruction.

[0035] Specifically, based on the shared feedback coordination strategy, the system acquires real-time coordination deviation information, including group distribution deviation, task completion deviation, collision risk deviation, and strategy consistency deviation. An adaptive stage cost function is constructed based on this real-time coordination deviation information, with the coordination incentive strength and shared feedback weights used as updatable parameters. These parameters are then updated according to a preset adaptive update rate. A coordination feedback control law is generated based on the updated coordination incentive strength and shared feedback weights, and its stability is verified by constraints. When the Lyapunov stability condition or preset safety constraints are met, the final coordination planning instruction is output. This instruction includes the waypoints, speed commands, heading adjustments, formation maintenance commands, task allocation results, and feedback adjustment parameters for each UAV.

[0036] This invention addresses the challenges of large-scale UAV swarm collaborative planning, including a large number of individuals, complex interactions, heavy global solution burden, and difficulty in uniformly characterizing collaborative behavior. It proposes a modeling framework for UAV swarm collaborative planning based on mean-field game theory. Instead of modeling the explicit interactions between any two UAVs individually, this framework constructs the coupling relationships between typical individual states, swarm distribution states, and swarm statistics. This transforms the large-scale swarm planning problem into a closed-loop optimization problem involving the interaction between optimal individual responses and mean-field evolution, thus providing a unified formal basis for collaborative decision-making, distribution evolution, and dynamic adjustment of UAV swarms.

[0037] Suppose that the drone swarm consists of a large number of homogeneous or nearly homogeneous individuals, and is described in continuous time. Let... Indicates the planning time interval. Definition This indicates that a typical drone is at a certain time. The state vector is used to describe information such as its position, velocity, heading, and remaining energy.

[0038] This indicates that the drone is at a certain time. The control strategy is used to describe speed commands, heading adjustments, or thrust adjustments.

[0039] Indicates time The distribution density of the cluster in the state space is used to characterize the average field distribution state of the entire drone cluster.

[0040] Indicates the distribution of groups The induced population statistics, among which It is a state mapping function, which can be defined as a statistical mapping such as location, energy or task load according to task requirements.

[0041] This represents the stage cost function of a typical UAV, used to characterize the control costs, coordination bias, safety costs, and mission completion status of a single UAV in its current state.

[0042] Indicates at time Status is The optimal value function at that time.

[0043] This represents a feedback strategy based on individual state and mean field distribution.

[0044] Based on this, the entire drone swarm collaborative planning process can be uniformly described as the following closed-loop model:

[0045] in, Represents the state evolution function of the UAV; Represents the disturbance intensity matrix; It represents standard Brownian motion and is used to characterize random effects such as wind disturbance and perceived noise. The terminal cost is used to describe terminal formation error, area coverage error, or mission completion deviation. As can be seen from the unified model described above, the individual UAV in this invention is no longer directly coupled to all other individuals in the cluster, but rather through mean field distribution information. and its group statistics It is associated with group behavior. This retains the distributed characteristics of individual autonomous decision-making, while also reflecting the overall collaborative trend of the cluster through mean field variables, thus forming a closed-loop planning system of "individual state - group distribution - control strategy - distribution feedback".

[0046] In this embodiment, based on the above-mentioned overall closed-loop model, the present invention further constructs an average-field game modeling layer for collaborative planning of UAV swarms. The core of this modeling layer is to couple the local decision-making behavior of typical UAVs in the swarm with the average field distribution of the overall swarm through a unified cost function, so that each individual can perceive and respond to the group formation state, task allocation state, and security risk state while considering its own control costs, thereby realizing collaborative planning modeling for large-scale swarms.

[0047] This invention will use typical drones at any time The stage cost function is defined as:

[0048] in: To control the weight matrix; Indicates the position components of the drone; Indicates the average position of the group; This represents the mapping from state to task output; Indicates time Task reference data, such as the center of the target area, coverage curve, or inspection trajectory; This represents the safety potential function, used to characterize the coupling relationship associated with high-density areas, obstacles, or collision risks. This represents the collaborative benefit function, used to characterize the positive contribution of shared coverage, collaborative search, or formation consistency. These are the weighting coefficients for each item.

[0049] The aforementioned stage cost function uniformly considers multiple factors such as control energy consumption, formation maintenance, mission tracking, safe collision avoidance, and collaborative benefits, thus organizing the UAV swarm collaborative planning problem into a typical mean-field game optimization problem. In this problem, individuals no longer make decisions by "playing against all other individuals one by one," but rather through distribution... Coupled with the average behavior of the group, this allows the model to naturally adapt to large-scale cluster scenarios.

[0050] Furthermore, to characterize the coupling relationship between individual optimal responses and population distribution evolution, this invention introduces the following mean-field game optimality system:

[0051] The first equation is the Hamilton-Jacobi-Bellman equation, used to describe the evolution of the optimal value function for a typical UAV under the current swarm distribution; the second equation is the Fock-Planck-Kolmogorov equation, used to describe the dynamic evolution of the swarm distribution under the action of the optimal feedback strategy. To maintain consistency of signs, variables that change with time in the equations are all represented by the subscript t or time parameters, where... Let be the average field distribution density at time t. For the reason Induced population statistics Let be the candidate control variable at time t. Let be the optimal feedback strategy at time t. t represents the partial derivative with respect to time t. x represents the gradient with respect to state x, Tr represents the matrix trace operation, and Σ represents the perturbation intensity matrix. These two elements are coupled together to form the mean-field modeling layer for collaborative planning of UAV swarms.

[0052] In this embodiment, after completing the mean-field game modeling, the present invention further constructs a cooperative strategy optimization layer to solve the problem given a population distribution. Under what conditions should a typical UAV adopt a feedback control strategy to achieve collaborative planning for the overall goal of the swarm while satisfying individual constraints?

[0053] This invention defines the optimal feedback control law as:

[0054] This equation shows that, at any time t, the optimal control u of a typical UAV in state x is jointly determined by the control cost term and the state value gradient term. Wherein, V(t,x) represents the gradient of the value function with respect to the state, used to characterize the sensitive direction of the current state's evolution toward the optimal goal; population statistics Through state evolution function Entering the control law; the terminal cost function, as the terminal boundary condition of the HJB equation, participates in the inverse solution of the value function V(t,x). Therefore, the optimal feedback strategy is essentially a control result formed by the joint coupling of "terminal target constraints, individual states, value function gradients and population statistics".

[0055] If we further assume that the state dynamics are affine with respect to the control input, that is...

[0056] The optimal feedback control law can then be further written in explicit form:

[0057] This formula illustrates that the control input of a typical UAV is adjusted along the descent direction indicated by the gradient of the value function and is jointly constrained by the control weight matrix R and the input mapping matrix B(x). Therefore, this invention transforms UAV swarm cooperative planning from a static trajectory setting problem into a dynamic optimal feedback planning problem, enabling each UAV to generate a cooperative control strategy in real time based on its own state and the swarm's average field distribution.

[0058] To further reflect the terminal objective of collaborative planning, this invention defines the terminal cost as:

[0059] Where: ρ1 and ρ2 represent the terminal position error weight and the terminal distribution error weight, respectively; Indicates the individual's position at time T; Indicates the target location of the terminal; This represents the population distribution at terminal time T; Indicates the desired terminal distribution; The distance represents the distribution, which can be a Wasserstein-type distance or other distribution metrics. This terminal cost function serves as the boundary condition for the HJB equation at the terminal time, used to pass the terminal formation, area coverage, and mission completion requirements to the optimal feedback strategy solution process.

[0060] The terminal cost simultaneously constrains the location of individual terminals and the distribution of group terminals, enabling the present invention to ensure that each drone completes its local task objective in the collaborative strategy optimization layer, while also ensuring that the entire drone swarm achieves the desired overall formation or coverage distribution at the terminal moment.

[0061] In this embodiment, in the collaborative planning of UAV swarms, relying solely on the mean field cost function and optimal feedback control can achieve a certain degree of group coordination. However, under complex task conditions, individuals may still tend to prioritize minimizing their own local costs, thus deviating from the overall task objective of the swarm. To address this, the present invention further proposes a collaborative game incentive compatibility mechanism. By explicitly introducing collaborative incentive terms into the individual optimization objectives, the optimal strategies of individuals tend to align with the group's collaborative objectives.

[0062] This invention defines the collaborative benefit function as:

[0063] in: This represents the expected center location of a task-oriented group. Indicates average control of the group; This represents the contribution function of an individual task, such as coverage contribution, communication contribution, or reconnaissance contribution. Indicates the average task contribution of the group; is the excitation coupling coefficient.

[0064] Based on this, the corrected optimal value function for a typical UAV can be further written as:

[0065] The purpose of this structure is to reduce the effective cost when the position, control behavior, and task contribution of an individual UAV are closer to the requirements of the swarm task; conversely, the effective cost increases when the individual behavior deviates significantly from the group's collaborative goal. Thus, the optimal response of an individual is guided towards a direction more conducive to group collaboration, thereby achieving consistency between individual rational decision-making and the swarm's collaborative planning goals.

[0066] Furthermore, to analyze the stability of the incentive-compatible mechanism, the modified Hamiltonian can be defined as:

[0067] If there exists η>0 such that the corrected Hamiltonian... Regarding control variables If strictly convex, then the cooperative strategy with modified incentives derived from the modified optimal value function is unique, and individual control deviates from the group average control. The degree of suppression is increased by η. Therefore, this invention enables the local optimal behavior of individual UAVs to converge mathematically to the collective cooperative goal through a cooperative game incentive compatibility mechanism.

[0068] In this embodiment, in a large-scale UAV swarm scenario, requiring each UAV to obtain complete status information of all other UAVs in real time would result in a high communication and computational burden, which is not conducive to engineering implementation. Therefore, this invention proposes a collaborative strategy sharing and feedback mechanism. By summarizing the swarm's strategic behavior in a low dimension and broadcasting it, individuals can perceive the swarm's collaborative trends and adjust their own decisions without needing to know the detailed status of all other UAVs.

[0069] This invention defines a sharing strategy summary as follows:

[0070] in, This represents a strategy summary mapping, which can be used to extract summary information such as population average direction, average velocity, energy distribution summary, task area density, and local risk density.

[0071] After obtaining the summary of the sharing strategy Subsequently, the control strategy for individual drones was further described as follows:

[0072] Therefore, each drone, when generating its own control input, depends not only on its own state. With mean field distribution Furthermore, a shared feedback quantity s_t, which is compressed from the overall behavior of the group, is also introduced. Therefore, This is the specific expression of the shared feedback collaboration strategy described in this invention. This approach maintains the low-dimensional modeling characteristics of the mean field while enhancing the individual's perception of the current group's collaborative behavior.

[0073] Furthermore, with the participation of shared feedback, the evolution of the group distribution satisfies the following form:

[0074] This formula shows that the evolution of population distribution is not only determined by mean-field statistics. It is also affected by the sharing strategy summary. Feedback adjustment. Through this shared feedback mechanism, the present invention can ensure that individual control behavior is consistent with the group's collaborative trend without significantly increasing communication complexity, thereby improving the overall stability and real-time performance of UAV swarm collaborative planning.

[0075] In this embodiment, based on the above-mentioned cooperative game incentive compatibility mechanism and cooperative strategy sharing feedback mechanism, the present invention further proposes a cooperative adaptive adjustment mechanism, which is used to adaptively adjust the cooperative incentive intensity and sharing feedback intensity according to the real-time changes in the cluster cooperative state, so that the system can maintain better cooperative planning performance under different task stages, different group distribution states and different disturbance conditions.

[0076] This invention introduces two adaptive coefficients: This represents the intensity of collaborative incentives at time t, used to adjust the impact of the collaborative benefit term on the individual's overall cost function; This represents the shared feedback strength at time t, used to adjust the amount of shared feedback. The strength of the constraint on the individual control law. Based on this, the stage cost is extended to:

[0077]

[0078] in, This represents a summary of the individual's policy extracted from the individual's state and control variables. This represents the policy-shared summary obtained from group policy compression. This value is used to characterize the degree of deviation between an individual's strategy summary and the group's shared summary. The larger the value, the more obvious the deviation between the individual's behavior and the cluster's sharing trend, and the stronger the feedback constraint will be.

[0079] To achieve dynamic adjustment of the intensity of collaborative incentives and the intensity of shared feedback, this invention defines the following adaptive update law:

[0080] in: Indicates the level of collaborative contribution to the objectives; Indicates the average task contribution of the group; Indicates individual task contribution; and Let represent the adaptive update rates of the collaborative incentive strength and the shared feedback strength, respectively. , >0; Δt represents the update step size. The above update law increases the cooperative excitation strength. and shared feedback intensity It can dynamically change with real-time collaborative deviation information.

[0081] Therefore, the present invention ultimately yields the following cooperative feedback control law:

[0082] This collaborative feedback control law integrates the stage cost function J, the optimal value function V, and the mean field distribution in the mean field game modeling layer. The collaborative benefit function in the collaborative incentive compatibility mechanism and its synergistic incentive strength And the policy summary operator Γ and shared feedback quantity in the policy sharing feedback mechanism. and shared feedback intensity All are incorporated into the same optimization framework. In this way, the present invention forms a complete closed-loop collaborative planning mechanism of "mean field modeling - optimal feedback planning - collaborative incentive correction - shared feedback constraints - adaptive closed-loop adjustment".

[0083] To demonstrate the theoretical reliability and feasibility of the proposed technical route of "average-field game modeling—cooperative strategy optimization—incentive compatibility correction—shared feedback constraints—adaptive adjustment," a formal analysis of the constructed closed-loop system is presented below. To maintain notation consistency, the aforementioned definitions are still used: This indicates a typical drone status. Indicates control input, Indicates population distribution density. Represents group statistics. Represents a value function. Represents the collaborative benefit function. Represents a summary of the sharing strategy. These represent the intensity of collaborative incentives and the intensity of shared feedback, respectively.

[0084] (I) Basic Assumptions To facilitate a rigorous analysis, the following assumptions are made: Assumption 1: State dynamics function about Continuous, and about and Local Lipschitz; perturbation matrix Bounded and continuous.

[0085] Assumption 2: Control Cost Matrix Stage cost function Regarding control variables Strictly convex, with respect to state variables and distribution variables Continuously differentiable.

[0086] Assumption 3: Collaborative Revenue Function Safety potential function and policy summary operator Bounded, continuous, and with respect to their respective variables Lipschitz.

[0087] Assuming 4 terminal costs about The second half is continuous and meets the conditions for moderate growth.

[0088] Assume that the mean-field coupling satisfies the monotonicity condition, that is, for any two distributions... ,

[0089] in This represents the cost term induced by distributed coupling. This type of monotonicity condition is one of the classic sufficient conditions for the uniqueness of equilibrium in mean-field games.

[0090] Under the above assumptions, the following conclusions hold true.

[0091] (ii) The well-determined nature of closed-loop systems and the uniqueness of individual state processes Lemma 1 For any given measurable policy and a given distribution trajectory stochastic differential equations

[0092] There exists a unique strong solution, and for any There exists a constant Make

[0093] prove Based on assumption 1, the closed-loop drift term Regarding state variables Local Lipschitz, diffusion term Since the equation is bounded and continuous, the closed-loop state equation satisfies the standard condition for the existence and uniqueness of strong solutions to stochastic differential equations. Using the Burkholder-Davis-Gundy inequality and the Grönwall inequality, the above equation can be obtained. Bounded moment estimation. Therefore, the state process of a typical UAV is well-definite given the mean field and policy.

[0094] This lemma demonstrates that the present invention is feasible at the level of "individual state evolution" and will not suffer from state divergence or non-unique solutions caused by control laws or mean field feedback.

[0095] (III) Existence and Uniqueness of Mean-Field Game Coupled Systems The mean-field game system of this invention is given by the following HJB-FPK coupling equation:

[0096] Theorem 1 states that, under assumptions 1-5, the above mean-field game system has at least one weak solution. If the stage cost is strictly convex with respect to control and the mean-field coupling satisfies the monotonicity condition, then the equilibrium solution is unique. This type of uniqueness conclusion is a standard result in mean-field game theory.

[0097] prove First fix the distribution trajectory Therefore, the HJB equation corresponds to an optimal control problem with a diffusion term. From Assumption 2... Due to the strict convexity of the control variable, the Hamiltonian has a unique minimum point with respect to the control variable, thus yielding the unique optimal feedback. Substituting this feedback into the FPK equation, we can obtain the result under a given condition. The distribution evolution under the given conditions. Therefore, a mapping is defined.

[0098] in It is by The distribution trajectory obtained under induced optimal feedback. This can be proven by hypotheses 1-4. Since the system is continuous and relatively compact in a suitable function space, Schauder's fixed-point theorem yields at least one set of fixed points, meaning the system has at least one set of equilibrium solutions. Furthermore, if the monotonicity condition in Assumption 5 is satisfied, then for any two sets of solutions... Considering the difference system, and performing paired integration of the HJB equation and the FPK equation, we can obtain...

[0099] Combining the monotonicity condition, we can only conclude that the integral is zero, therefore we have Furthermore, the uniqueness of the Bellman equation leads to... Therefore, the equilibrium solution is unique. This proof is consistent with the Lasry-Lions monotonicity uniqueness framework. Theorem 1 shows that the "game modeling layer + cooperative strategy optimization layer" of this invention is mathematically solvable and is not a heuristic system constructed solely based on experience.

[0100] (iv) Existence, uniqueness and explicit computability of the optimal feedback strategy In affine dynamics

[0101] The optimal feedback strategy is written as follows:

[0102] Proposition 1 in Under the given conditions, the optimal feedback strategy exists and is unique, and satisfies...

[0103] prove Define Hamiltonian

[0104] Depend on It can be known about It is strictly convex, therefore the unique minimum point is given by the first-order optimality condition:

[0105] Organize and obtain

[0106] Therefore, the proposition is true.

[0107] This proposition indicates that the cooperative strategy optimization layer in this invention not only "exists" but also has an explicit feedback form under common affine dynamics, thus possessing engineering solution feasibility.

[0108] (V) Analysis of incentive compatibility and synergy consistency of mechanism A In mechanism A, the collaborative benefit function is defined as follows:

[0109] The corrected individual optimization problem is:

[0110] Theorem 2 If Then the collaborative incentive term severely punishes the degree to which an individual deviates from the group's collaborative goal; if the modified Hamiltonian is related to If strictly convex, then the individual's optimal strategy shrinks towards the group's collaborative goal in all three dimensions: position, control, and task contribution.

[0111] prove Depend on As can be seen from the definition,

[0112] Therefore, the revised cost function includes three quadratic penalty terms related to the cooperative center, the average control of the group, and the average contribution of the group. For any state or control deviating from the cooperative center, the incremental cost is strictly positive. Thus, if an individual deviates from the group's cooperative goal, its effective cost monotonically increases; if it moves closer to the cooperative goal, its effective cost monotonically decreases. Furthermore, due to strict convexity, the minimum solution is unique; therefore, the individual's optimal response must shrink to a point closer to the cooperative goal. Therefore, the mechanism achieves consistency between "individual rational optimality" and "group collaborative optimality".

[0113] This invention provides a drone swarm cooperative planning system based on mean-field game theory, such as... Figure 2 ,include: The information acquisition and processing module is used to acquire the raw state information of the UAV cluster, perform standardization processing, feature extraction and dimension mapping to obtain a set of group state features; The mean field estimation module is used to estimate the mean field of the population state feature set to obtain mean field distribution information and population statistics. The game modeling module is used to perform mean-field game modeling on mean-field distribution information and population statistics, and to construct mean-field game models. The strategy solving module is used to solve the optimal feedback for the average field game model and obtain the initial optimal feedback strategy. The incentive correction module is used to perform cooperative incentive compatibility correction on the initial optimal feedback strategy to obtain the incentive-corrected cooperative strategy. The shared feedback module is used to perform strategy sharing feedback processing on the incentive-corrected collaborative strategy to obtain a shared feedback collaborative strategy. The adaptive adjustment module is used to combine real-time collaborative deviation information to adaptively adjust the shared feedback collaborative strategy in a closed loop, so as to obtain the final collaborative planning instruction.

[0114] Based on the same inventive concept, and since the principle by which the system solves the problem is similar to that of the aforementioned method, the implementation of this system can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0115] In the game modeling layer, this invention abstracts large-scale UAV swarms from discrete individual systems into continuous distributed systems. By introducing a population distribution density function and statistics to characterize the overall behavior, individual decisions are transformed from relying on the state of all individuals to relying on population distribution information, thereby achieving a dimensionality reduction expression of high-dimensional coupled problems. Furthermore, by constructing a coupled optimal control equation system, a unified description of individual behavior and population evolution is provided.

[0116] Secondly, at the collaborative strategy optimization layer, individual feedback control strategies are constructed based on optimality conditions, enabling the UAV's control input to be dynamically adjusted according to the group distribution and collaborative objectives, achieving an adaptive response of individual behavior to group evolution. Simultaneously, by constructing an optimization structure with convexity and continuity, the strategy solution process is guaranteed to have good computability and convergence.

[0117] Furthermore, at the collaborative mechanism design layer, a collaborative incentive compatibility mechanism and a strategy sharing feedback mechanism are introduced: by constructing a collaborative benefit function, the degree to which individual behavior deviates from the group goal is transformed into an explicit penalty term, so that individuals tend toward the group's collaborative optimum while pursuing their own optimal benefit; at the same time, group information is compressed through strategy summary and fed back to individuals in a distributed weighted form, so as to achieve distributed consistency adjustment under information-constrained conditions and reduce communication and computing overhead.

[0118] Finally, in the adaptive adjustment layer, a dynamic parameter update mechanism based on system state and cooperative deviation is constructed. This mechanism transforms group cooperative error and policy consistency error into driving signals, allowing for online adjustment of excitation intensity and feedback weights. Furthermore, a closed-loop optimization framework is built using Lyapunov stability analysis, enabling the system to achieve continuous evolution and stable convergence in a dynamic environment. Through this multi-layered cooperative design, a UAV swarm cooperative planning method that combines theoretical completeness with engineering feasibility is formed.

[0119] This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements a method for collaborative planning of drone swarms based on average field game theory.

[0120] This embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements a drone swarm collaborative planning method based on mean-field game theory.

[0121] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0123] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for collaborative planning of unmanned aerial vehicle (UAV) swarms based on mean-field game theory, characterized in that, include: Step 1: Obtain the original state information of the drone swarm, perform standardization processing, feature extraction and dimension mapping to obtain a set of swarm state features; Step 2: Perform mean field estimation on the set of population state characteristics to obtain mean field distribution information and population statistics; Step 3: Perform mean-field game modeling on the mean-field distribution information and population statistics to obtain the mean-field game model; Step 4: Solve the optimal feedback problem for the mean-field game model to obtain the initial optimal feedback strategy; Step 5: Perform cooperative incentive compatibility correction on the initial optimal feedback strategy to obtain the incentive-corrected cooperative strategy; Step 6: Perform policy sharing feedback processing on the incentive-modified cooperative strategy to obtain a shared feedback cooperative strategy; Step 7: Combine real-time collaborative deviation information to perform adaptive closed-loop adjustment on the shared feedback collaborative strategy to obtain the final collaborative planning instruction.

2. The method as described in claim 1, characterized in that, The mean-field game model is constructed in the following way: Based on the mean field distribution information and group statistics, a typical UAV's state evolution equation, stage cost function, and terminal cost function are defined. The stage cost function includes at least a control cost term, a mission tracking cost term, a formation coordination cost term, a safety collision avoidance cost term, and a coordination gain term. The terminal cost function is determined after the stage cost function based on the terminal target position, the expected terminal distribution, and the mission completion deviation, and serves as the boundary condition of the Hamilton-Jacobi-Bellman equation at the terminal moment. Define a population distribution evolution equation that describes the overall behavior evolution of the cluster; By coupling the state evolution equation, terminal cost function, stage cost function, and population distribution evolution equation, a mean-field game model composed of Hamilton-Jacobi-Bellman equation and Fock-Planck-Kolmogorov equation is obtained.

3. The method as described in claim 2, characterized in that, Solving the optimal feedback problem for the mean-field game model specifically includes: Given the mean field distribution information, the terminal cost function is used as the boundary condition of the Hamilton-Jacobi-Bellman equation at the terminal time. The Hamilton-Jacobi-Bellman equation is then solved to obtain the value function and its gradient. The optimal feedback control law is generated based on the value function gradient and the state evolution equation, and the optimal feedback control law is used as the control law expression of the initial optimal feedback strategy. The optimal feedback control law is substituted into the Fock-Planck-Kolmogorov equation to update the mean field distribution. Repeatedly perform value function solving, optimal feedback control law generation, and mean field distribution update until the distribution residual is less than a preset threshold, to obtain the initial optimal feedback strategy jointly characterized by the optimal feedback control law and its corresponding mean field distribution.

4. The method as described in claim 3, characterized in that, The initial optimal feedback strategy is modified to be compatible with the incentive mechanism, specifically including: Based on the optimal value function corresponding to the initial optimal feedback strategy, a collaborative benefit function and a deviation penalty term are introduced to construct the modified optimal value function and the modified Hamiltonian. The deviation between the individual position and the expected position of the group, the deviation between the individual control quantity and the average control quantity of the group, and the deviation between the individual task contribution and the average task contribution of the group are calculated. The calculated deviation is used to modify the individual comprehensive cost function, so that individuals that deviate from the group's cooperative goal are subject to higher cost constraints. The incentive-modified cooperative strategy is generated based on the gradient of the modified optimal value function or the modified Hamiltonian.

5. The method as described in claim 4, characterized in that, The modified incentive-based collaborative strategy undergoes policy-sharing feedback processing, specifically including: The incentive-modified cooperative strategy is compressed into a low-dimensional summary, and the group average direction, average velocity, task area density, local risk density and energy distribution summary are extracted to obtain the strategy sharing summary. The strategy sharing summary is used as a shared feedback quantity and sent to each UAV via broadcast or neighborhood communication. It is then embedded into the individual control law to form a shared feedback cooperative strategy with individual state, mean field distribution and strategy sharing summary as inputs.

6. The method as described in claim 1, characterized in that, Based on real-time collaborative deviation information, the shared feedback collaborative strategy is adaptively adjusted in a closed loop, specifically including: Based on the shared feedback collaboration strategy, the group distribution deviation, task completion deviation, collision risk deviation, and strategy consistency deviation are obtained as real-time collaboration deviation information. An adaptive stage cost function is constructed based on the real-time collaborative deviation information, and the collaborative incentive strength and shared feedback weight are updated according to a preset adaptive update rate. A collaborative feedback control law is generated based on the updated collaborative incentive strength and shared feedback weight, and the stability constraint of the collaborative feedback control law is verified. When the preset stability condition is met, the final collaborative planning instruction is output.

7. The method as described in claim 1, characterized in that, The original state information includes the UAV's number, location, speed, heading angle, remaining energy, mission payload, communication adjacency, obstacle distance, target area information, and environmental disturbance information.

8. A drone swarm collaborative planning system based on mean-field game theory, characterized in that, include: The information acquisition and processing module is used to acquire the raw state information of the UAV cluster, perform standardization processing, feature extraction and dimension mapping to obtain a set of group state features; The mean field estimation module is used to perform mean field estimation on the set of population state features to obtain mean field distribution information and population statistics. The game modeling module is used to perform mean-field game modeling on the mean-field distribution information and group statistics, and to construct a mean-field game model. The strategy solving module is used to solve the optimal feedback for the mean field game model and obtain the initial optimal feedback strategy. The incentive correction module is used to perform cooperative incentive compatibility correction on the initial optimal feedback strategy to obtain the incentive-corrected cooperative strategy. The shared feedback module is used to perform policy sharing feedback processing on the incentive-corrected collaborative strategy to obtain a shared feedback collaborative strategy. An adaptive adjustment module is used to adaptively adjust the shared feedback coordination strategy by combining real-time coordination deviation information to obtain the final coordination planning instruction.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements a drone swarm collaborative planning method based on average field game theory as described in any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a drone swarm collaborative planning method based on average field game theory as described in any one of claims 1 to 7.

Images