An unmanned aerial vehicle cluster cooperative decision-making method based on a differentiable model predictive control

Abstract

The present application relates to the technical field of unmanned aerial vehicle cluster control, and particularly relates to a method for cooperative decision-making of unmanned aerial vehicle cluster based on a differentiable model predictive control, comprising: performing dynamic modeling on a quadrotor unmanned aerial vehicle cluster, mapping a nonlinear dynamics to impulse space to obtain a linear model; constructing a double-layer optimization framework with a multi-agent proximal policy optimization as an outer layer and a differentiable model predictive control as an inner layer; constructing a symmetric positive definite cost matrix and a dynamic reference trajectory based on local observation; converting the inner layer optimization into a linear quadratic regulator problem, and obtaining optimal impulses by solving the problem through Riccati backward recursion; and converting the optimal impulses into physically executable control instructions through geometric control mapping. The present application guarantees that the control instructions for cooperative decision-making of the unmanned aerial vehicle cluster are physically feasible, reduces the training algorithm consumption, and improves the dynamic adaptability and formation flexibility of the cluster.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) swarm control technology, and in particular to a collaborative decision-making method for UAV swarms based on differentiable model predictive control. Background Technology

[0002] Cooperative control of UAV swarms requires balancing high-level collaborative strategies with precise low-level dynamic execution, making it a key research area in robotics and multi-agent systems. It is widely applied in agile collaborative operation scenarios such as cooperative formation, cooperative traversal, cooperative target tracking, and cooperative attack. Currently, technologies used for cooperative control of UAV swarms mainly include multi-agent reinforcement learning methods, model predictive control methods, differentiable model predictive control (MPC, referred to in this invention), and distributed trajectory optimization methods. Among these, multi-agent reinforcement learning methods, represented by multi-agent proximal policy optimization and multi-agent deep deterministic policy gradient, generally employ a centralized training and distributed execution architecture, enabling distributed decision-making in UAV swarms. Model predictive control methods ensure physical feasibility by explicitly solving constrained optimization problems. Differentiable model predictive control methods treat the optimization problem as a differentiable layer of a neural network to combine data-driven learning with model-based control. Distributed trajectory optimization methods utilize the differential flatness property to generate smooth trajectories for UAVs. The above methods all revolve around collaborative decision-making and trajectory planning of UAV swarms in complex environments, constituting the main technical route for current collaborative control of UAV swarms.

[0003] However, the aforementioned existing technologies still have insurmountable shortcomings in practical applications. On the one hand, multi-agent reinforcement learning methods treat the dynamics of quadcopter drones as a black box, ignoring the prior physical laws governing drone motion, resulting in low sample efficiency. Furthermore, the strategies output by neural networks are prone to generating high-frequency switching control signals, causing control command jitter and rendering the control commands physically infeasible, making it difficult to deploy on practical agile hardware platforms. On the other hand, differentiable model predictive control methods rely on general iterative numerical solvers to satisfy optimality conditions. Their gradient calculations require expanding optimization loops or using implicit function theorems, involving computationally intensive matrix decomposition. The complexity increases cubically with the number of decision variables, leading to excessively long training cycles in large-scale drone swarm training, thus lacking engineering practicality. Meanwhile, the cost function weights of traditional model predictive control methods are manually set and fixed, making it difficult to adapt to dynamically changing task scenarios. Distributed trajectory optimization methods have fixed cost functions and rigid geometric constraints, making it difficult to achieve elastic deformation and adaptive reconstruction of formations. It is evident that existing technologies struggle to balance the adaptability of high-level learning with the physical feasibility of low-level control, and suffer from issues such as high training computational power consumption, poor adaptability to dynamic environments, and limited cluster mobility.

[0004] Chinese Patent Publication No. CN121742505A discloses a method and system for controlling UAV swarms based on hierarchical reinforcement learning, including: establishing a leader-follower hierarchical swarm model; constructing a multi-layer communication topology, wherein UAVs are divided into layers according to the number of communication hops with the leader; determining the shortest communication path from each follower to the leader; calculating the output error between UAVs in adjacent layers based on the path; constructing an optimal control model for output consistency with the goal of minimizing the inter-layer output error, combined with a UAV dynamics model; and finally solving the optimal control using a reinforcement learning strategy iterative algorithm. It is evident that the aforementioned method and system for controlling UAV swarms based on hierarchical reinforcement learning have the following problems: Firstly, this method... The reinforcement learning strategy directly solves for control input commands through iterative iteration, without introducing a physical constraint filtering mechanism at the control output. This results in control strategies prone to high-frequency chattering signals, making the control commands physically infeasible and difficult to deploy on practical agile hardware platforms. Secondly, the state weight matrix and control weight matrix in its cost function are fixed and adjustable parameters, which cannot adjust the relative priority of optimization objectives in real time according to dynamic task scenarios, resulting in insufficient adaptability to complex environments such as changing obstacles and changing task requirements. Thirdly, this method relies on a preset hierarchical communication topology and shortest path structure, leading to rigid formation geometric constraints. In spatiotemporal bottleneck scenarios such as dynamic narrow gate crossings, it cannot achieve elastic shrinkage and adaptive reconstruction of the formation, limiting the overall mobility of the cluster. Summary of the Invention

[0005] To address this, the present invention provides a collaborative decision-making method for UAV swarms based on differentiable model predictive control, which overcomes the problems of physical infeasibility of control commands for UAV swarms, high computational cost of training, and poor adaptability to dynamic environments in the prior art.

[0006] To achieve the above objectives, this invention provides a collaborative decision-making method for unmanned aerial vehicle (UAV) swarms based on differentiable model predictive control, comprising: Step S1: Perform dynamic modeling on a distributed drone cluster consisting of several quadcopter drones to obtain a nonlinear dynamic model, and map the nonlinear dynamic model to the impulse space to obtain a linear dynamic model. Step S2: Construct a two-layer optimization framework with MAPPO as the outer framework and differentiable MPC as the inner framework. Step S3: Map the potential parameters determined based on the local observations of the quadcopter UAV into a lower triangular matrix, construct a symmetric positive definite SPD cost matrix based on the lower triangular matrix, and determine the dynamic reference trajectory based on the SPD cost matrix and the collaborative task of the UAV swarm. Step S4: The MPC problem of the inner framework is transformed into an LQR problem, and the optimal impulse is obtained by solving the closed analytical solution through Riccati back recursion. Step S5: The optimal impulse is mapped into physical control commands that can be executed by the quadcopter UAV through geometric control.

[0007] Furthermore, in the nonlinear dynamic model of step S1: The state of each quadcopter UAV is defined on the manifold ; ; ; in, Position of the inertial frame, For the linear velocity of the inertial frame, The pose is represented by a unit quaternion. For total thrust, For volume coordinates, angular velocity To control the input, It is a state vector; The discrete-time nonlinear dynamic model is as follows: ; in, For the mass of quadcopter drones, The gravity vector This is the rotation matrix corresponding to the quaternion. The thrust axis unit vector, This is quaternion multiplication. The sampling interval is denoted as .

[0008] Furthermore, in the linear dynamic model of step S1: Planning state is defined as ; The pseudo-control input is impulse. ; The discrete-time linear dynamic model is as follows: , Wherein, the state transition matrix Input matrix They are respectively: .

[0009] Furthermore, the mathematical form of the two-layer optimization framework is: ; in For policy network parameters, For high-level intent parameters, Let be the cost function of the LQR of the inner framework. To predict the time domain, This represents the mapping relationship for differentiable MPCs.

[0010] Further, step S3 includes: Step S31: Using MLP as a distributed actor network, input the local observations of each quadcopter UAV and output the latent parameters, wherein the activation functions are Mish and LayerNorm; Step S32: Map the potential parameters to the lower triangular matrix; Step S33, through Construct the SPD cost matrix, where L is the lower triangular matrix. For the SPD cost matrix, the diagonal elements of L are obtained through... The mapping is a linear mapping of the off-diagonal elements of L to positive real numbers, used to make Strictly positive definiteness, For numerically stable terms, For potential parameters; Step S34: Based on the SPD cost matrix and the collaborative task of the UAV swarm, the distributed actor network outputs three sets of SPD weight matrices corresponding to the collaborative task, respectively. The SPD weight matrix includes... ; Step S35: Based on the SPD cost matrix, synthesize and determine the total cost matrix: ; Step S36: Determine the dynamic reference trajectory by weighted averaging based on the SPD weight matrix: .

[0011] Further, step S4 includes: Step S41, define the TV-LQR cost function; Step S42: Starting from the terminal cost, calculate the optimal gain and cost-to-deliver matrix based on Riccati's reverse iteration. Step S43: Based on the optimal gain and the cost-to-delivery matrix, determine the optimal impulse using an affine feedback law.

[0012] Furthermore, the local observations of the quadcopter UAV include: its own motion state, neighboring states, formation error, obstacle states, and endpoint information.

[0013] Further, step S5 includes: Step S51: Integrate the optimal impulse to obtain the optimal acceleration; Step S52: Obtain the desired thrust vector after compensating for gravity with the optimal acceleration, and take the Euclidean norm as the total thrust. Step S53: Align the z-axis of the body coordinate system with the normalized desired thrust vector, construct orthogonal body coordinate system axes based on the reference yaw angle, and obtain the desired rotation matrix; Step S54, in the special orthogonal group The geometric attitude controller is designed on the upper part, and the angular velocity of the body coordinate system is determined based on the attitude error quaternion; Step S55: Determine the physical control command based on the total thrust and the angular velocity of the body coordinate system.

[0014] Furthermore, it also includes: Step S6 involves using Riccati's backward recursion as an explicit computation graph, deriving gradients using the envelope theorem, and then training the policy network by propagating the gradients from the physical control commands to the policy network parameters of the outer framework through end-to-end gradient propagation. This includes: Step S61: Perform gradient decomposition based on the chain rule to determine the gradient of the policy loss with respect to the network parameters. Step S62: Use the envelope theorem to simplify the differential of the Riccati equation and obtain the differential relationship between the cost and the transition matrix. The spectral radius of the closed-loop state transition matrix is ​​less than 1, which is used to suppress gradient explosion. Step S63: Using the MAPPO pruning alternative target optimization policy network, the advantage value is obtained based on generalized advantage estimation, and the total loss function is determined.

[0015] Furthermore, in step S63, the total loss function includes the policy loss, the squared error loss of the value function, and an entropy regularization term, which is used to prevent premature convergence of the policy. Gaussian sampling is used during the training of the policy network, and Gaussian noise is added to the analytical solution.

[0016] Compared with the prior art, the beneficial effects of the present invention are that by constructing a two-layer optimization framework of outer layer reinforcement learning and inner layer differentiable model predictive control, and combining symmetric positive definite cost weight learning and analytical optimization solution, the high-level cooperative strategy and the low-level physical constraints are unified into end-to-end optimization. This solves the problems of physical infeasibility of control commands, large training computing power consumption and poor adaptability to dynamic environments in the prior art, and realizes adaptive cooperative decision-making and physically feasible control of UAV swarms in dynamic environments.

[0017] Furthermore, this invention decouples translation and yaw by mapping the nonlinear dynamic model to the linear canonical form of impulse space, providing a linearized model basis for subsequent optimization solutions.

[0018] Furthermore, this invention decouples collaborative decision-making from dynamic execution by outputting high-level intent parameters through outer-layer reinforcement learning and solving for physically feasible instructions through inner-layer differentiable model predictive control, thereby reducing the learning difficulty of the policy network.

[0019] Furthermore, this invention constructs a symmetric positive definite cost matrix using a lower triangular matrix, ensuring the strict convexity of the inner optimization, enabling the cost weights to characterize the coupling relationship between state dimensions and achieve dynamic modulation of multi-objective priorities.

[0020] Furthermore, this invention replaces the iterative numerical solver by using Ricardi's reverse recursive solution to obtain the closed-form analytical solution of the linear quadratic regulator, thereby reducing computational complexity and overcoming the computational bottleneck of large-scale cluster training.

[0021] Furthermore, this invention transforms the optimal impulse into physically executable control commands through geometric control mapping, ensuring the smoothness and physical feasibility of the control commands and avoiding high-frequency control signal jitter.

[0022] Furthermore, this invention achieves end-to-end optimization by using Riccati recursion as an explicit computation graph and deriving gradients using the envelope theorem, thus avoiding the high computational overhead of implicit differentiation and suppressing gradient explosion during long-term training.

[0023] Furthermore, this invention enables UAV swarms to autonomously adjust their formation configuration when crossing obstacles by dynamically modulating the multi-objective cost weights, thereby achieving elastic shrinkage and adaptive reconfiguration of the formation. Attached Figure Description

[0024] Figure 1 This is a flowchart of the UAV swarm collaborative decision-making method based on differentiable model predictive control according to the present invention; Figure 2 This is a flowchart of step S3 of the UAV swarm collaborative decision-making method based on differentiable model predictive control of the present invention; Figure 3 This is a flowchart of step S4 of the UAV swarm cooperative decision-making method based on differentiable model predictive control of the present invention; Figure 4 This is a flowchart of step S5 of the UAV swarm collaborative decision-making method based on differentiable model predictive control of the present invention; Figure 5 This is a performance trend comparison chart of the UAV swarm collaborative decision-making method based on differentiable model predictive control under different dynamic gate initial angle conditions in Embodiments 1 to 4 of the present invention. Detailed Implementation

[0025] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0026] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0027] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0028] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0029] Please see Figure 1 The diagram shows a flowchart of the UAV swarm cooperative decision-making method based on differentiable model predictive control according to the present invention. The present invention provides a UAV swarm cooperative decision-making method based on differentiable model predictive control, comprising: Step S1: Perform dynamic modeling on a distributed drone cluster consisting of several quadcopter drones to obtain a nonlinear dynamic model, and map the nonlinear dynamic model to the impulse space to obtain a linear dynamic model. Specifically, in the nonlinear dynamic model of step S1: The state of each quadcopter UAV is defined on the manifold ; ; ; in, Position of the inertial frame, For the linear velocity of the inertial frame, The pose is represented by a unit quaternion. For total thrust, For volume coordinates, angular velocity To control the input, It is a state vector; The discrete-time nonlinear dynamic model is as follows: ; in, For the mass of quadcopter drones, The gravity vector This is the rotation matrix corresponding to the quaternion. The thrust axis unit vector, This is quaternion multiplication. The sampling interval is denoted as .

[0030] In one specific embodiment, a cluster simulation environment consisting of three quadcopter drones is built in NVIDIA Isaac Sim, and the dynamic model of the Bitcraze Crazyflie 2.1 quadcopter drone is imported to realize the dynamic modeling of a distributed drone cluster consisting of several quadcopter drones, obtain a nonlinear dynamic model, and then map the nonlinear dynamic model to the impulse space to obtain a linear dynamic model. in, g' is the scalar of gravitational acceleration, which will not be elaborated here.

[0031] It is understood that the nonlinear dynamic model consists of a translational part and a rotational part. The translational part describes the evolution of the position and linear velocity of the inertial frame over time according to the Newton-Euler equations, while the rotational part describes the evolution of the attitude represented by the unit quaternion with the angular velocity of the body coordinate system according to quaternion kinematics. The thrust axis unit vector is used to indicate that the total thrust acts along a specific axis of the body coordinate system, and the rotation matrix corresponding to the quaternion is used to project the thrust in the body coordinate system to the inertial frame, thus forming the resultant external force acting on the UAV together with the gravity vector. The total thrust and the angular velocity of the body coordinate system together constitute the control input applied to the system. The sampling interval is used to discretize the continuous motion relationship according to a fixed time step to obtain the recursive relationship for predicting the state at the next time step from the state at the current time step.

[0032] It is understandable that quadcopter UAVs are underactuated rigid body systems, whose motion includes both translation and rotation, and whose thrust direction changes with attitude. Therefore, position and velocity are described in the inertial frame, and attitude is used to characterize the rotational relationship between the body coordinate system and the inertial coordinate system. Using unit quaternions to represent attitude avoids the singularity and gimbal lock problems of Euler angles, maintaining the stability of the attitude description during agile maneuvers at large angles. Defining the state on a manifold ensures that the attitude always satisfies its unit length constraint, preventing numerical calculations from deviating from the set of valid attitudes. The translational relationship is established based on the physical laws of net external force and mass. The axial thrust is projected onto the inertial frame through attitude rotation and superimposed with gravity to determine the changes in position and velocity. The rotational relationship is established based on rigid body kinematics, with the attitude continuously changing due to the angular velocity of the body coordinate system. Since control decisions and numerical calculations are performed at discrete moments, the continuous motion relationship is discretized at fixed sampling intervals, presenting it as a recursive form that calculates the state at the next moment from the current state and control input, facilitating iterative calculations and subsequent processing on a digital platform.

[0033] Specifically, in the linear dynamic model of step S1: Planning state is defined as ; The pseudo-control input is impulse. ; The discrete-time linear dynamic model is as follows: , Wherein, the state transition matrix Input matrix They are respectively: .

[0034] In one specific embodiment, the differential flatness characteristic of a quadcopter UAV is utilized to map the nonlinear dynamic model to the impulse space, resulting in a linear dynamic model in linear canonical form of the impulse space; wherein, the planning state of the linear dynamic model is defined as... ; Corresponding position Corresponding speed, Corresponding acceleration.

[0035] Where I represents a 3×3 identity matrix and 0 represents a 3×3 zero matrix, which will not be elaborated further.

[0036] It is understood that the linear dynamics model describes the evolution of the planned states over time steps in the impulse space, with position, velocity, and acceleration as the planned states and impulse as the pseudo-control input. The state transition matrix represents how, when the pseudo-control input is zero, the current position, velocity, and acceleration are recursively transferred to the next time step according to the kinematic integral relationship. The input matrix represents the effect of impulse on acceleration, velocity, and position. The yaw angle is decoupled from the optimization of the linear dynamics model and tracked separately as a reference value to ensure that the optimization of the translational channel remains in a linear canonical form, unaffected by the attitude yaw degrees of freedom.

[0037] Understandably, the original dynamics of a quadcopter UAV are highly nonlinear, with attitude and translation coupled together. Direct optimization on these components is computationally expensive and difficult to guarantee real-time performance. However, quadcopter UAVs possess differential flatness; their position and yaw can be represented as flat outputs, and all states and control variables can be expressed by these flat outputs and their finite-order derivatives. Therefore, the originally nonlinear motion relationship can be equivalently transformed into a description within an impulse space composed of position and its derivatives. Within this space, with position, velocity, and acceleration as the planning states and impulse as the pseudo-control input, the motion relationship presents a linear canonical form. This isolates the attitude-related nonlinear elements from the translational channel solution, allowing subsequent solutions to be performed within a linear framework without directly addressing the original nonlinear constraints. Since the yaw degree of freedom does not affect the linear structure of the translational channel, decoupling it from optimization and tracking a given reference value independently allows for independent yaw control without sacrificing the linearity of the translational solution. The linear dynamic model obtained in this way retains the physical essence of the UAV's motion while simplifying the problem to a linear form, providing a prerequisite for transforming the model predictive control problem of the inner framework into a linear quadratic regulator problem and solving it analytically and recursively.

[0038] Step S2: Construct a two-layer optimization framework with MAPPO as the outer framework and differentiable MPC as the inner framework. Specifically, the mathematical form of the two-layer optimization framework is: ; in, For policy network parameters, For high-level intent parameters, Let be the cost function of the LQR of the inner framework. To predict the time domain, Let ≜ be the mapping relationship of differentiable MPC, and ≜ represents the definition.

[0039] It is understood that the two-layer optimization framework optimizes the policy network parameters using the outer layer MAPPO, and solves the cost function in the prediction time domain using the inner layer differentiable MPC under the condition of given high-level intent parameters. Here, the policy network parameters are used to determine the policy of the outer framework to generate high-level intent parameters based on local observations, the high-level intent parameters are used as the optimization objective passed from the outer framework to the inner framework, the prediction time domain is used to limit the time step range covered by the solution of the inner framework, and the mapping relationship of the differentiable MPC is used to map the high-level intent parameters to the optimal solution result of the inner framework, so that the outer and inner layers form a top-down optimization transfer within the same framework.

[0040] It is understandable that the collaborative control of UAV swarms needs to simultaneously consider high-level collaborative decision-making and low-level dynamic feasibility. If only reinforcement learning is used to directly output low-level control quantities, it is difficult to guarantee the physical feasibility of the generated commands. If only model predictive control is used, the weights of its cost function need to be manually set and it is difficult to adapt to dynamically changing task scenarios. To this end, the proposed two-layer optimization framework couples the above two methods in layers. The outer framework uses MAPPO to optimize the policy network parameters and generates high-level intent parameters based on the local observations of each quadcopter UAV. These high-level intent parameters are not low-level control quantities, but rather intent information representing the optimization objective and the relative priority of multiple objectives. The inner framework uses differentiable MPC to receive the high-level intent parameters, solves for the optimal solution corresponding to the cost function in the prediction time domain, and maps it to a result that conforms to dynamic constraints. Thus, the outer framework focuses on high-level decisions about which collaborative intent to adopt in which scenario, while the inner framework focuses on how to obtain a physically feasible solution given the intent. The two are connected through high-level intent parameters, decoupling complex collaborative decisions from the underlying physical execution. At the same time, since the differentiable MPC of the inner framework has a differentiable mapping relationship, the policy performance of the outer framework can influence the optimization of the policy network parameters in reverse through this mapping relationship. This allows the entire two-layer optimization framework to be optimized end-to-end as a whole, thereby enabling the high-level policy to dynamically adjust its collaborative intent according to the scenario while ensuring the physical feasibility of the underlying control.

[0041] Step S3: Map the potential parameters determined based on the local observations of the quadcopter UAV into a lower triangular matrix, construct a symmetric positive definite SPD cost matrix based on the lower triangular matrix, and determine the dynamic reference trajectory based on the SPD cost matrix and the collaborative task of the UAV swarm. Please continue reading. Figure 2 The diagram shows a flowchart of step S3 in the UAV swarm cooperative decision-making method based on differentiable model predictive control according to the present invention; specifically, step S3 includes: Step S31: Using MLP as a distributed actor network, input the local observations of each quadcopter UAV and output the latent parameters, wherein the activation functions are Mish and LayerNorm; Specifically, the local observations of the quadcopter drone include: its own motion status, neighbor status, formation error, obstacle status, and endpoint information.

[0042] In one specific embodiment, an MLP is used as a distributed actor network. Local observations from each quadcopter UAV are input, and the latent parameters are output. The activation functions of the MLP are Mish and LayerNorm. In this embodiment, the MLP preferably adopts a structure with 3 hidden layers and 256 neurons in each layer. Step S32: Map the potential parameters to the lower triangular matrix; Step S33, through Construct the SPD cost matrix, where L is the lower triangular matrix. For the SPD cost matrix, the diagonal elements of L are obtained through... The mapping is a linear mapping of the off-diagonal elements of L to positive real numbers, used to make Strictly positive definiteness, For numerically stable terms, For potential parameters; In one specific embodiment, the numerical stability term Preferably, 10 -4 .

[0043] Step S34: Based on the SPD cost matrix and the collaborative task of the UAV swarm, the distributed actor network outputs three sets of SPD weight matrices corresponding to the collaborative task, respectively. The SPD weight matrix includes... ; In one specific embodiment, the cooperative tasks are formation maintenance, obstacle tracking, and destination navigation, which are respectively correlated with the SPD weight matrix. One-to-one correspondence.

[0044] Step S35: Based on the SPD cost matrix, synthesize and determine the total cost matrix: ; Step S36: Determine the dynamic reference trajectory by weighted averaging based on the SPD weight matrix: .

[0045] It is understood that in step S33, the SPD cost matrix is ​​constructed by multiplying the lower triangular matrix with its transpose, ensuring that the constructed matrix is ​​always a symmetric matrix. Specifically, the diagonal elements of the lower triangular matrix are mapped to positive real numbers to guarantee the strict positive definiteness of the constructed matrix. The numerical stability term is used to prevent the diagonal elements from taking too small a value, leading to numerical ill-conditioning. The off-diagonal elements of the lower triangular matrix are determined by a linear mapping to ensure that the constructed matrix can characterize the coupling correlation between various state dimensions. In steps S34 to S36, the distributed actor network outputs corresponding SPD weight matrices for the three collaborative tasks of formation maintenance, obstacle tracking, and endpoint navigation. The total cost matrix is ​​then synthesized based on the SPD cost matrices, and the dynamic reference trajectory is determined by weighted averaging based on the SPD weight matrices. This unifies the optimization objectives of multiple collaborative tasks within the solution of the inner framework.

[0046] Understandably, the LQR problem of the inner framework only possesses strict convexity and a unique solution when its cost-weight matrix is ​​an SPD matrix. If the weight matrix directly output by the MLP does not satisfy symmetric positive definiteness, the solution to the inner framework will be unsolvable or numerically collapse. Therefore, instead of having the MLP directly output the cost-weight matrix, it outputs the latent parameters and maps them to lower triangular matrices. Then, using the inverse construction method of Cholesky decomposition, the cost-weight matrix is ​​constructed by multiplying the lower triangular matrix with its transpose. The resulting matrix is ​​structurally symmetric and, after mapping the diagonal elements to positive real numbers and superimposing numerically stable terms, is necessarily positive definite. This construction method guarantees the symmetric positive definiteness of the SPD matrix, ensuring that the inner solver always faces a well-state convex optimization problem. Compared to using a diagonal weight matrix, this Cholesky decomposition construction allows off-diagonal elements to take non-zero values, enabling the cost weight matrix to characterize the coupling relationships between different state dimensions. This allows the MLP to learn dense weights with inter-axis coupling, thus possessing the ability to reshape the optimization objective in real time according to the scenario, such as relaxing constraints in a certain direction when passing through a narrow gate. Furthermore, since the UAV swarm needs to simultaneously complete multiple collaborative tasks such as formation maintenance, obstacle tracking, and destination navigation, the MLP outputs corresponding SPD weight matrices for each task, which are then synthesized into a total cost matrix and weighted to obtain a dynamic reference trajectory. This allows the relative priority of each task to be dynamically adjusted according to changes in local observations, thereby achieving dynamic modulation of multi-objective priorities within a unified cost framework. In this way, the MLP only needs to output high-level parameters representing the optimization intention, without directly outputting low-level control variables, decoupling high-level collaborative decision-making from low-level physical solutions and reducing the learning difficulty of the network.

[0047] Step S4: The MPC problem of the inner framework is transformed into an LQR problem, and the optimal impulse is obtained by solving the closed analytical solution through Riccati back recursion. Please continue reading. Figure 3 The diagram shows a flowchart of step S4 in the UAV swarm cooperative decision-making method based on differentiable model predictive control according to the present invention; specifically, step S4 includes: Step S41, define the TV-LQR cost function; In a specific embodiment, the TV-LQR cost function is defined as: ; in, R is a positive definite symmetric impulse penalty matrix, which can be taken as R = rI, r > 0; Q k Q is the total cost matrix synthesized in step S35. H This is the terminal cost matrix.

[0048] Step S42: Starting from the terminal cost, calculate the optimal gain and cost-to-deliver matrix based on Riccati's reverse iteration. In one specific embodiment, from the terminal cost Beginning, from Iterate backwards to zero to calculate the optimal gain and cost matrix: ; ; Among them, K k Pk is the optimal gain matrix, and Pk is the cost-to-go matrix, which still satisfies positive definite symmetry. Step S43: Based on the optimal gain and the cost-to-delivery matrix, determine the optimal impulse using an affine feedback law.

[0049] In one specific embodiment, the optimal impulse is obtained through an affine feedback law: ; in, This is the optimal impulse.

[0050] Understandably, the TV-LQR cost function consists of a state deviation term and an impulse penalty term. The state deviation term, weighted by the SPD cost matrix, is used to constrain the deviation of the planned state from the dynamic reference trajectory. The impulse penalty term, weighted by the impulse penalty matrix R, is used to suppress excessive spurious control inputs. In step S42, starting from the terminal cost, the process iterates backward from the prediction time domain H along the time steps to zero, progressively calculating the optimal gain K for each time step. k With cost to the matrix P k The cost to the matrix P kThe optimal cost accumulated from the current time step to the terminal, the optimal gain K. k The feedback coefficient is used to characterize the optimal impulse generated from the current planning state; in step S43, the optimal impulse at each time step is obtained by affine feedback law based on the optimal gain and the cost-to-deliver matrix.

[0051] Understandably, after differential flattening, the optimization problem of the inner framework takes the form of a linear canonical function in the impulse space, with a quadratic cost function and linear dynamics constraints. Therefore, the MPC problem can be equivalently transformed into a TV-LQR problem. For this type of problem, the optimal solution satisfies the Riccati recursion derived from dynamic programming. It eliminates the need for general iterative numerical solvers such as SQP and interior-point methods. Instead, it iterates backwards along the time steps from the terminal cost, gradually obtaining the optimal gain and cost-to-move matrix for each time step. Based on this, the optimal impulse is obtained analytically in a closed-form manner using affine feedback. The reason for this approach is that general iterative numerical solvers need to repeatedly solve nonlinear equations satisfying optimality conditions and perform matrix decomposition. Their computational complexity increases cubically with the number of decision variables, making it unbearable in large-scale training of UAV swarms. In contrast, the Riccati backward recursion consists only of matrix multiplication and matrix inversion, with a regular computational structure that can be highly parallelized on a graphics processor. Its computational complexity is only related to the dimension of the planning state, thus avoiding the computational bottleneck of general solvers. Meanwhile, since the solution process is in closed analytical form, each step of the operation is differentiable, which provides a basis for subsequent end-to-end gradient propagation using Riccati reverse recursion as an explicit computation graph; furthermore, by using the SPD cost matrix as the weight of the state deviation term and supplementing it with an impulse penalty term, the obtained optimal impulse is constrained while tracking the dynamic reference trajectory, so that the control quantity obtained by its transformation is limited to a smooth range that conforms to the physical characteristics of the UAV.

[0052] Step S5: The optimal impulse is mapped into physical control commands that can be executed by the quadcopter UAV through geometric control.

[0053] Please continue reading. Figure 4 The diagram shown is a flowchart of step S5 of the UAV swarm collaborative decision-making method based on differentiable model predictive control according to the present invention. Specifically, step S5 includes: Step S51: Integrate the optimal impulse to obtain the optimal acceleration; In a specific embodiment, the optimal acceleration is denoted as... .

[0054] Step S52: Obtain the desired thrust vector after compensating for gravity with the optimal acceleration, and take the Euclidean norm as the total thrust. In one specific embodiment, there is ; in, For total thrust, Let be the desired thrust vector.

[0055] Total thrust Constraints are within the physical execution range, denoted as .

[0056] Step S53: Align the z-axis of the body coordinate system with the normalized desired thrust vector, construct orthogonal body coordinate system axes based on the reference yaw angle, and obtain the desired rotation matrix; In a specific embodiment, the desired rotation matrix is ​​denoted as: .

[0057] Step S54, in the special orthogonal group The geometric attitude controller is designed on the upper part, and the angular velocity of the body coordinate system is determined based on the attitude error quaternion; In one specific embodiment, the attitude error quaternion is: ; Where, q des Let q be the quaternion corresponding to the desired pose. curr The current attitude quaternion, This is quaternion multiplication; q e Write as the scalar part q w，e and vector part q xyz，e The angular velocity control law for the body coordinate system is: ; in, For volume coordinates, angular velocity To control the bandwidth time constant.

[0058] Step S55: Determine the physical control command based on the total thrust and the angular velocity of the body coordinate system.

[0059] In one specific embodiment, there is ; in, These are physical control commands.

[0060] It is understood that in steps S51 to S52, the optimal acceleration is obtained by integrating the optimal impulse, and the desired thrust vector is obtained by compensating for gravity based on the optimal acceleration. The Euclidean norm of the desired thrust vector is taken as the total thrust, and the total thrust is constrained within the physical execution range of the quadcopter UAV to ensure that the desired thrust is achievable in hardware. In step S53, the z-axis of the body coordinate system is aligned with the normalized desired thrust vector, and the axes of the body coordinate system are constructed by cross product with the reference yaw angle to obtain the desired rotation matrix, which is used to characterize the desired attitude that the quadcopter UAV should achieve to generate the desired thrust vector. In step S54, a geometric attitude controller is designed on a special orthogonal group, and the angular velocity of the body coordinate system is calculated based on the attitude error quaternion between the current attitude and the desired attitude. The control bandwidth time constant is used to adjust the speed at which the angular velocity is generated by the attitude error.

[0061] Understandably, the optimal impulse obtained by the inner framework through Riccati's back recursion is a pseudo-control input in impulse space, not a physical quantity that the quadcopter UAV's underlying flight controller can directly execute. Therefore, it is necessary to utilize the differential flatness characteristic of the quadcopter UAV to inversely map it through geometric control into an actually executable physical control command. Specifically, since acceleration corresponds to thrust under the differential flatness framework, the optimal acceleration is first restored by integrating the optimal impulse, and then the desired thrust vector is obtained by compensating for gravity. Its direction determines the attitude that the airframe should point to, and its magnitude corresponds to the required total thrust. Considering that the quadcopter UAV is an underactuated system, its thrust can only be generated along a certain fixed axis of the airframe. Therefore, by aligning the airframe axis with the normalized desired thrust vector and combining it with the separately tracked reference yaw angle, a desired attitude that satisfies orthogonality can be uniquely determined and represented by a desired rotation matrix. To drive a quadrotor UAV from its current attitude to the desired attitude, a geometric attitude controller is designed on a special orthogonal group. The deviation between the current and desired attitudes is directly measured using attitude error quaternions on the rotating manifold where the attitude lies, and the volume coordinate angular velocity is generated accordingly. This avoids the singularity problem of Euler angles describing large-angle attitudes. The control bandwidth time constant is used to determine the time scale for attitude error convergence. Thus, the total thrust and volume coordinate angular velocity obtained through geometric control inverse mapping constitute the physical control commands that conform to the dynamic constraints of the quadrotor UAV and are constrained within their physical execution range. This ensures that the optimal solution obtained by the inner framework can be smoothly and feasiblely executed on actual hardware.

[0062] Specifically, the UAV swarm collaborative decision-making method based on differentiable model predictive control of the present invention also includes: Step S6, using Riccati's backward recursion as an explicit computation graph, and deriving gradients using the envelope theorem, is used for end-to-end gradient propagation from the physical control commands to the policy network parameters of the outer framework, thereby training the policy network, including: Step S61: Perform gradient decomposition based on the chain rule to determine the gradient of the policy loss with respect to the network parameters. In a specific embodiment, policy loss The gradient with respect to the network parameter θ can be decomposed as: ; The actual gradient calculation is performed automatically along the computation graph recursively constructed by Riccati using an automatic differentiation framework.

[0063] Step S62: Use the envelope theorem to simplify the differential of the Riccati equation and obtain the differential relationship between the cost and the transition matrix. The spectral radius of the closed-loop state transition matrix is ​​less than 1, which is used to suppress gradient explosion. In one specific embodiment, the envelope theorem is used to simplify the differential of the Riccati equation, yielding the differential relationship between the cost-to-departure matrix; the total differential of the high-level intent parameter z is then obtained, denoted as... Then, using the envelope theorem, the differential relation of the cost-to-delivery matrix can be obtained as follows: ; in, The closed-loop state transition matrix has a spectral radius. This ensures that the error signal decays during backpropagation, thereby effectively suppressing gradient explosion.

[0064] Step S63: Using the MAPPO pruning alternative target optimization policy network, the advantage value is obtained based on generalized advantage estimation, and the total loss function is determined.

[0065] Specifically, in step S63, The total loss function includes the policy loss, the squared error loss of the value function, and an entropy regularization term, which is used to prevent premature convergence of the policy. Gaussian sampling is used during the training of the policy network, and Gaussian noise is added to the analytical solution.

[0066] In a specific embodiment, the total loss function is: ; in, As a substitute for the strategy, The squared error loss of the value function. For entropy regularization, , The adjustment coefficient is used; Gaussian sampling is used during the training of the policy network, that is, Gaussian noise is added to the optimal action analytical solution obtained by the inner framework to ensure the safety of exploration.

[0067] The training process uses Gaussian sampling, adding Gaussian noise to the analytical solution to ensure the safety and effectiveness of the exploration.

[0068] Understandably, in step S61, according to the chain rule, the gradient of the policy loss with respect to the policy network parameter θ is decomposed into the gradient of the policy loss with respect to the physical control command, the gradient of the physical control command with respect to the optimal impulse, and the stepwise product of the gradient of the optimal impulse with respect to the policy network parameter θ via the inner framework, thereby establishing a gradient propagation path from the physical control command to the policy network parameter θ; in step S62, the Riccati equation is differentiated using the envelope theorem to obtain the cost-to-decision matrix P. k The differential relation, wherein the differential relation contains a closed-loop state transition matrix, and the spectral radius of the closed-loop state transition matrix is ​​less than 1, is used to gradually attenuate the error signal as it propagates backward along the time step, thereby suppressing gradient explosion; in step S63, the MAPPO pruning replacement target is used as the policy loss, and the advantage value is obtained based on the generalized advantage estimation, and then combined with the squared error loss of the value function and the entropy regularization term to form the total loss function, wherein the entropy regularization term is used to maintain the exploratory nature of the policy and prevent the policy from prematurely converging, and the Gaussian sampling is used to add Gaussian noise to the analytical solution obtained by the inner framework to achieve exploration.

[0069] It is understandable that for a two-layer optimization framework to achieve end-to-end training, the policy loss of the outer framework must be able to propagate back through the inner framework to the policy network parameters. The solution of the inner framework is an optimization process conditioned on the high-level intention parameters output by the policy network. If numerical differentiation is used, errors are introduced due to finite differences; if implicit function theorems are used to differentiate the optimality conditions, the massive Hessian matrix needs to be calculated and inverted. Neither approach is suitable for large-scale training. Therefore, this invention treats the Riccati back-recursion as an explicit computational graph composed of deterministic operations, allowing the gradient to propagate back along this graph step by step according to the chain rule. This chain rule decomposes the gradient of the policy loss with respect to the policy network parameters into a product of several local gradients. When differentiating the cost-to-derivative matrix, the envelope theorem allows neglecting terms arising from parameter variations in the optimal solution itself, retaining only the explicit dependence of the cost function on the parameters. This avoids the calculation and inversion of the Hessian matrix caused by differentiating the optimality conditions, simplifying the gradient derivation. Furthermore, the differential relation derived from the Riccati recursion contains a closed-loop state transition matrix. The spectral radius of this matrix is ​​less than 1, implying that its repeated action exhibits contraction characteristics. Consequently, the error signal gradually decays rather than cumulatively amplifies during backward propagation along the time step, thus suppressing gradient explosion in long-term training and ensuring numerical stability. Building upon this, the outer framework employs MAPPO's pruning substitution objective to optimize the policy. This pruning mechanism limits the magnitude of policy updates, and generalized advantage estimation achieves a trade-off between bias and variance to obtain the advantage value. Simultaneously, a squared error loss of the value function is introduced to train the critic, and an entropy regularization term is introduced to maintain the policy's exploratory nature and prevent premature convergence to a local policy. When generating actions, the deterministic analytical solution obtained by the inner framework is not directly executed. Instead, Gaussian noise is superimposed on the analytical solution for Gaussian sampling, thereby ensuring the policy possesses sufficient exploratory capability while maintaining the explored actions within the physically feasible range.

[0070] Example 1 Example 1 applies the UAV swarm cooperative decision-making method based on differentiable model predictive control provided by this invention to a mission scenario in which a formation of three quadcopter UAVs traverses a suspended swing dynamic gate, to specifically illustrate the implementation process and technical effects of this method. The UAV swarm cooperative decision-making method based on differentiable model predictive control provided by this invention is denoted as MAPPO-dMPC, specifically: A distributed cluster simulation environment was established consisting of three Bitcraze Crazyflie 2.1-class quadrotor UAVs. The physical parameters of each quadrotor UAV were set according to the measured values ​​of this model, and the mass was approximately 0.027 kg. Following the method described in step S1, a nonlinear dynamic model was established for each UAV. The state was defined on the manifold, and the state vector consisted of the inertial frame position, inertial frame linear velocity, and attitude expressed as a unit quaternion. The control inputs were the total thrust and the body coordinate angular velocity. Utilizing the differential flatness of the quadrotor, the nonlinear dynamic model was mapped to a linear model in impulse space. The planned states were taken as position, velocity, and acceleration, and the pseudo-control input was taken as impulse. The state transition matrix and input matrix were given according to the definitions in step S1, where the dimension of the identity matrix and the zero matrix were both 3×3.

[0071] The initial configuration of the UAV swarm is a V-shaped formation, with the initial relative coordinates of the three UAVs being {[0, 0, 0], [-0.3, 0.3, 0], [-0.3, -0.3, 0]} meters, and a nominal lateral span of approximately 0.6 meters. A suspended, swinging dynamic gate is deployed in the flight environment. The gate frame can undergo nonlinear periodic oscillations along the lateral direction, with the oscillation period dynamically changing between 2.5 and 4.0 seconds. The net width of the gate frame is 0.4 meters, which is less than the nominal lateral span of the formation, to force the swarm to elastically compress and recover during passage.

[0072] Following step S2, a two-layer optimization framework is constructed with the outer layer being MAPPO and the inner layer being differentiable MPC, and a centralized training and distributed execution architecture is adopted.

[0073] Each drone is deployed with a distributed actor network, employing a multilayer perceptron (MLP) architecture with three hidden layers, each containing 256 neurons. The Mish activation function is used, and LayerNorm normalization is applied between layers. The input to the actor network consists of local observations from each drone, including its own motion state, relative states of its neighbors, formation error, relative positions of the four corner points of the dynamic gate, and relative position of the endpoint. The output of the actor network is high-level intent parameters, which are latent parameters used to construct the cost matrix and dynamic reference trajectory, rather than directly outputting the final control command.

[0074] The inner MPC problem is transformed into a time-varying linear quadratic regulator (TV-LQR) problem. The prediction time domain H can be set to 2, 5, or 10 time steps according to actual needs. In this embodiment, unless otherwise specified, H=5 is used. The inner solution does not rely on a general iterative numerical solver. Instead, according to step S4, the closed-form analytical solution is directly calculated through Riccati's back recursion to obtain the optimal impulse sequence. The optimal impulse at the current moment is then used to generate control commands.

[0075] The latent parameters output by the actor network are first mapped to lower triangular matrices according to step S3. Then, a strictly symmetric positive definite SPD cost matrix is ​​constructed by multiplying the lower triangular matrix with its transpose and adding numerically stable terms. For the three sub-objectives of formation maintenance, obstacle tracking, and destination navigation in the cooperative task, the actor network outputs three sets of corresponding SPD weight matrices, which are then combined into a total cost matrix through weighted summation. Simultaneously, based on the SPD weight matrix, a reference trajectory for the current moment is dynamically generated using a regularized weighted average method, allowing the relative priority of multiple objectives to be adjusted in real time according to changes in local observations.

[0076] Integrating the optimal impulse obtained in step S4 yields the optimal acceleration for the first step in the prediction time domain; after compensating for gravity, the desired thrust vector is obtained, and its Euclidean norm is taken as the total thrust, which is then constrained within the physical execution range [f]. min f max Next, the z-axis of the body coordinate system is aligned with the normalized desired thrust vector, and orthogonal axes of the body coordinate system are constructed in conjunction with the reference yaw angle to form the desired rotation matrix. A geometric attitude controller is designed on the special orthogonal group SO(3), and the body coordinate system angular velocity control law is calculated using the attitude error quaternion. Finally, the total thrust and the body coordinate system angular velocity together constitute the physically executable control command. This process does not require solving nonlinear optimization and has extremely high execution efficiency.

[0077] During the training phase, the MAPPO algorithm is used to optimize the policy network by pruning alternative targets, and the advantage function is calculated based on generalized advantage estimation. Following step S6, the Riccati algorithm is recursively constructed into an explicit computation graph, and the end-to-end gradient from physical control commands to the outer policy network parameters is derived using the envelope theorem to directly minimize the policy loss. To prevent premature convergence, a policy entropy regularization term is added to the total loss function; to ensure safety during exploration, Gaussian noise is superimposed on the analytical solution during action sampling. After training convergence, the centralized commentator network is removed, and each drone can achieve distributed, real-time autonomous decision-making solely through a distributed actor network. Example 2

[0078] The direct action learning method GART-MAPPO is used instead of the two-layer optimization framework in Example 1, and the rest is the same as in Example 1. Example 3

[0079] The Dist-Opt distributed trajectory optimization method is used to replace the two-layer optimization framework of Example 1, and the rest is the same as Example 1. Example 4

[0080] The hierarchical High-MPC method is used to replace the two-layer optimization framework of Example 1, and the rest is the same as Example 1. Example 5

[0081] The MAPPO-MLP method is adopted, meaning the outer layer still uses the MAPPO multi-agent reinforcement learning framework, but the inner layer does not include any differentiable model predictive control layer. Specifically, the distributed actor network directly maps local observations to the physical control commands of each UAV, without structured dynamics filtering. During training and inference, the policy network output is the final control quantity, without including analytical solution steps based on the dynamics model; otherwise, it is the same as in Example 1. Example 6

[0082] The MAPPO-Acados method is used instead of the Riccati back-recursive analytical solution for the inner-layer differentiable MPC in Example 1. Specifically, the inner-layer model predictive control solver of MAPPO-Acados uses the general nonlinear optimization solver Acados, which solves the MPC problem in real time using iterative numerical methods such as sequential quadratic programming or interior point methods, and utilizes the solver's built-in implicit differentiation or internal differentiation interface to perform gradient calculation to maintain end-to-end differentiability; the rest is the same as in Example 1. Example 7

[0083] The MAPPO-Theseus method replaces the analytical solution scheme based on Riccati recursion in Example 1. The inner layer of MAPPO-Theseus uses the Theseus, a differentiable MPC solver based on implicit function theorems, to obtain the gradient by implicitly differentiating the optimality conditions (KKT conditions) without relying on the explicit Riccati recursive computation graph; the rest is the same as in Example 1. Example 8

[0084] The MAPPO-General Differentiable MPC method is adopted, in which the inner model predictive control uses a general iterative numerical solver to solve the MPC problem, and differentiation is performed by expanding the computational graph of the entire optimization iteration process. This method does not depend on the analytical structure of a specific problem and has strong versatility, but the size of the computational graph expands cubically with the prediction time domain and the number of decision variables; otherwise, it is the same as in Example 1.

[0085] To verify the technical effectiveness of this embodiment, under the condition that the initial swing angle of the dynamic gate is random, the MAPPO-dMPC method of this invention and the existing methods GART-MAPPO, Dist-Opt, and High-MPC were thoroughly tested. The evaluation indicators were mission success rate, average flight speed, average angular velocity, average jump, average path length, Laplacian cost, root mean square value of formation error, and single-step forward inference delay. Each method was tested independently; the statistical results are shown in Table 1 and [Table data missing]. Figure 5 As shown.

[0086]

[0087] Table 1. Comparison of overall performance of Examples 1 to 4 under the initial swing angle condition of the random dynamic gate. As shown in Table 1, the method in this embodiment achieved a success rate of 99.00% in dynamic gate crossing tasks. Regarding the smoothness of control commands, the average angular velocity of this method is 3.23 rad / s, a reduction of approximately 65.1% compared to GART-MAPPO's 9.25 rad / s; the average jump is 7.50 m / s. 3 Compared to GART-MAPPO's 162.93 m / s 3 The performance reduction was approximately 95.4%. Meanwhile, the average path length, root mean square value of formation error, and Laplacian cost remained at low levels, indicating that this method significantly reduced high-frequency chattering of control commands while maintaining a high mission success rate. The output control quantity conformed to the physical constraints of quadrotor dynamics and possessed high hardware executability. Under different initial swing angles of the dynamic gate, the performance indicators of this method also showed strong stability.

[0088] Furthermore, by Figure 5 As shown, this is a performance trend comparison diagram of the UAV swarm collaborative decision-making method based on differentiable model predictive control under different dynamic gate initial angle conditions in embodiments 1 to 4 of the present invention. When the change in the initial swing angle of the dynamic gate leads to a change in the task difficulty, the present invention maintains a stable trade-off between success rate, path length, angular velocity, jump, and time delay. Although the direct action learning method has a lower forward time delay, the action jump is significantly higher. Traditional optimization or hierarchical control methods are prone to problems of decreased success rate or increased time delay under dynamic obstacle and narrow channel conditions.

[0089] To verify the technical effectiveness of this embodiment, Examples 1 and 5-8 were trained under different prediction time domain conditions H=2, 5, and 10. The total training time, frames processed per second, and single-step forward inference latency were statistically analyzed. The statistical results are shown in Table 2.

[0090] Table 2. Comparison of training efficiency of differentiable predictive control implementation in different prediction time domains for Examples 1 and 5-8. As shown in Table 2, Example 1 exhibits a remarkably gradual increase in training time and computational overhead as the prediction time domain increases. Even with a prediction time domain of H=10, it maintains a training throughput of approximately 7910 frames / second, with a single-step inference latency of only 0.0061 seconds, demonstrating significant computational efficiency advantages and engineering practicality. In contrast, the general solver schemes represented by Examples 6 to 8 show a sharp increase in training time as the prediction time domain increases. Some schemes reach training times of tens or even hundreds of days when H=10, resulting in a significant decrease in throughput and making it difficult to meet the needs of large-scale reinforcement learning training. Although Example 5 has the fastest training speed, it lacks inner-layer physical constraints, resulting in insufficient physical feasibility of control instructions. This comparison fully verifies the outstanding contribution of the present invention's use of analytical Riccati back-recursion as an inner-layer differentiable solver to training efficiency.

[0091] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A collaborative decision-making method for unmanned aerial vehicle (UAV) swarms based on differentiable model predictive control, characterized in that, include: Step S1: Perform dynamic modeling on a distributed drone cluster consisting of several quadcopter drones to obtain a nonlinear dynamic model, and map the nonlinear dynamic model to the impulse space to obtain a linear dynamic model. Step S2: Construct a two-layer optimization framework with MAPPO as the outer framework and differentiable MPC as the inner framework. Step S3: Map the potential parameters determined based on the local observations of the quadcopter UAV into a lower triangular matrix, construct a symmetric positive definite SPD cost matrix based on the lower triangular matrix, and determine the dynamic reference trajectory based on the SPD cost matrix and the collaborative task of the UAV swarm. Step S4: The MPC problem of the inner framework is transformed into an LQR problem, and the optimal impulse is obtained by solving the closed analytical solution through Riccati back recursion. Step S5: The optimal impulse is mapped into physical control commands that can be executed by the quadcopter UAV through geometric control.

2. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 1, characterized in that, In the nonlinear dynamic model in step S1: The state of each quadcopter UAV is defined on the manifold ; ； ； in, Position of the inertial frame, For the linear velocity of the inertial frame, The pose is represented by a unit quaternion. For total thrust, For volume coordinates, angular velocity To control the input, It is a state vector; The discrete-time nonlinear dynamic model is as follows: ； in, For the mass of quadcopter drones, The gravity vector This is the rotation matrix corresponding to the quaternion. The thrust axis unit vector, This is quaternion multiplication. The sampling interval is denoted as .

3. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 2, characterized in that, In the linear dynamic model in step S1: Planning state is defined as ; The pseudo-control input is impulse. ; The discrete-time linear dynamic model is as follows: ， Wherein, the state transition matrix Input matrix They are respectively: 。 4. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 3, characterized in that, The mathematical form of the two-layer optimization framework is: ； in For policy network parameters, For high-level intent parameters, Let be the cost function of the LQR of the inner framework. To predict the time domain, This represents the mapping relationship for differentiable MPCs.

5. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 4, characterized in that, Step S3 includes: Step S31: Using MLP as a distributed actor network, input the local observations of each quadcopter UAV and output the latent parameters, wherein the activation functions are Mish and LayerNorm; Step S32: Map the potential parameters to the lower triangular matrix; Step S33, through Construct the SPD cost matrix, where L is the lower triangular matrix. For the SPD cost matrix, the diagonal elements of L are obtained through... The mapping is a linear mapping of the off-diagonal elements of L to positive real numbers, used to make Strictly positive definite; For numerically stable terms, For potential parameters; Step S34: Based on the SPD cost matrix and the collaborative task of the UAV swarm, the distributed actor network outputs three sets of SPD weight matrices corresponding to the collaborative task, respectively. The SPD weight matrix includes... ; Step S35: Based on the SPD cost matrix, synthesize and determine the total cost matrix: ； Step S36: Determine the dynamic reference trajectory by weighted averaging based on the SPD weight matrix: 。 6. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 5, characterized in that, Step S4 includes: Step S41, define the TV-LQR cost function; Step S42: Starting from the terminal cost, calculate the optimal gain and cost-to-deliver matrix based on Riccati's reverse iteration. Step S43: Based on the optimal gain and the cost-to-delivery matrix, determine the optimal impulse using an affine feedback law.

7. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 6, characterized in that, The local observations of the quadcopter UAV include: its own motion status, neighbor status, formation error, obstacle status, and endpoint information.

8. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 7, characterized in that, Step S5 includes: Step S51: Integrate the optimal impulse to obtain the optimal acceleration; Step S52: Obtain the desired thrust vector after compensating for gravity with the optimal acceleration, and take the Euclidean norm as the total thrust. Step S53: Align the z-axis of the body coordinate system with the normalized desired thrust vector, construct orthogonal body coordinate system axes based on the reference yaw angle, and obtain the desired rotation matrix; Step S54, in the special orthogonal group The geometric attitude controller is designed on the upper part, and the angular velocity of the body coordinate system is determined based on the attitude error quaternion; Step S55: Determine the physical control command based on the total thrust and the angular velocity of the body coordinate system.

9. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 8, characterized in that, Also includes: Step S6 involves using Riccati's backward recursion as an explicit computation graph, deriving gradients using the envelope theorem, and then training the policy network by propagating the gradients from the physical control commands to the policy network parameters of the outer framework through end-to-end gradient propagation. This includes: Step S61: Perform gradient decomposition based on the chain rule to determine the gradient of the policy loss with respect to the network parameters. Step S62: Use the envelope theorem to simplify the differential of the Riccati equation and obtain the differential relationship between the cost and the transition matrix. The spectral radius of the closed-loop state transition matrix is ​​less than 1, which is used to suppress gradient explosion. Step S63: Using the MAPPO pruning alternative target optimization policy network, the advantage value is obtained based on generalized advantage estimation, and the total loss function is determined.

10. The UAV swarm cooperative decision-making method based on differentiable model predictive control according to claim 9, characterized in that, In step S63, the total loss function includes the policy loss, the squared error loss of the value function, and an entropy regularization term, which is used to prevent premature convergence of the policy. Gaussian sampling is used during the training of the policy network, and Gaussian noise is added to the analytical solution.

Images