A safe multi-uav cooperative formation method combining physical information neural network, reinforcement learning and model predictive control

Abstract

The application relates to the field of unmanned aerial vehicle control and discloses a safe multi-unmanned aerial vehicle cooperative formation method combining physical information neural networks, reinforcement learning and model predictive control, which comprises a physical information neural prediction formation control framework, integrates physical information neural networks, reinforcement learning and model predictive control, directly embeds unmanned aerial vehicle dynamics and safety constraints into a neural architecture, eliminates the need for manual parameter adjustment, prevents gradient conflicts by introducing physical laws as internal components, establishes a loss function priority hierarchy with safety first, combines the adaptive ability of DDPG reinforcement learning and the optimality and safety constraint guarantee of MPC, realizes safe cooperative formation of distributed multi-unmanned aerial vehicles in a dynamic obstacle environment, does not need to re-adjust parameters in different obstacle density scenes, has excellent formation keeping ability, extremely low near-collision rate and good scene adaptability.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control, specifically to a safe multi-UAV cooperative formation method that combines physical information neural networks, reinforcement learning, and model predictive control. Background Technology

[0002] In obstacle-filled environments, multi-UAV (Unmanned Aerial Vehicle) formation control is a fundamental technical challenge in the field of autonomous systems. The core requirement is to coordinate the UAV team to maintain the desired geometric configuration while ensuring its safe operation in dynamic environments. This technology has important application value in scenarios such as aerial-to-ground vision, search and rescue, and autonomous transportation.

[0003] Existing multi-UAV formation control methods can be mainly divided into three categories: classical formation control methods (consensus algorithms, leader-follower methods), which are suitable for known dynamics in simple environments but struggle to cope with dynamic obstacles; graph theory extension methods and orientation-based design schemes, which improve robustness but lack safety guarantees in environments with perception delays and dynamic clutter; and model predictive control (MPC) methods, which balance formation objectives and collision avoidance through constraint optimization, but suffer from model mismatch, difficulty in scaling up computations for dense formations, and extreme sensitivity to weight parameters. Furthermore, reinforcement learning (RL) methods are adaptable to unmodeled dynamics, and safe RL combines obstacle functions and conservative policies, but these learning methods require precise reward calibration and lack safety guarantees during the exploration process.

[0004] Existing technologies generally suffer from two key limitations that hinder practical deployment:

[0005] ① The parameters are highly sensitive: When maintaining the formation and avoiding collisions, the weight parameters need to be manually adjusted for specific scenarios. Parameters that are effective in sparse environments may lead to overly conservative or unsafe behavior of drones in dense environments.

[0006] ② Gradient conflict: When optimizing multiple competing objectives such as formation maintenance and collision avoidance at the same time, the gradient direction of tightening the formation is opposite to the gradient direction of maintaining safety, which leads to instability in the learning process and suboptimal control performance. Summary of the Invention

[0007] The purpose of this invention is to provide a safe multi-UAV cooperative formation method that combines physical information neural networks, reinforcement learning, and model predictive control to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control. This method employs the Physical Information Neural Network (PINN) predictive formation control (PINPFC) framework, which integrates the Physical Information Neural Network (PINN), DDPG-based reinforcement learning, and model predictive control (MPC) in a distributed manner. Each UAV independently performs control calculations based on local observations and neighbor communication. Specifically, the method includes the following steps:

[0009] Step S1, Initialization: Set up the PINN network, the Actor-Critic network of DDPG, the MPC solver and the inter-UAV communication graph, input the initial state of the UAV and the formation offset, and configure the reference trajectory for the navigator UAV.

[0010] Step S2, Parallel Operations within a Time Step: In each control time step, all UAVs synchronously perform information interaction, desired position calculation, control generation, and learning processes; Information Interaction: Update their own local observation state vector, detect obstacles within the sensor range, and exchange state and threat information with neighboring UAVs; Desired Position Calculation: The desired position of the leader UAV is used as the reference trajectory, and the follower UAVs calculate their desired positions based on neighbor information and preset formation offsets; Control Generation: Use the multi-branch PINN network to generate physical information control adjustment quantities, use the DDPG Actor network to generate RL control adjustment quantities, fuse the two types of adjustment quantities to obtain the modified target position, input it into the MPC solver to solve the optimization problem, and output the final control input; Learning Process: Update the weights of the DDPG Critic and Actor networks, update the weights of the PINN network, and dynamically adjust the learning rate based on the near-near collision rate;

[0011] Step S3: Repeat step S2 until the multi-drone formation flight mission is completed.

[0012] Preferably, the PINN employs a multi-branch network design, receiving the UAV's local observation state vector as input and generating a control adjustment vector as output. The local observation state vector includes the UAV's current position, velocity, formation error, and threat information. The multi-branch network first processes the input local observation state vector through a shared encoder, then processes the encoded state into three dedicated branches: a dynamics branch, a safety branch, and a formation branch. Simultaneously, an attention mechanism is configured to dynamically weight the contribution of each branch. The weighted output is processed by an activation function to generate the final control adjustment, ensuring the boundedness of the control output. Specifically, the dynamics branch captures the UAV's motion pattern, ensuring that the control actions conform to the UAV's physical dynamics characteristics; the safety branch implements collision avoidance functionality, enhancing sensitivity to near-range threats; and the formation branch maintains the UAV's desired formation geometry based on the formation error.

[0013] Preferably, the training process of PINN employs a physical information loss function, which is a hierarchical weighted combination of physical consistency loss, enhanced safety loss, formation consistency loss, and policy consistency loss. The weights establish a priority hierarchy of "safety first," with collision avoidance loss having the highest weight, followed by physical consistency loss and formation maintenance loss. Physical consistency loss ensures that control actions follow UAV dynamics; enhanced safety loss enforces collision avoidance by imposing exponential penalties on approach violations; formation consistency loss maintains the desired formation geometry of the UAV; and policy consistency loss ensures that the control actions generated by PINN are consistent with the RL policy.

[0014] Preferably, in the DDPG-based reinforcement learning, each UAV maintains a local Actor-Critic network; the Actor network maps the local state of the UAV to control adjustment variables, and the Critic network estimates the action value function; the reward function of reinforcement learning is configured to balance formation maintenance, collision avoidance safety and energy efficiency objectives, providing positive reinforcement for collision-free operation, penalizing formation error and control output, and rewarding progress toward the formation objective.

[0015] Preferably, the model predictive control (MPC) operates based on the fused modified target position. After defining the total position error, an MPC optimization problem is constructed and solved. The objective of the MPC optimization problem is to balance tracking performance and control consumption, while satisfying the dynamic constraints, safety constraints, and formation constraints of the UAV.

[0016] Preferably, the dynamic modulation learning rate is implemented as follows: the real-time learning rate is calculated based on the base learning rate of different network components, the security sensitivity coefficient, and the recent near collision rate; the near collision rate is calculated within 50-200 time steps and a minimum lower bound is set to ensure the continuous adaptive capability of the system; the security sensitivity coefficient controls the decay of the learning rate during periods when the system is insecure.

[0017] Preferably, the local observation state vector of the UAV includes the UAV's own position, speed, desired position, desired speed, the position of the nearest threat, threat type, threat distance, threat radius, and formation error; the local knowledge set of the UAV includes the set of neighboring UAVs, the set of local obstacles, and the preset desired formation relative position offset.

[0018] According to the above-mentioned safe multi-UAV cooperative formation method that combines physical information neural networks, reinforcement learning, and model predictive control, the communication topology of the multi-UAVs adopts a leader-follower pattern; the formation geometry is configured according to requirements, adopting a rhombus or other preset structure.

[0019] Compared with the prior art, the beneficial effects of this invention are as follows:

[0020] Addressing the issues of parameter sensitivity and gradient conflict: By combining PINN, RL, and MPC through the unified PINPFC framework, UAV dynamics and safety constraints are directly embedded into the neural architecture, completely eliminating the need for manual parameter adjustment; Physical laws are introduced as intrinsic components of the network rather than external penalty terms, effectively preventing gradient conflicts during multi-objective optimization and improving the stability and control performance of the learning process.

[0021] Ensuring physical consistency: Design a physical information neural architecture that embeds UAV dynamics, collision avoidance mechanisms, and formation requirements into a differentiable loss function. Through training with a hierarchically weighted total loss function, strict physical consistency is ensured throughout the network learning process, and control actions always follow the laws of UAV dynamics.

[0022] Scene adaptability without parameter tuning: In obstacle scenarios with different densities, it can achieve excellent formation control and safe collision avoidance performance without reading any parameters, solving the problem of repeated manual parameter tuning required by existing methods in different scenarios, and greatly improving the convenience of actual deployment.

[0023] Distributed operation and robustness: The method operates in a distributed manner, with each UAV completing control calculations based solely on local observations and neighbor communication, eliminating the need for global information, thus reducing the communication burden and improving the system's robustness to single UAV failures; MPC, as the final decision-making layer, provides hard safety constraints to prevent safety violations during the exploration process;

[0024] Excellent overall performance: Experimental results show that this method has an extremely low near-collision rate in dynamic obstacle environments, maintains a very low root mean square formation error, and the formation can quickly recover its configuration after obstacle avoidance maneuvers. At the same time, it controls energy consumption reasonably and has a considerable mission success rate under stringent evaluation criteria. Attached Figure Description

[0025] Figure 1 This is a flowchart of the present invention;

[0026] Figure 2 Three-dimensional flight trajectory diagrams of four UAVs generated by the PINPFC method of this invention under 15 obstacle scenarios;

[0027] Figure 3 The diagram shows the UAV formation configuration and obstacle distribution of the PINPFC method of this invention at key time points (t=4.1s, 28.7s, 45.1s, 50.5s) in 15 obstacle scenarios.

[0028] Figure 4The loss convergence curves of the PINN network in the PINPFC method of this invention are shown for 15 obstacle scenarios, including single UAV loss and average loss.

[0029] Figure 5 The convergence curves of the reward function of DDPG reinforcement learning in the PINPFC method of this invention are shown for 15 obstacle scenarios, including single UAV reward and average reward.

[0030] Figure 6 The figures show the formation error variation curves of the PINPFC method of this invention under 15 obstacle scenarios, including single UAV formation error and average formation error.

[0031] Explanation of reference numerals in the attached figures: Figure 2 In the diagram, ReferenceTrajectory represents the navigator's reference trajectory, and UAV1-UAV4 represent four drones. Figure 3 In the diagram, X(m) represents the X-axis coordinate in three-dimensional space, and different colors indicate different drones and obstacles. Figure 4-6 In this context, Average is the average metric for all drones, TrainingStep is the training step size, Episode is the training round, and TimeStep is the control time step. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example

[0034] Please see Figure 1 The diagram illustrates a safe multi-UAV cooperative formation method that combines physical information neural networks, reinforcement learning, and model predictive control. It employs the PINPFC (Physical Information Neural Network Predictive Formation Control) framework, which integrates the PINN, DDPG-based reinforcement learning, and MPC (Model Predictive Control) in a distributed manner. Each UAV independently performs control calculations based on local observations and neighbor communication. The specific steps include:

[0035] Step S1, Initialization: Set up the PINN network, the Actor-Critic network of DDPG, the MPC solver and the inter-UAV communication graph, input the initial state of the UAV and the formation offset, and configure the reference trajectory for the navigator UAV.

[0036] Step S2, Parallel Operations within a Time Step: In each control time step, all UAVs synchronously perform information interaction, desired position calculation, control generation, and learning processes; Information Interaction: Update their own local observation state vector, detect obstacles within the sensor range, and exchange state and threat information with neighboring UAVs; Desired Position Calculation: The desired position of the leader UAV is used as the reference trajectory, and the follower UAVs calculate their desired positions based on neighbor information and preset formation offsets; Control Generation: Use the multi-branch PINN network to generate physical information control adjustment quantities, use the DDPG Actor network to generate RL control adjustment quantities, fuse the two types of adjustment quantities to obtain the modified target position, input it into the MPC solver to solve the optimization problem, and output the final control input; Learning Process: Update the weights of the DDPG Critic and Actor networks, update the weights of the PINN network, and dynamically adjust the learning rate based on the near-near collision rate;

[0037] Step S3: Repeat step S2 until the multi-drone formation flight mission is completed.

[0038] Consider a The system, consisting of autonomous unmanned aerial vehicles, has its state vector configured as follows: ,in Indicates spatial location, Indicates airspeed, and These represent the track inclination angle and track azimuth angle, respectively.

[0039] The nonlinear dynamics model of the UAV satisfies the following equations:

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046] in, To control the input vector, Indicates aerodynamic drag. Representing external disturbances, the local knowledge set of each drone Configured as follows:

[0047]

[0048] in, Indicates the mass of drone i;

[0049] Represents the state vector of the neighboring drone i;

[0050] Information about obstacles is provided, where m is a parameter representing the number of obstacles;

[0051]

[0052] make Indicates the first The drone relative to the reference trajectory The specified position offset, It is the dimension symbol. This indicates that the variable is Vehicular drones and drones The expected relative formation vector between them is defined as ,satisfy:

[0053]

[0054] in, For adjacent drone groups, It is a collection of local obstacles. Includes the preset expected formation relative position offset, expected position The update logic is calculated as follows:

[0055]

[0056] Where t represents time;

[0057] Location information for reference trajectory;

[0058] This invention addresses a multi-objective optimization challenge by balancing three core objectives: formation maintenance, safety, and energy efficiency. The cost functions for each objective are configured as follows:

[0059]

[0060]

[0061]

[0062] in, This indicates the location information of drone i; Indicates the desired location of drone i; This indicates the cost of maintaining the formation; This indicates the cost of maintaining security; This indicates the energy required for control;

[0063] in, UAV based on relative distance quantization The risk of collision with adjacent drones, Multi-objective optimization is used to measure the risk of collision with obstacles, and its purpose is to find the optimal control strategy. :

[0064]

[0065] Representative seeking The maximum value of this function; Representative seeking The mean of this function; Represents the reward function; The strategy represented by the choice;

[0066] The multi-objective reward function is defined as follows: ,and The weighting coefficients are used to configure the system's overall performance evaluation index as the root mean square formation error (RMSE). ) and near-collision rate ( ):

[0067]

[0068]

[0069] in, Indicates at time drones Minimum distance to any obstacle Indicates the current running time step. Indicates the number of drones. This indicates the set safe distance.

[0070] The method proposed in this invention combines the learning ability of neural networks with domain knowledge constraints, thereby eliminating the need for manual hyperparameter tuning while ensuring the safe operation of the system.

[0071] At the heart of PINPFC lies its Physical Information Neural Network (PINN), which is configured to directly incorporate domain knowledge about UAV dynamics, safety constraints, and formation requirements into the learning process. Unlike traditional neural networks that learn purely from data, the PINN architecture explicitly enforces physical laws through a specific network design and loss function. The PINN architecture employs a multi-branch design, configured to capture different feature dimensions of the formation control problem, and the network receives local observation state vectors. As input, and generate control adjustment vectors As output, the state vector contains the current position, velocity, formation error, and threat information. The input state is first processed by a shared encoder.

[0072]

[0073] It is a local observation state vector. These are the parameters of the neural network;

[0074] Subsequently, the encoded state is processed by three dedicated branches:

[0075] 1) DynamicsBranch: Configured to capture the drone's motion patterns and ensure that control actions conform to the physical dynamics characteristics of the drone's nonlinear dynamics model.

[0076]

[0077] For the input of the shared encoder, These are the parameters of the neural network;

[0078] 2) Safety Branch: Focuses on collision avoidance capabilities, configured to enhance sensitivity to close-range threats.

[0079]

[0080] For the input of the shared encoder, These are the parameters of the neural network;

[0081] 3) Formation Branch: Configured to maintain the desired formation geometry based on formation error.

[0082]

[0083] For the input of the shared encoder, For the parameters of the neural network

[0084] Furthermore, an attention mechanism is configured to dynamically weight the contributions of each branch:

[0085]

[0086] in, This represents the attention weight vector, where the sum of its elements is 1. The weighted outputs are combined to generate the final control adjustment.

[0087]

[0088] in, The activation function is configured to ensure the boundedness of the control output. These are the parameters of the neural network.

[0089] The training process of PINN includes several physical information loss components, configured to directly encode domain knowledge into the learning process: 1) Physical consistency loss ( ): Configured to ensure control actions follow UAV dynamics:

[0090]

[0091] in, For discrete time steps, As the acceleration constraint weight, and The maximum permissible acceleration amplitude, For activation functions;

[0092] 2) Enhanced safety loss ( ): Configured to enforce collision avoidance by imposing exponential penalties on near-miss violations:

[0093]

[0094] For the safe radius of the drone body, For the safety factor related to collisions between drones, The safety factor related to collisions with obstacles.

[0095] 3) Formation consistency loss ( ): Configured to maintain the desired formation geometry:

[0096]

[0097] in, For formation offset The desired location;

[0098] 4) Strategy consistency loss ): Configured to ensure that the control actions generated by PINN are consistent with the RL policy, thereby encouraging smooth cooperation between physical information components and data-driven components and preventing policy divergence:

[0099]

[0100] Actions generated by the PINN network This represents the action generated by reinforcement learning.

[0101] PINN's total loss function is configured as a hierarchical weighted combination of its components:

[0102]

[0103] in, , , and These are weighting coefficients for the corresponding losses that can be set by the user. The weights establish a "safety first" priority hierarchy, where collision avoidance is given the highest priority, followed by physical consistency and formation maintenance.

[0104] The reinforcement learning component employs the Deep Deterministic Policy Gradient (DDPG) algorithm, suitable for distributed formation control. Each UAV maintains a local Actor-Critic network, configured to learn the optimal policy based on local observations. The Actor network maps local states to control adjustments.

[0105]

[0106] It is a local observation state vector. Parameters of the Actor network

[0107] The Critic network is configured to estimate the action value function:

[0108]

[0109] in, As a discount factor, For information about historical actions, This represents the historical value that maximizes the Critic network value, and its reward function is configured to balance multiple objectives:

[0110]

[0111] in, Provides positive reinforcement for collision-free operation. This indicates the time from the start of operation to the current moment. Indicators of whether a collision has occurred, and when a collision occurs. When no collision occurs , and These are the penalty coefficients for formation error and control output, respectively. Used to reward progress towards the formation objective. It indicates the degree of completion of the entire path.

[0112] The MPC component is configured to provide optimality guarantees while integrating recommended actions from the PINN and RL modules. MPC runs based on the modified target location.

[0113]

[0114] For the desired position, To what extent should PINN's recommended output actions be considered, and what are the customizable variables? To what extent should RL recommend output actions be considered, a customizable variable can be set.

[0115] Therefore, the total position error is defined as:

[0116]

[0117] The MPC optimization problem is conceived as solving the following objective:

[0118]

[0119]

[0120]

[0121]

[0122]

[0123] in, For predicting the step size (horizon). and This is a weight matrix used to balance tracking performance and control overhead.

[0124] The PINPFC system employs a continuous learning mechanism with adaptive safety constraints, and its learning rate is dynamically modulated based on safety performance.

[0125]

[0126] in, This represents the base learning rate for different network components. Controlling sensitivity to safety performance, and This indicates the recent near-collision rate.

[0127] Adaptive learning parameters: The base learning rate is configured as follows: Security sensitivity Configured to control the decay of the learning rate during unsafe periods, the near collision rate is calculated over 50-200 time steps, and a minimum lower bound is set. To ensure the system's continuous adaptive capability.

[0128] Algorithm 1 presents the overall execution flow of PINPFC for distributed multi-UAV formation control, which integrates physical information neural networks, reinforcement learning, and model predictive control within a unified framework. The algorithm operates in a distributed manner, where each UAV independently performs control calculations based on local observations and limited neighbor communication.

[0129] Algorithm 1: Physical Information Neural Prediction Formation Control (PINPFC)

[0130] Input: Initial state Formation offset (in ); Reference trajectory (Navigator only).

[0131] Output: All drones control input .

[0132] Step 1: Initialize the PINN, DDPG networks, MPC solver, and communication graph. .

[0133] Step 2: At each time step Inside, each drone performs the following operations in parallel:

[0134] o Information interaction: Updating local state Detection sensor range Obstacles within the area, and exchange information with neighbors;

[0135] o Expected position calculation: If it is the navigator, the expected position is set as the reference trajectory; if it is the follower, the expected position is calculated based on the neighbor information and the expected offset.

[0136] o Controls generation:

[0137] • Generate physical information adjustment amount using multi-branch PINN network ;

[0138] • Generate RL adjustment using the DDPGActor network ;

[0139] • Integrate the above adjustments to generate the modified target position. ;

[0140] Solve the MPC optimization problem and output the final control instructions. ;

[0141] o Learning process (if training is enabled): Update the Critic and Actor networks of DDPG; update the PINN weights; and apply a dynamic adaptive learning rate.

[0142] Its core innovation lies in integrating physical information neural networks, reinforcement learning adaptation, and model predictive control optimization into a unified framework. This integrated approach combines the advantages of physical information learning, reinforcement learning, and optimal control, thereby achieving robust formation control while ensuring safety.

[0143] To verify the effectiveness and superiority of the Physical Information Neural Prediction Formation Control (PINPFC) framework proposed in this invention, comprehensive simulation experiments were conducted in multi-UAV formation flight scenarios with different obstacle densities. (See attached document.) Figures 2-6 The experiment focused on the PINPFC method of this invention, and statistically analyzed multiple performance indicators such as security, formation maintenance capability and computational (energy) efficiency. To ensure statistical significance, 100 test scenarios were randomly generated for Monte Carlo experiments.

[0144] Specific Implementation Environment Setup: In this embodiment, each UAV is modeled as a six-degree-of-freedom (6-DOF) rigid body system incorporating aerodynamic drag and gravity effects. Its key physical parameters are configured as follows: mass... kg, reference wing area m Drone safety radius m, maximum permissible acceleration m / s Maximum speed m / s; communication topology, multi-UAV system ( The inter-agent communication in this system adopts a leader-follower pattern, defined by the following Laplace matrix:

[0145]

[0146] In this configuration, UAV 1 is set as the navigator of the formation. This topology ensures that each follower communicates with its neighboring agents while maintaining the connectivity of the entire formation. The formation geometry, with the desired relative positional offsets between the multiple UAV systems configured as a rhombus structure, is specifically defined as follows:

[0147]

[0148] Runtime parameter configuration, total experiment duration set to s, control update frequency configuration to Hz (i.e., time step) s), the safe distance threshold is set to m, communication range m, sensor detection range m, to rigorously evaluate robustness, introduces within the space A dynamic obstacle, its radius m and speed of motion m / s.

[0149] Training convergence and learning dynamics: The training dynamics of the PINPFC system were analyzed for 15 highly challenging obstacle scenarios.

[0150] The results show that the reinforcement learning (RL) components included in this invention achieve rapid convergence (stabilizing after approximately 20 epochs). However, the PINN loss function exhibits a characteristic spike in the early stages (around the 30th epoch). Detailed component analysis reveals that this spike is primarily attributed to the activation of safety constraints (the safety penalty loss when the network encounters a complex distribution of obstacles). (Significant increase).

[0151] This convergence property proves that the increase in the loss function represents that the system has successfully learned the multi-objective constraint mechanism, rather than that the training is unstable. The system has successfully established a complex logic that can adaptively balance "safety" and "formation maintenance" in complex navigation scenarios.

[0152] In this embodiment, PINPFC conducted 100 independent Monte Carlo tests on a diamond formation of four drones in a three-dimensional environment containing 15 dynamic obstacles. Its core performance indicators are as follows:

[0153] 1. Training convergence and learning dynamics

[0154] • Fast and stable RL rewards: In a complex environment with 15 obstacles, the reinforcement learning (RL) component in the system achieves fast convergence after about 20 training rounds, with the average reward stabilizing at 17.68.

[0155] • Adaptive activation of physical constraints: The loss of the Physical Information Neural Network (PINN) shows a characteristic peak around the 30th round (reaching a maximum of 1244.1, and then stabilizing at around 256.7), indicating that the system successfully activated the safety penalty mechanism when encountering complex obstacle distributions (the safety loss surged from 0.025 to 85.57), perfectly learning how to balance "collision avoidance" and "formation maintenance" in extreme environments.

[0156] 2. Formation maintenance capability

[0157] • Stable overall error: The root mean square formation error of the system during the entire 70-second flight mission ( It was kept under stable control. extremely low levels of m;

[0158] • Controllable maximum deformation: The maximum formation error of the system when performing intensive obstacle avoidance maneuvers ( )for The number m indicates that the formation can quickly recover after flexibly deforming and avoiding obstacles, maintaining good overall cohesion.

[0159] 3. Collision avoidance and safety performance

[0160] • Extremely low near-collision rate: When faced with 15 dynamic obstacles, the near-collision rate (NCR) of PINPFC is only 2.10% (0.0210).

[0161] • Ample safety margin: During flight, the average minimum distance between the drone and any threat (other drones or obstacles). ) Keep at The distance is m, far exceeding the 1.2m safety distance threshold, providing absolute physical safety.

[0162] 4. Task success rate and energy consumption

[0163] • Strict Success Rate Performance: Under the most stringent evaluation criteria that simultaneously require "strict compliance with maximum formation error" and "absolutely no collisions throughout the entire process," PINPFC achieved a strict success rate of 39% in this 15-obstacle scenario. );

[0164] Control energy consumption: The average control energy consumption of the system in completing this scenario task during continuous dynamic obstacle avoidance and formation correction. The value is 17097.0J.

[0165] The key system parameters of this invention include communication range. Sensor range Safe distance and near-collision threshold The neural network parameters are given by the corresponding subscripts. express.

[0166] Regarding the first The local observation state vector of the drone. Configured as follows:

[0167]

[0168] in, Indicates location, Indicates speed, These represent the desired position and desired velocity, respectively. Indicates the location of the most recent threat (obstacle). Indicate the type of threat. For the threat distance, For the threat radius, and This indicates formation error.

[0169] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0170] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control, characterized in that, The Physical Information Neural Prediction Formation Control (PINPFC) framework is adopted, which integrates the Physical Information Neural Network (PINN), DDPG-based reinforcement learning, and Model Predictive Control (MPC). It operates in a distributed manner, with each UAV independently performing control calculations based on local observations and neighbor communication. Specifically, the framework includes the following steps: Step S1, Initialization: Set up the PINN network, the Actor-Critic network of DDPG, the MPC solver and the inter-UAV communication graph, input the initial state of the UAV and the formation offset, and configure the reference trajectory for the navigator UAV. Step S2, Parallel operation within a time step: In each control time step, all UAVs synchronously perform information interaction, desired position calculation, control generation, and learning processes; Information interaction: update their own local observation state vector, detect obstacles within the sensor range, and exchange state and threat information with neighboring UAVs; Desired position calculation: The desired position of the navigator drone is the reference trajectory, and the desired position of the follower drone is calculated based on neighbor information and preset formation offset; Control generation: Physical information control adjustment is generated using a multi-branch PINN network, and RL control adjustment is generated using the Actor network of DDPG. The two types of adjustment are fused to obtain the modified target position, which is then input into the MPC solver to solve the optimization problem and output the final control input. Learning process: Update the weights of the Critic and Actor networks in DDPG, update the weights of the PINN network, and dynamically adjust the learning rate based on the near-near collision rate; Step S3: Repeat step S2 until the multi-drone formation flight mission is completed.

2. The safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control as described in claim 1, characterized in that: The PINN employs a multi-branch network design, receiving a local observation state vector of the UAV as input and generating a control adjustment vector as output. The local observation state vector includes the UAV's current position, velocity, formation error, and threat information. The multi-branch network first processes the input local observation state vector through a shared encoder, then processes the encoded state into three dedicated branches: a dynamics branch, a safety branch, and a formation branch. An attention mechanism is configured to dynamically weight the contribution of each branch. The weighted output is processed by an activation function to generate the final control adjustment, ensuring the boundedness of the control output. Specifically, the dynamics branch captures the UAV's motion patterns, ensuring that the control actions conform to the UAV's physical dynamics characteristics; the safety branch implements collision avoidance functionality, enhancing sensitivity to near-field threats; and the formation branch maintains the desired formation geometry of the UAV based on the formation error.

3. The safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control according to claim 2, characterized in that: The training process of PINN employs a physical information loss function, which is a hierarchical weighted combination of physical consistency loss, enhanced safety loss, formation consistency loss, and policy consistency loss. The weights establish a "safety first" priority hierarchy, with collision avoidance loss having the highest weight, followed by physical consistency loss and formation maintenance loss. Physical consistency loss ensures that control actions follow UAV dynamics; enhanced safety loss enforces collision avoidance by imposing exponential penalties on approach violations; formation consistency loss maintains the desired formation geometry of the UAV; and policy consistency loss ensures that the control actions generated by PINN are consistent with the RL policy.

4. The safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control according to claim 3, characterized in that: In the DDPG-based reinforcement learning, each UAV maintains a local Actor-Critic network; the Actor network maps the local state of the UAV to control adjustments, and the Critic network estimates the action value function; the reward function of reinforcement learning is configured to balance formation maintenance, collision avoidance safety and energy efficiency objectives, providing positive reinforcement for collision-free operation, penalizing formation error and control output, and rewarding progress toward the formation objective.

5. A safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control as described in claim 4, characterized in that: The Model Predictive Control (MPC) operates based on the fused modified target position. After defining the total position error, an MPC optimization problem is constructed and solved. The objective of the MPC optimization problem is to balance tracking performance and control consumption, while satisfying the dynamic constraints, safety constraints, and formation constraints of the UAV.

6. A safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control as described in claim 5, characterized in that: The dynamic modulation learning rate is specifically implemented as follows: the real-time learning rate is calculated based on the base learning rate of different network components, the security performance sensitivity coefficient, and the recent near collision rate; the near collision rate is calculated within 50-200 time steps, and a minimum lower bound is set to ensure the continuous adaptive capability of the system. The safety performance sensitivity coefficient controls the decay of the learning rate when the system is in an unsafe state.

7. A safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control as described in claim 6, characterized in that: The local observation state vector of the UAV includes the UAV's own position, speed, desired position, desired speed, the position of the nearest threat, threat type, threat distance, threat radius, and formation error; the local knowledge set of the UAV includes the set of neighboring UAVs, the set of local obstacles, and the preset desired formation relative position offset.

8. A safe multi-UAV cooperative formation method combining physical information neural networks, reinforcement learning, and model predictive control according to any one of claims 1-7, characterized in that: The communication topology of the multiple UAVs adopts a leader-follower pattern; the formation geometry is configured according to requirements, using a diamond shape or other preset structures.

Images