Multi-unmanned aerial vehicle cooperative navigation method based on deep reinforcement learning

Abstract

The invention relates to the technical field of artificial intelligence and unmanned aerial vehicles, and discloses a multi-unmanned aerial vehicle cooperative navigation method based on deep reinforcement learning, the method is suitable for multi-agent task execution in a complex dynamic environment, the method is constructed by adopting a multi-agent dual deep Q network model, a centralized training and distributed execution architecture is combined, and in a training stage, the multi-agent dual deep Q network model is constructed; through a global state optimization cooperation strategy, each unmanned aerial vehicle autonomously decides according to local observation in an execution stage, and dynamically adjusts an information synchronization frequency based on a communication load sensing mechanism so as to reduce communication bandwidth occupation and improve system real-time performance, and the method also introduces an experience playback and priority sampling mechanism to improve model stability and convergence speed. The system can sense obstacle information, energy consumption level and other unmanned aerial vehicle states in real time, realizes path dynamic optimization and formation keeping, has good task adaptability and expandability, and is suitable for collaborative navigation application of a heterogeneous unmanned aerial vehicle system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and unmanned aerial vehicles, specifically a multi-unmanned aerial vehicle cooperative navigation method based on deep reinforcement learning, suitable for multi-unmanned aerial vehicle formation cooperation, path planning and dynamic obstacle avoidance tasks in complex dynamic environments. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, multi-unmanned aerial vehicle cooperation systems are widely used in various fields, however, multi-unmanned aerial vehicle systems face challenges such as obstacle interference, task coordination and path conflict in complex dynamic environments. To address these issues, methods based on deep reinforcement learning and multi-sensor fusion are commonly used, aiming to improve the autonomy, robustness and overall performance of multi-unmanned aerial vehicle systems in complex environments.

[0003] Patent No. CN118760233A discloses a multi-agent reinforcement learning-based unmanned aerial vehicle cluster path planning method, which uses a partially observable Markov decision process to model cluster cooperation, including obstacle avoidance, target tracking and flight dynamics.

[0004] Patent No. CN119355654A discloses a deep reinforcement learning-based unmanned aerial vehicle cluster path deception method, which enables the unmanned aerial vehicle cluster to deceive radar systems by modifying the flight path.

[0005] Patent No. CN116242364A discloses a multi-unmanned aerial vehicle intelligent navigation method, which uses deep reinforcement learning to model in a three-dimensional simulation environment, combining global and local observation information.

[0006] Patent No. CN117193320A discloses a multi-agent obstacle avoidance navigation control method based on deep reinforcement learning, which uses deep reinforcement learning and attention mechanisms for obstacle detection and feature extraction, optimizing multi-agent cooperative obstacle avoidance and navigation tasks in unknown environments.

[0007] Patent No. CN119002511A discloses a deep reinforcement learning-based multi-task control method for unmanned aerial vehicles, which trains agents in multiple task environments, enabling unmanned aerial vehicles to simultaneously complete point control and obstacle avoidance tasks.

[0008] Patent No. CN118896609A discloses a multi-unmanned aerial vehicle autonomous cooperative navigation method based on graph neural networks, which models the interaction between unmanned aerial vehicles through graph neural networks, combining reinforcement learning to optimize the cooperative navigation performance of unmanned aerial vehicle clusters.

[0009] Similar disclosed patents also include CN113218400A, CN118915818A, CN119376410A, etc. However, the above methods mostly focus on the navigation or obstacle avoidance tasks themselves, lack of real-time coordination strategies between multiple unmanned aerial vehicles in complex dynamic environments, information synchronization mechanisms under bandwidth constraints, and system modeling and optimization control of formation shape maintaining capabilities, making it difficult to meet the real-time and reliability requirements of multi-target tasks in high-density cluster environments. SUMMARY

[0010] The purpose of the present application is to provide a multi-unmanned aerial vehicle cooperative navigation method based on deep reinforcement learning, which solves the problems raised in the background art.

[0011] The method constructs a reinforcement learning model based on MA-DDQN, and each unmanned aerial vehicle can autonomously navigate based on local information in the execution phase, and in the training phase, the global state is summarized in a centralized manner to optimize the overall strategy. At the same time, a communication bandwidth-aware state synchronization mechanism is introduced, which dynamically adjusts the information exchange frequency according to the system bandwidth load to reduce redundant communication and ensure the synchronization of critical data. To support collaborative applications on heterogeneous platforms, the present application introduces "unmanned aerial vehicle category label" as auxiliary input in the state modeling stage, so that different types of unmanned aerial vehicles can train differentiated strategies in a unified framework, improving system adaptability and task completion efficiency.

[0012] The present application effectively reduces the communication burden and modeling complexity while improving the performance of cooperative navigation, and is suitable for multi-unmanned aerial vehicle cooperation scenarios in dynamic, complex, and communication-limited environments, with good scalability and engineering practical value.

[0013] To achieve the above purpose, the present application provides the following technical solutions: a multi-unmanned aerial vehicle cooperative navigation method based on deep reinforcement learning, at least comprising:

[0014] Multi-agent deep reinforcement learning model construction: a deep reinforcement learning model based on MA-DDQN algorithm is established for each unmanned aerial vehicle. The model learns the interaction behavior between the unmanned aerial vehicle and the environment and other unmanned aerial vehicles, and optimizes the cooperative obstacle avoidance and navigation strategy. The model adopts a double network structure, including a main network and a target network, and introduces a delay update mechanism to alleviate the problem of Q value overestimation, thereby improving the stability and convergence speed of training.

[0015] Real-time acquisition of unmanned aerial vehicle state information: through multi-modal sensors such as laser radar, binocular camera, IMU, ultrasonic sensor, etc., real-time perception of environmental information such as obstacle position, speed, size, and state of neighboring unmanned aerial vehicles, and encoding them into a multi-dimensional feature vector input into the deep reinforcement learning model.

[0016] Dynamic obstacle avoidance and formation keeping: During the execution process, the UAVs perceive the obstacle information in the surrounding environment through sensors and calculate the optimal obstacle avoidance strategy according to the deep reinforcement learning model to avoid collisions with obstacles or other UAVs, while ensuring the smooth progress of the flight task; each UAV dynamically adjusts the flight path and relative position according to the task requirements and environmental changes to ensure the collaboration and flight formation stability of the multi-UAV system.

[0017] Centralized training and distributed execution framework, in the training stage, collect the global state information of multiple UAVs, optimize the overall cooperation strategy; in the execution stage, each UAV makes action decision based on local observation information, realizes the balance between individual autonomy and system cooperation optimization.

[0018] Experience replay mechanism, build experience replay buffer, store interaction data (state, action, reward, next state), use priority experience replay strategy, improve sample utilization efficiency, reduce training sample correlation, enhance the generalization ability of the strategy.

[0019] According to the above scheme, the multi-agent deep reinforcement learning model mainly includes multi-agent model construction, state space design, action space design, and reward function design.

[0020] The multi-agent model includes an input layer for receiving environmental state information, including UAV position coordinates (x, y, z), velocity (v), heading angle (θ), remaining energy (E), relative position and size information of surrounding obstacles, and relative position and velocity of adjacent UAVs; a hidden layer composed of 3 convolutional layers and 2 fully connected layers, each convolutional layer containing 32, 64 and 128 filters, respectively, with batch normalization and ReLU activation function after each convolutional layer, for extracting environmental state features and performing nonlinear feature mapping, the fully connected layer contains 256 and 128 neurons respectively, to further improve the fitting ability of the model; a Q value output layer that outputs Q values corresponding to each action to guide action selection. The MA-DDQN model uses a double network structure, i.e. the main network (Online Network) for decision-making and the target network (Target Network) for stable training, which reduces the variance and bias of Q value estimation through a delayed update mechanism.

[0021] State space design, the UAVs obtain environmental information in real time through the multi-modal sensors mounted on them, and construct a state vector S, including but not limited to their own position, velocity, heading angle, remaining energy; relative position and size of the nearest N obstacles; relative position and velocity of the nearest M adjacent UAVs.

[0022] Action space design, the human-machine action space A includes two types of discrete actions and continuous actions. Discrete actions include rising, falling, left turning, right turning, advancing, retreating, hovering; continuous actions include speed increment (Δv), heading angle adjustment (Δθ), height adjustment (Δz).

[0023] Reward function design, guide the UAV to complete the obstacle avoidance task, design a multi-objective weighted reward function: R = ω1R goal + ω2R safe + ω3R energy + ω4R formation ;

[0024] Wherein: R goal is the task completion reward, the reward is positive when the UAV approaches the target point, and the reward is negative when the UAV moves away from the target point;

[0025] R safe is the safety reward, the reward is zero when no collision occurs, and a penalty is given when a collision occurs;

[0026] R energy is the energy consumption reward, the reward per unit time is -α×energy consumption;

[0027] R formation is the formation keeping reward, the reward is positive when the preset formation is maintained, and a penalty is given according to the error when the formation deviates.

[0028] The weight coefficients ω1, ω2, ω3, ω4 are the weights of the target, obstacle avoidance, safety, energy consumption and formation reward, respectively, and the value range is 0-1, which can be adjusted according to the specific task scene to balance the task completion efficiency, safety, energy consumption and formation requirements.

[0029] In the reward function:

[0030] The task completion reward R goal , the distance d of the target point is calculated by the formula:

[0031] R goal = C1-k1×d;

[0032] Wherein C1 is the target point reward base, and k1 is the distance penalty coefficient;

[0033] The calculation formula of the safety reward R safe is:

[0034]

[0035] The calculation formula of the energy consumption reward R energy is:

[0036] R energy = -β×E consume ;

[0037] wherein E consume is the energy consumption per unit time;

[0038] Formation keeping reward R formation is calculated as follows:

[0039] R formation = -k2 x D formation ;

[0040] wherein D formation is the error distance of the UAV relative to the preset formation.

[0041] According to the above scheme, the real-time acquisition of the UAV information refers to that the UAV perceives the surrounding environment and its own state in real time through the multi-modal sensor system carried by the UAV, including but not limited to a laser radar, a binocular camera, an infrared imaging device, an ultrasonic sensor, an IMU (inertial measurement unit), a GPS module, etc., and acquires state information including but not limited to the following contents:

[0042] Self-state information: including the current three-dimensional position coordinates (x, y, z) of the UAV, the speed v, the heading angle θ, the acceleration, the remaining power E, etc.

[0043] Environmental state information: including the spatial position, size, speed and dynamic attribute (such as moving direction, acceleration) of the surrounding obstacles;

[0044] Adjacent UAV state information: including the relative position, relative speed, formation position deviation, energy consumption state, etc.

[0045] Task-related information: including the target point position, path planning reference information, task constraint conditions, etc.

[0046] After the state information is collected by the sensor, it is preprocessed and encoded by the data processing module to form a high-dimensional state vector St, which is input into the MA-DDQN to generate real-time obstacle avoidance and path optimization decisions. The UAV can upload part of the state information to the central training server through the wireless communication module to realize global information sharing in the centralized training stage; in the distributed execution stage, each UAV makes independent decisions based on local observation state information to realize autonomous obstacle avoidance and path optimization.

[0047] According to the above scheme, the present application introduces the optimization design of dynamic obstacle avoidance and formation keeping in the multi-UAV cooperative task. Each UAV not only needs to avoid collision with obstacles during flight, but also needs to maintain appropriate formation and relative position with other UAVs according to the task requirements. Specifically as follows:

[0048] Dynamic obstacle avoidance: Each drone perceives obstacles in the surrounding environment in real-time through sensors such as lidar, cameras, etc. Based on a deep reinforcement learning model, it calculates the optimal obstacle avoidance strategy. The model adjusts the flight path according to environmental changes and task requirements, avoiding obstacles or other drones to ensure flight safety and task smoothness. The deep reinforcement learning algorithm continuously optimizes obstacle avoidance decisions through real-time feedback, enabling the system to respond to complex changes in dynamic environments.

[0049] Formation maintenance: In multi-drone tasks, maintaining a reasonable flight formation is crucial for the efficiency of collaborative tasks. Through the formation maintenance reward (R formation ) in the multi-objective reward function, each drone can adjust its position and heading during flight to maintain its relative position with other drones. Formation maintenance not only optimizes task execution efficiency, but also ensures coordination and stability during flight. When encountering environmental changes or task adjustments, the system can dynamically adjust the formation to avoid formation disruption due to obstacles or changes in flight path.

[0050] According to the above scheme, the centralized training and distributed execution framework refers to separating the training and execution processes of multiple agents to effectively combine collaborative strategy learning and independent decision execution. Specifically, the following steps and mechanisms are included:

[0051] Centralized training phase: All drones upload their local observation state information (including position, velocity, heading angle, obstacle information, neighboring drone state, etc.) to the central training server through the wireless communication network; the central training server aggregates the global state information, constructs a joint state space as the training input; the training goal is to optimize the collaborative obstacle avoidance strategy of the multi-drone system, generating a globally optimal Q-value function or strategy function, and optimizing the decision network parameters of each drone through backpropagation algorithm and gradient update rule; during training, the experience replay mechanism is combined to store the interaction experience (state, action, reward, next state) of each drone for batch updates, improving the diversity and stability of training samples; the optimized strategy parameters are synchronized back to the local models of each drone to implement strategy distribution.

[0052] Distributed execution phase: Each drone independently makes obstacle avoidance and path optimization decisions based on its local deep reinforcement learning model (MA-DDQN network) during task execution; it does not need to rely on real-time global information, and each drone can rely on the strategy learned during the training phase to autonomously implement obstacle avoidance and collaborative flight; drones can exchange some key flight state information (such as position, velocity, formation deviation, etc.) through wireless communication modules to maintain system collaboration to some extent, but do not need to rely on global synchronous decision-making.

[0053] To further improve the communication efficiency of the distributed execution stage, the application introduces a communication bandwidth-aware state synchronization mechanism in the UAV system. Based on the preset communication load threshold, the state information interaction frequency between UAVs is dynamically adjusted. When the system detects that the current communication bandwidth usage rate exceeds the set threshold (for example, 80%), each UAV switches to the "compressed synchronization mode" and only transmits key state variables (such as position information, heading angle, etc.), and omits secondary information (such as energy consumption, power, etc.), thereby reducing the communication load and ensuring the timeliness of the control information and the responsiveness of the navigation model.

[0054] Through centralized training, global information can be fully utilized to improve the comprehensiveness and diversity of training samples, avoid local optimization, reduce communication load, improve system real-time performance and response ability, and ensure the independence and flexibility of each UAV. Training and execution are decoupled, which facilitates system expansion and adapts to different scales of multi-UAV formation tasks.

[0055] The application is suitable for homogeneous and heterogeneous multi-UAV cooperative tasks, especially in scenarios where there are mixed formations of different types of UAVs (such as fixed-wing, quadcopter, and multi-rotor). The system has good adaptability. To adapt to the differences in maneuverability, perception range, and energy consumption characteristics of various UAVs, the application introduces a "UAV type label" as an additional feature dimension in the state vector during centralized training. This label participates in the input encoding process of the model, allowing each type of aircraft to learn individualized strategies tailored to its own characteristics.

[0056] In addition, during the training process, parameter fine-tuning strategies are used for different models. Based on the shared backbone network parameters, the response ability of specific types of UAVs to key task indicators (such as obstacle avoidance performance or energy efficiency) is enhanced through sub-network adjustment. Experimental verification shows that this heterogeneous support mechanism can significantly improve the stability and task completion rate of the system under mixed formation.

[0057] According to the above scheme, the experience replay mechanism refers to a historical interaction data storage and reuse method introduced in the multi-agent deep reinforcement learning training process to improve training efficiency, reduce sample correlation, and improve model convergence speed and stability. Specifically, it includes the following contents:

[0058] Experience replay buffer construction: In the training phase, an independent or shared experience replay buffer (Replay Buffer) is configured for each UAV or the entire multi-agent system to store the four-tuple data generated during interaction with the environment:

[0059] (s t ,a t ,r t ,st+1 );

[0060] where s t is the current state, a t is the current action, r t is the reward obtained, s t+1 is the next state.

[0061] Sample storage and update: whenever the UAV performs an action and obtains feedback, the corresponding interaction quadruple is stored in the experience replay buffer in real time. The buffer capacity is set to a fixed size, and when the buffer is full, the first-in-first-out (FIFO) strategy or a priority-based sample replacement mechanism is used to update the stored data.

[0062] Sample sampling and training: during the model training process, batch samples are randomly or based on priority from the experience replay buffer at a certain frequency for network parameter update. Random sampling: uniformly random training samples are drawn from the buffer to reduce the temporal correlation between data. Prioritized experience replay: the priority of the sample is calculated according to the TD error, and high TD error samples are preferentially selected for training, thereby accelerating the learning progress and improving the strategy performance.

[0063] TD error calculation and priority assignment:

[0064] The TD error calculation method is: delta = |r t + gamma * Q target (s t+1 ,a') - Q(s t ,a t |;

[0065] where gamma is the discount factor, Q tartget is the target Q network output, and delta represents the importance indicator of the sample. According to the size of delta, the priority weight of the sample is assigned to guide the sampling probability.

[0066] The present application provides a multi-UAV cooperative navigation method based on deep reinforcement learning. The multi-UAV cooperative navigation method based on deep reinforcement learning has the following beneficial effects:

[0067] (1) The present application significantly improves the autonomous decision-making ability and system robustness of the multi-UAV system in complex dynamic environments. By constructing a deep reinforcement learning model based on MA-DDQN, each UAV can make efficient obstacle avoidance and path optimization decisions based only on local observation information, has good scalability, supports large-scale heterogeneous UAV formation cooperative tasks, adopts a centralized training and distributed execution architecture, and balances global strategy optimization and local real-time response, while maintaining the overall collaborative performance of the system, and improving the independent adaptability of each agent in actual flight tasks.

[0068] (2) Fusion multi-modal state perception mechanism (including laser radar, IMU, camera sensor data) and experience replay mechanism and priority sampling strategy, effectively improve the utilization rate of training samples and the convergence speed of strategy network, so as to enhance the stability and generalization ability of the model, introduce the state synchronization mechanism of communication bandwidth perception, dynamically adjust the information exchange strategy according to the network load, effectively reduce the frequency of system redundant communication, guarantee the synchronization of key state data, and improve the real-time collaboration ability of multi-agent system. By designing a multi-objective weighted reward function, balance between task completion rate, safe obstacle avoidance, energy consumption control and formation keeping and other performance indicators is achieved, so as to comprehensively improve the comprehensive task completion ability of the system in complex environment. BRIEF DESCRIPTION OF DRAWINGS

[0069] Fig. 1 It is a whole structure schematic diagram of the multi-agent deep reinforcement learning based unmanned aerial vehicle navigation and obstacle avoidance method of the present application.

[0070] Fig. 2 It is a multi-unmanned aerial vehicle experience replay mechanism and reward function schematic diagram.

[0071] Fig. 3 It is a centralized training and distributed execution framework schematic diagram.

[0072] Fig. 4 It is a system flowchart of the present application. DETAILED DESCRIPTION

[0073] In order to have a clearer understanding of the technical features, objects and effects of the present application, the specific embodiments of the present application will be described with reference to the drawings.

[0074] A preferred embodiment of the multi-unmanned aerial vehicle cooperative navigation method based on deep reinforcement learning provided by the present application is shown in the figure. Figs. 1 to 4 As shown in the figure: the environment perception module, in the embodiment of the present application, each unmanned aerial vehicle is integrated with multi-modal sensor device, including laser radar, binocular camera, IMU, ultrasonic module, etc., for real-time perception of the position, speed, size of obstacles in the environment, and the relative position of adjacent unmanned aerial vehicles and other spatial state information. After the above perception information is fused and preprocessed, it is encoded into a multi-dimensional state vector, which is input to the deep reinforcement learning model, so as to realize accurate situation awareness and modeling in complex dynamic environment. This module ensures that the unmanned aerial vehicle can continuously obtain high-reliability local and relative information throughout the flight, providing support for downstream intelligent decision-making.

[0075] An intelligent decision control module is adopted, which is a reinforcement learning model based on MA-DDQN. The model structure includes a main network and a target network, and the policy stability is improved through a delayed update mechanism. After the input state vector is extracted through a convolution layer and a fully connected layer, the Q values of each action in the action space are output to support decision-making behaviors such as obstacle avoidance, path adjustment, and formation control.

[0076] In the centralized training phase, the global state data of all unmanned aerial vehicles are uploaded to a central server, and the server updates the policy parameters in a joint optimization manner. The experience data are stored in a global experience replay buffer, and a priority experience sampling mechanism is adopted to weight the training samples, so as to improve the utilization rate of key experience and the training efficiency.

[0077] In the distributed execution phase, each unmanned aerial vehicle independently runs a local reinforcement learning model and autonomously selects the optimal action based on the observed local state information. To ensure the coordination of cooperative behaviors, a bandwidth-aware mechanism is used to control the state synchronization frequency, so that efficient information sharing is realized within the allowable range of communication conditions.

[0078] A multi-objective weighted reward function is designed, which comprehensively considers the following indicators: flight task completion degree; minimum safe distance from obstacles; energy consumption level; and formation keeping deviation degree.

[0079] After the above indicators are normalized and weighted according to the preset weights, they are combined as the instant feedback signal in the deep reinforcement learning training. During the training process, all interaction data are continuously stored in the experience replay buffer, and the system can update the model based on the newly collected data, so as to realize adaptive learning and dynamic optimization of environmental changes.

[0080] The above description is only a specific embodiment of the present application, and is not intended to limit the scope of the present application. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present application shall fall within the scope of the present application. It should be noted that the components of the present application are not limited to the above overall application, and the technical features described in the specification of the present application can be selected individually or combined for use, therefore, the present application naturally covers other combinations and specific applications related to the present application.

Claims

1. A multi-UAV collaborative navigation method based on deep reinforcement learning, characterized in that: The following steps are involved: S1. Deep Reinforcement Learning Model Construction: Build a deep reinforcement learning model based on a multi-agent dual deep Q-network and deploy it on each drone to learn its interaction with the environment and other drones, achieving autonomous navigation and obstacle avoidance strategy optimization. S2. Dynamic Obstacle Avoidance and Formation Maintenance: During the execution phase, each UAV uses lidar, binocular cameras, and an inertial measurement unit to obtain real-time information about the position, speed, and size of obstacles. This information is used as state input and the current action is calculated based on a Q-value function, including speed, heading, and position adjustments to avoid obstacles and collisions. Based on mission objectives and environmental changes, each UAV dynamically adjusts its flight path and relative position to maintain a stable formation. S3, Centralized Training and Distributed Execution: During the centralized training phase, the global state information of all UAVs is aggregated, and a joint optimization strategy is used to improve collaborative performance. During the distributed execution phase, each UAV makes autonomous decisions based on local observation information and selectively synchronizes core state information based on a communication bandwidth awareness mechanism to reduce network load. S4. Experience replay mechanism: Apply the experience replay mechanism to recycle historical interaction data during execution to optimize flight strategies and achieve a balance between individual autonomy and system collaboration.

2. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1 is characterized in that: The MA-DDQN model includes a main network and a target network, and adopts a Double Q-Learning mechanism to alleviate Q-value overestimation through delayed updates, thereby improving model convergence and flight decision accuracy.

3. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1 is characterized in that: An independent MA-DDQN model was constructed for each drone. The model includes: an input layer that receives environmental information such as position coordinates, speed, heading angle, remaining energy, relative position and size of obstacles, and the status of neighboring drones; a hidden layer that includes three convolutional layers and two fully connected layers, with 32, 64, and 128 convolution kernels and 256 and 128 neurons respectively, equipped with batch normalization and ReLU activation functions; and an output layer that outputs the Q value corresponding to the action to guide the selection of navigation behavior.

4. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1 is characterized in that: The dynamic obstacle avoidance and formation maintenance include: sensing environmental obstacles and aircraft status based on sensors; predicting the optimal obstacle avoidance path using a deep reinforcement learning model; adjusting its own flight trajectory according to the status of other drones; continuously and dynamically adjusting position, speed and heading to maintain formation; automatically reconstructing the formation according to mission requirements when the environment changes, and synchronizing the status through wireless communication.

5. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1 is characterized in that: The centralized training achieves collaborative strategy learning of multiple UAVs through multi-agent joint strategy optimization and global state aggregation; the distributed execution achieves autonomous decision-making based on local perception, and dynamically adjusts the frequency of information interaction based on the communication bandwidth threshold. Under low bandwidth conditions, only core state variables such as position and heading angle are synchronized.

6. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1 is characterized in that: The multiple drones share flight status information, including position, speed, heading angle and energy consumption parameters, through a wireless communication network to achieve linkage optimization of path planning, obstacle avoidance strategy and task collaboration.

7. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1 is characterized in that: The experience replay mechanism continuously updates the flight strategy network by recycling historical interaction data, improving the model's generalization ability, obstacle avoidance effect, and flight efficiency, and supports a priority experience sampling strategy based on TD error.

8. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1 is characterized in that: This method is applicable to heterogeneous UAV systems, where different types of UAVs have different speeds, energy consumption and perception capabilities. By introducing category labels and personalized training models, differentiated strategy optimization is achieved.

9. The multi-UAV collaborative navigation method based on deep reinforcement learning according to claim 1, characterized in that: Each drone includes: an environmental perception module: used to collect and pre-process information such as obstacle position, speed, direction, etc. as model input; a decision-making control module: used to control flight attitude, speed and path in real time according to model output, to achieve dynamic obstacle avoidance and path adjustment; a wireless communication module: to achieve real-time sharing of flight status data, support formation coordination, task division and dynamic path adjustment.

Images