Multi-unmanned aerial vehicle cooperation system based on federal learning

Abstract

The invention discloses a multi-unmanned aerial vehicle cooperation system based on federated learning, and relates to the technical field of unmanned aerial vehicle cooperation and target tracking, and the system comprises a plurality of unmanned aerial vehicles, and carries out the local processing of computing resources through a federated learning mechanism to form a local model. Each unmanned aerial vehicle can independently detect and track a target in the visual field of the unmanned aerial vehicle through the convolutional neural network; the central server collects model parameters aggregated by the plurality of unmanned aerial vehicles, updates a global model, re-distributes the enhanced global model to each unmanned aerial vehicle, and dynamically adjusts task allocation and tracking strategies of the plurality of unmanned aerial vehicles by using a dynamic cooperative tracking mechanism; the distributed communication network includes a peer-to-peer communication network and a centralized communication link. According to the multi-unmanned aerial vehicle cooperation system based on federal learning, the data transmission safety is improved, the data transmission amount is reduced, low-delay and efficient model improvement is ensured through a dynamic cooperation tracking mechanism, and the system has robustness and expandability in a dynamic environment.

Description

Technical Field

[0001] The present invention relates to the technical field of UAV collaboration and target tracking, and particularly to a multi-UAV collaboration system based on federated learning. Background Art

[0002] UAVs are widely used for monitoring and tracking illegal border crossing activities. By detecting and tracking targets in real time, UAVs significantly improve the efficiency and security of monitoring.

[0003] However, when a single UAV performs tasks, its coverage and processing capabilities are limited, making it difficult to cope with complex and dynamic environments. To solve this problem, multi-UAV collaboration systems have emerged. Through the coordinated work of multiple UAVs, the execution efficiency of tasks can be significantly improved, the monitoring range can be expanded, and complex tasks can be completed through distributed sensing and decision-making capabilities. In addition, UAV swarms can dynamically allocate tasks and optimize flight paths to adapt to changing environments, thereby improving the overall system's robustness and flexibility.

[0004] Although multi-UAV collaboration systems have many advantages, they still face many challenges. How to effectively integrate multi-view data from multiple UAVs to ensure high-precision target detection and tracking remains an urgent problem to be solved. Although algorithms such as the KLT tracker, CNN-based detectors, and distributed Kalman filtering are effective in some cases, they are hindered by heavy communication requirements and poor adaptability in non-linear dynamic environments. These limitations highlight the need for improvement in multi-UAV collaboration methods, especially in terms of privacy, communication load, and adaptability. In addition, in multi-UAV collaboration tasks, each UAV may collect data containing sensitive information. Traditional centralized data processing methods require uploading all data to a central server, increasing the risk of data privacy leakage, and the high communication cost also limits the practical application of the system. Summary of the Invention

[0005] The main purpose of the present invention is to provide a multi-UAV collaboration system based on federated learning to solve the problems of: the centralized data processing method requires uploading all data to a central server, which may lead to the risk of data privacy leakage; high communication cost: the transmission of a large amount of raw data occupies bandwidth and increases the communication burden; low adaptability and efficiency: in complex and dynamic environments, the task allocation and collaboration efficiency between UAVs are low.

[0006] To achieve the above purpose, the present invention provides the following technical solutions: A multi-UAV collaboration system based on federated learning, including multiple UAVs, a central server, and a distributed communication network; Each drone is equipped with a camera for capturing real-time visual data and a processing unit. The processing unit locally processes the computing resources through a federated learning mechanism to form a local model. Each drone can independently detect and track targets within its field of view through a convolutional neural network; The central server collects the model parameters aggregated by multiple drones, updates the global model, and redistributes the enhanced global model to each drone, using a dynamic cooperative tracking mechanism to dynamically adjust the task allocation and tracking strategies of multiple drones; The distributed communication network includes a peer-to-peer communication network and a centralized communication link. Among them, multiple drones use the peer-to-peer communication network for direct data sharing, and multiple drones send model updates to the central server through the centralized communication link. The peer-to-peer network allows low-latency tracking data exchange between drones, while the central server also uses the centralized communication link to send the updated model to multiple drones, ensuring secure and efficient model aggregation and redistribution.

[0007] Preferably, the federated learning mechanism enables drones to locally train local models without sharing raw data. At the same time, the federated learning integration mechanism adopts key technologies to optimize the transmission and distribution of local models within the drone network. Model quantization reduces the size of the local model by converting 32-bit floating-point parameters into 8-bit or 16-bit fixed-point formats. The use of the Paxos protocol ensures consistency during the distribution of local models. After aggregating the local model parameters, the central server acts as the "proposer" using the Paxos protocol to ensure that all drones receive the same version of the global model.

[0008] Preferably, the update of the global model adopts an incremental update strategy, only transmitting the part of the model parameter change relative to the previous version instead of the entire model. This method significantly reduces the amount of data transmission, speeds up the update speed, improves the response ability of the system, and prevents inconsistent model performance among drones in the network.

[0009] Preferably, the specific description of the federated learning mechanism is as follows. In each round of training, assume that the local dataset of the -th drone is , and its loss function is , where are the model parameters. Each drone updates its local model parameters by minimizing the local loss function: , where is the learning rate, represents the training round. Each drone uploads the updated local model parameters to the central server.

[0010] Preferably, the central server of the federated learning mechanism receives the local model parameters uploaded by all drones and aggregates these parameters using the weighted average method to obtain new global model parameters: , where is the number of drones participating in the training, represents the -th drone's dataset size. The central server distributes the aggregated global model parameters to each drone for the next round of local training.

[0011] Preferably, the specific description of the dynamic collaborative tracking mechanism is as follows: The dynamic collaborative tracking mechanism dynamically adjusts the task assignment and tracking strategy based on the real-time evaluated task status. Each drone regularly evaluates its own status and the target status and calculates the task priority. The task priority calculation formula is: , where is the task priority of the -th drone, is the remaining battery power, is the signal strength, is the task urgency, is the weight coefficient.

[0012] Preferably, the specific steps of the dynamic collaborative tracking mechanism are as follows: Step 1: Each drone dynamically adjusts the tracking strategy according to the task priority; Step 2: When the target moves out of the field of view of one drone, the central server automatically hands over the tracking task to another drone that can cover the target through the shared global model and task status information to ensure the continuity of target tracking; Step 3: Each drone dynamically adjusts the local model parameters according to the environmental changes and tracking effects, and feeds back the adjusted model parameters to the central server after the task is completed for the global model update of the next round of federated learning.

[0013] Preferably, each drone can independently detect and track the targets within its field of view through a convolutional neural network. The convolutional neural network serves as the target detection model, and a lightweight version is deployed on each drone to adapt to the limitations of edge computing resources. The loss function of the target detection model is the cross-entropy loss function, which is defined as follows: , where is the number of samples, is the input image, is the target label, is the model parameter, is the model output.

[0014] Preferably, when the drone tracks the target part within its field of view, the Kalman filtering algorithm is used to predict the target position, and the detection result output by the convolutional neural network is combined to update the target state. The state update equation of the Kalman filter is: , where is the state estimate at time , is the Kalman gain, is the observation value, is the observation matrix.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. In the present invention, through the use of the dynamic cooperative tracking mechanism by the central server, efficient cooperative target tracking among multiple drones is ensured during the coordination process. At the same time, multiple drones continuously improve the model performance locally through the federated learning integration mechanism. Each drone is equipped with a camera and a processing unit, capable of capturing real-time visual data and independently detecting and tracking targets using a convolutional neural network. Different from traditional raw data sharing, the drones train local models on the captured data without sharing the raw data. The dynamic cooperative tracking mechanism optimizes the cooperation among the drones. The drones send updated local model parameters to the central server instead of the raw tracking data. The server aggregates these local models to create a global model. The global model benefits from the diverse data encountered by all drones in different data environments, and then redistributes the global model to each drone to enhance the tracking ability of the drones by integrating the collective knowledge of the network. As the environmental conditions change, the drones continue to adjust their local models and regularly send updates back to the server for aggregation, thus ensuring that the global model always remains effective. The distributed communication network combines the peer-to-peer exchange of local model updates and centralized aggregation, ensuring low latency and efficient model improvement, making the system robust and scalable in a dynamic environment.

[0016] 2. In the present invention, through the setting of the federated learning mechanism, a distributed federated learning architecture is used to complete local model training locally on the drones, sharing only model parameters instead of raw data, without the need to transmit all data to the central server, improving data security. At the same time, the transmission and distribution of local models within the drone network are optimized. Model quantization reduces the size of the local model by converting 32-bit floating-point parameters into 8-bit or 16-bit fixed-point formats. The use of the Paxos protocol ensures consistency during the distribution process of local models. After aggregating the local model parameters, the central server, as the "proposer", uses the Paxos protocol to ensure that all drones receive the same version of the global model, effectively reducing the amount of data transmission.

[0017] III. In the present invention, through the setting of the dynamic collaborative tracking mechanism, on the one hand, the task allocation and tracking strategy can be dynamically adjusted based on real-time information such as the target position, UAV battery power, signal strength, etc., and on the other hand, the task continuation between UAVs can be supported through the global model to ensure the continuity and accuracy of target tracking.

[0018] IV. In the present invention, the UAVs adopt a point-to-point communication network method for low-latency data sharing, and multiple UAVs and the central server transmit data to each other through a centralized communication link to achieve efficient model aggregation and distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic flowchart of the present invention; Figure 2 is a comparison table of independent detection and tracking algorithms on different data sets in the present invention; Figure 3 is a performance evaluation table of the dynamic collaborative tracking mechanism in the present invention; Figure 4 is the test result of this system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] In order to make the technical means, creative features, achieved purposes and functions of the present invention easy to understand, the present invention will be further described below in conjunction with specific embodiments.

[0021] Example 1: As Figures 1 - 3 shown, a multi-UAV collaboration system based on federated learning includes multiple UAVs, a central server, and a distributed communication network; Each UAV is equipped with a camera for capturing real-time visual data and a processing unit. The processing unit locally processes the computing resources through the federated learning mechanism to form a local model. Each UAV can independently detect and track the targets within its field of view through a convolutional neural network; The central server collects the model parameters aggregated by multiple UAVs, updates the global model, and redistributes the enhanced global model to each UAV, and uses the dynamic collaborative tracking mechanism to dynamically adjust the task allocation and tracking strategy of multiple UAVs; The distributed communication network includes a point-to-point communication network and a centralized communication link. Among them, multiple UAVs use the point-to-point communication network for direct data sharing, and multiple UAVs send model updates to the central server through the centralized communication link. The point-to-point network allows low-latency tracking data exchange between UAVs, and the central server also uses the centralized communication link to send the updated model to multiple UAVs, ensuring safe and efficient model aggregation and redistribution.

[0022] The drones adopt a point-to-point communication network for low-latency data sharing. Multiple drones and the central server transmit data to each other through a centralized communication link to achieve efficient model aggregation and distribution. The federated learning mechanism enables drones to perform local model training locally without sharing raw data. At the same time, the federated learning integration mechanism uses key technologies to optimize the transmission and distribution of local models within the drone network. Model quantization reduces the size of the local model by converting 32-bit floating-point parameters into 8-bit or 16-bit fixed-point formats. The use of the Paxos protocol ensures consistency during the local model distribution process. After aggregating the local model parameters, the central server acts as the "proposer" using the Paxos protocol to ensure that all drones receive the same version of the global model.

[0023] Using a distributed federated learning architecture, local model training is completed locally on the drones, sharing only model parameters instead of raw data. There is no need to transmit all data to the central server, improving data security. At the same time, it optimizes the transmission and distribution of local models within the drone network, effectively reducing the amount of data transmission. The update of the global model adopts an incremental update strategy, transmitting only the part of the model parameter changes relative to the previous version instead of the entire model. This method significantly reduces the amount of data transmission, speeds up the update speed, improves the system's response ability, and prevents inconsistent model performance among drones in the network.

[0024] The specific description of the federated learning mechanism is as follows. In each round of training, assume that the local dataset of the -th drone is , and its loss function is , where are the model parameters. Each drone updates its local model parameters by minimizing the local loss function: , where is the learning rate, represents the training round. Each drone uploads the updated local model parameters to the central server.

[0025] The central server of the federated learning mechanism receives the local model parameters uploaded by all drones and aggregates these parameters using the weighted average method to obtain new global model parameters: , where, is the number of drones participating in the training, represents the size of the dataset of the -th drone. The central server distributes the aggregated global model parameters to each drone for the next round of local training.

[0026] The specific description of the dynamic collaborative tracking mechanism is as follows: The dynamic collaborative tracking mechanism dynamically adjusts the task allocation and tracking strategy based on the task status evaluated in real time. Each drone regularly evaluates its own status and the target status, and calculates the task priority. The task priority calculation formula is: , where is the task priority of the th drone, is the remaining battery power, is the signal strength, is the task urgency, is the weight coefficient.

[0027] The specific steps of the dynamic collaborative tracking mechanism are as follows: Step 1: According to the task priority, each drone dynamically adjusts the tracking strategy; Step 2: When the target moves out of the field of view of one drone, the central server automatically hands over the tracking task to another drone that can cover the target through the shared global model and task status information, ensuring the continuity of target tracking; Step 3: Each drone dynamically adjusts the local model parameters according to the environmental changes and tracking effects, and feeds back the adjusted model parameters to the central server after the task is completed for the global model update of the next round of federated learning.

[0028] In this way, on the one hand, the task allocation and tracking strategy can be dynamically adjusted based on real-time information such as the target location, drone battery power, and signal strength. On the other hand, the task continuation between drones can be supported through the global model, ensuring the continuity and accuracy of target tracking; Each drone can independently detect and track the targets within its field of view through a convolutional neural network. The convolutional neural network serves as the target detection model, and a lightweight version is deployed on each drone to adapt to the limitations of edge computing resources. The loss function of the target detection model is the cross-entropy loss function, which is defined as follows: , where is the number of samples, is the input image, is the target label, is the model parameter, is the model output.

[0029] When the drone is tracking the target part within its field of view, the Kalman filter algorithm is used to predict the target position, and the detection result output by the convolutional neural network is combined to update the target status. The state update equation of the Kalman filter is: , where is the state estimate at time , is the Kalman gain, is the observation value, is the observation matrix.

[0030] The use of a dynamic collaborative tracking mechanism through a central server ensures efficient collaborative target tracking among multiple drones during the coordination process. At the same time, multiple drones continuously improve the model performance locally through a federated learning integration mechanism. Each drone is equipped with a camera and a processing unit, capable of capturing real-time visual data and independently detecting and tracking targets using a convolutional neural network. Different from traditional raw data sharing, drones train local models on the captured data without sharing the raw data. The dynamic collaborative tracking mechanism optimizes the collaboration among drones. Drones send updated local model parameters to the central server instead of the raw tracking data. The server aggregates these local models to create a global model. The global model benefits from the diverse data encountered by all drones in different data environments and then redistributes the global model to each drone, enhancing the tracking ability of drones by integrating the collective knowledge of the network. As the environmental conditions change, drones continue to adjust their local models and regularly send updates back to the server for aggregation, ensuring that the global model always remains effective. The distributed communication network combines the peer-to-peer exchange of local model updates and centralized aggregation, ensuring low latency and efficient model improvement, making the system robust and scalable in a dynamic environment.

[0031] Embodiment 2: As Figures 1 - 3 shown, a multi-drone collaboration system based on federated learning includes multiple drones, a central server, and a distributed communication network; Each drone is equipped with a camera for capturing real-time visual data and a processing unit. The processing unit locally processes the computing resources through a federated learning mechanism to form a local model. Each drone can independently detect and track targets within its field of view through a convolutional neural network; The central server collects the aggregated model parameters of multiple drones, updates the global model, and redistributes the enhanced global model to each drone, and uses a dynamic collaborative tracking mechanism to dynamically adjust the task assignment and tracking strategy of multiple drones; The distributed communication network includes a peer-to-peer communication network and a centralized communication link. Among them, multiple drones use the peer-to-peer communication network for direct data sharing, and multiple drones send model updates to the central server through the centralized communication link. The peer-to-peer network allows for low-latency tracking data exchange between drones, and the central server also uses the centralized communication link to send the updated model to multiple drones, ensuring secure and efficient model aggregation and redistribution.

[0032] Simulation experiments were conducted on a desktop computer with an NVIDIA GeForce RTX 3060 GPU, an Intel(R) Core(TM) i5-10400F CPU @ 2.90 GHz, and 12 GB of VRAM. The simulation was carried out on the Gazebo platform, and the Robot Operating System (ROS) was used for UAV flight control and mission simulation. PX4 SITL (Software-in-the-Loop) was used for flight control simulation, while QGroundControl was used for planning and executing virtual flight missions. Three datasets were used in the experiments: UAVDT, DOTA, and VisDrone.

[0033] UAVDT: This dataset is mainly used to test the ability of algorithms to detect and track vehicles and pedestrians in complex traffic scenarios.

[0034] VisDrone: This dataset is used to evaluate the performance of algorithms in complex urban environments. It includes a wide range of target types and various flight conditions, providing in-depth insights into the adaptability and accuracy of algorithms in real-world scenarios.

[0035] DOTA: This dataset is designed specifically for object detection in aerial images, including various complex scenarios with objects such as buildings, ships, vehicles, and airplanes. The high-resolution images with multiple object classes make it suitable for evaluating the detection ability and accuracy of algorithms in high-resolution settings.

[0036] Training and testing were performed using TensorFlow and PyTorch. The experiments were conducted in a virtual UAV flight environment created using the Gazebo simulation platform, where ROS was responsible for handling flight control and mission simulation, and PX4 SITL was responsible for managing flight control simulation. QGroundControl was used for planning and executing virtual flight missions.

[0037] The simulated UAV performed object detection and tracking tasks at different flight altitudes and a speed of 3 m / s, focusing on evaluating the performance of the algorithms in scenarios with target motion and complex situations. To comprehensively evaluate the adaptability of the algorithms, different weather conditions and flight altitudes were introduced for the UAV in the object detection and tracking tasks. In each experiment, we recorded metrics such as detection accuracy, tracking accuracy, processing time per frame, frames per second, and communication overhead to comprehensively evaluate the performance of the algorithms.

[0038] Figure 2 and 3 showed that our proposed algorithm for independent detection and tracking of targets based on the federated learning mechanism and convolutional neural network performed excellently in terms of detection accuracy, tracking accuracy, processing time, frames per second, and communication overhead. Specifically, our algorithm outperformed YOLOv3 and SSD, especially in scenarios involving complex backgrounds and frequent target motion.

[0039] In the UAVDT dataset, our algorithm achieved a detection accuracy of 92.5% and a tracking accuracy of 88.7% at a height of 10 meters on sunny days, exceeding YOLOv3 by about 2 percentage points while maintaining a low communication overhead. In the VisDrone dataset, even under challenging night conditions in complex urban environments, our algorithm could maintain a high detection accuracy of 88.2%. In the DOTA dataset, our algorithm performed excellently in high-resolution aerial images, achieving a detection accuracy of 94.8% while maintaining a stable frame rate during the processing of high-resolution images. In terms of processing time, our algorithm had an average processing time of 25 to 30 milliseconds per frame, which was better than SSD's 30 to 34 milliseconds, especially in high-load scenarios such as congested traffic environments. In terms of FPS, our algorithm always ran at 40 to 45 FPS under various experimental conditions, ensuring real-time processing capabilities.

[0040] In addition, our algorithm significantly reduced the communication overhead while maintaining high accuracy, reducing it by about 20% and 25% on the UAVDT and VisDrone datasets respectively, which is beneficial for large-scale drone system collaboration. Overall, our algorithm demonstrated excellent performance and wide adaptability in different test environments, outperforming mainstream algorithms in terms of accuracy, efficiency, and communication overhead. It was particularly outstanding in complex and dynamic scenarios, showing excellent robustness and practical application potential.

[0041] Embodiment 3: As Figures 1 - 4 shown, a multi-drone collaboration system based on federated learning includes multiple drones, a central server, and a distributed communication network: Each drone is equipped with a camera for capturing real-time visual data and a processing unit. The processing unit locally processes the computing resources through a federated learning mechanism to form a local model. Each drone can independently detect and track targets within its field of view through a convolutional neural network; The central server collects the aggregated model parameters of multiple drones, updates the global model, and redistributes the enhanced global model to each drone, using a dynamic collaborative tracking mechanism to dynamically adjust the task allocation and tracking strategies of multiple drones; The distributed communication network includes a peer-to-peer communication network and a centralized communication link. Among them, multiple drones use the peer-to-peer communication network for direct data sharing, and multiple drones send model updates to the central server through the centralized communication link. The peer-to-peer network allows low-latency tracking data exchange between drones, while the central server also uses the centralized communication link to send the updated model to multiple drones, ensuring secure and efficient model aggregation and redistribution.

[0042] A fleet of eight custom drones was specifically deployed, and each drone was equipped with NVIDIA Jetson Nano as an edge computing platform. Jetson Nano has a 128-core Maxwell GPU and 4GB of RAM, runs JetPack SDK 4.4, and supports the deployment and inference of deep learning models. The drones use the PX4 open-source flight control system and are equipped with a Raspberry Pi Camera V2, capable of capturing 1080p video at 30 frames per second. The drones are powered by 4S LiPo batteries, providing a flight time of 20 - 25 minutes, enabling efficient object detection and tracking in highly dynamic environments. Collaboration among multiple drones promotes distributed computing and communication, enhancing the system's coverage and the robustness of task execution. Each Jetson Nano runs on Ubuntu 18.04 and uses TensorFlow 2.3.0 and PyTorch 1.6.0 for deep learning task inference. The drones are controlled and task-planned through QGroundControl, ensuring they follow predefined paths and speeds to perform object detection and tracking tasks. The drones communicate via a low-latency network to coordinate tasks and share data, maintaining the continuity and accuracy of target tracking.

[0043] The model was further fine-tuned using the image data collected in real-time by the drones to adapt to the specific requirements of object detection and tracking in various environments. The research was conducted in multiple real-world scenarios, including open fields, urban areas, and suburban environments. Each environment was tested under different lighting and weather conditions, such as sunny, cloudy, night, rainy, and foggy weather. The drones flew at an altitude of 10 to 20 meters and a speed of 3 meters per second. Collaboration among multiple drones simulated complex large-scale scenarios involving multiple targets, further validating the performance of the algorithm. Throughout the experiment, the eight drones collaborated to collect a large amount of real-time image and video data, covering object detection and tracking tasks in various environments and conditions. The collected data was manually annotated to generate a "ground truth" dataset, including bounding boxes and class information of the target objects. Then, these ground truth datasets were used to evaluate the detection accuracy, tracking accuracy, and recall rate of the algorithm, further validating its performance in real-world environments.

[0044] Figure 4The experimental data shows that the algorithm can operate effectively in different scenarios. In an open area with sufficient sunlight, the algorithm achieved a high detection accuracy of 93.4%, a tracking accuracy of 89.6%, and a recall rate of 91.2%. The processing time was 28 milliseconds per frame, and the frame rate was 42 FPS. The communication overhead was minimized at 12 MB. However, in urban areas at night, the algorithm encountered more challenges, with the detection accuracy dropping to 89.4%, the tracking accuracy dropping to 85.0%, and the recall rate dropping to 87.6%. The processing time increased to 34 milliseconds per frame, and the communication overhead rose to 18 MB. These results demonstrate the adaptability of the algorithm, which can maintain reasonable performance even in more challenging environments. The following metrics were used, and the results are shown in Figure 3 as follows: Redundancy rate: This metric indicates the frequency at which multiple drones detect the same target. In an empty field, the redundancy rate was low at 5.8%, indicating effective coordination and minimal overlap. However, in more complex environments, such as urban areas during rain or at night, the redundancy rate increased to 15.8%, indicating potential inefficiency due to overlapping detections.

[0045] Task assignment balance: This measures the uniformity of task distribution among drones. In an open area, the balance was 87.5%, indicating an even distribution of tasks and enabling efficient operation. In more challenging environments, such as urban areas at night or in the rain, the balance dropped to 77.3%, indicating that some drones were more heavily loaded than others, which could reduce the efficiency of the entire system.

[0046] Response time: This metric measures the average delay in task switching or coordination among drones. The response time in an open area was 150 milliseconds, which was very fast, enabling the system to quickly adapt to environmental changes. However, in the urban night scenario, the response time increased to 230 milliseconds, reflecting a slower coordination speed and potential challenges in maintaining real-time performance.

[0047] Tracking continuity: This measures the seamlessness of target tracking during handover among drones. In an open area, the system achieved a high continuity of 98.7%, indicating smooth handover and stable tracking. In more complex environments, such as urban areas during rain or at night, the continuity dropped to 90.1%, indicating brief interruptions during target switching, but the system generally maintained effective tracking.

[0048] In an open environment, the system achieved high coverage and efficient coordination. Even in complex scenarios such as urban areas at night, the system could maintain stable task assignment and tracking continuity, demonstrating its adaptability and reliability.

[0049] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification is only to illustrate the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. A multi-UAV collaboration system based on federated learning, characterized in that, It includes multiple drones, a central server, and a distributed communication network; Each drone is equipped with a camera for capturing real-time visual data and a processing unit. The processing unit locally processes computing resources through a federated learning mechanism to form a local model. Each drone can independently detect and track targets within its field of view through a convolutional neural network; The central server collects the aggregated model parameters of multiple drones, updates the global model, and redistributes the enhanced global model to each drone, using a dynamic collaborative tracking mechanism to dynamically adjust the task allocation and tracking strategies of multiple drones; The distributed communication network includes a peer-to-peer communication network and a centralized communication link. Among them, multiple drones use the peer-to-peer communication network for direct data sharing, and multiple drones send model updates to the central server through the centralized communication link. The peer-to-peer network allows low-latency tracking data exchange between drones, and the central server also uses the centralized communication link to send the updated model to multiple drones, ensuring secure and efficient model aggregation and redistribution.

2. The multi-UAV cooperation system based on federated learning according to claim 1, wherein: The federated learning mechanism enables drones to locally train local models without sharing raw data. At the same time, the federated learning integration mechanism adopts key technologies to optimize the transmission and distribution of local models within the drone network. Model quantization reduces the size of the local model by converting 32-bit floating-point parameters into 8-bit or 16-bit fixed-point formats. The use of the Paxos protocol ensures consistency during the distribution process of local models. After aggregating local model parameters, the central server acts as a "proposer" using the Paxos protocol to ensure that all drones receive the same version of the global model.

3. A multi-UAV cooperative system based on federated learning according to claim 2, characterized in that: The update of the global model adopts an incremental update strategy, only transmitting the part of the model parameter changes relative to the previous version instead of the entire model. This method significantly reduces the data transmission volume, speeds up the update speed, improves the system's response ability, and prevents inconsistent model performance among drones in the network.

4. The multi-UAV cooperation system based on federated learning according to claim 3, wherein: The specific description of the federated learning mechanism is as follows. In each round of training, assume that the local dataset of the th drone is , and its loss function is , where are the model parameters. Each drone updates its local model parameters by minimizing the local loss function: , where is the learning rate, represents the training round. Each drone uploads the updated local model parameters to the central server.

5. The multi-UAV cooperation system based on federated learning according to claim 4, wherein: The central server of the federated learning mechanism receives the local model parameters uploaded by all drones and aggregates these parameters using the weighted average method to obtain new global model parameters: , where is the number of drones participating in the training, represents the -th drone's dataset size. The central server distributes the aggregated global model parameters to each drone for the next round of local training.

6. The multi-UAV collaborative system based on federated learning according to claim 5, characterized in that: The specific description of the dynamic collaborative tracking mechanism is as follows: The dynamic collaborative tracking mechanism dynamically adjusts the task allocation and tracking strategy based on the task status evaluated in real time. Each UAV regularly evaluates its own status and the target status, and calculates the task priority. The formula for calculating the task priority is: , where is the task priority of the th UAV, is the remaining battery power, is the signal strength, is the task urgency, is the weight coefficient.

7. A multi-UAV cooperation system based on federated learning according to claim 6, characterized in that: The specific steps of the dynamic collaborative tracking mechanism are as follows: Step 1: Each drone dynamically adjusts the tracking strategy according to the task priority; Step 2: When the target moves out of the field of view of one drone, the central server automatically hands over the tracking task to another drone that can cover the target through the shared global model and task status information to ensure the continuity of target tracking; Step 3: Each drone dynamically adjusts the local model parameters according to environmental changes and tracking effects, and feeds back the adjusted model parameters to the central server after the task is completed for the global model update in the next round of federated learning.

8. The multi-UAV cooperation system based on federated learning according to claim 7, wherein: Each of the drones is capable of independently detecting and tracking targets within its field of view through a convolutional neural network, which serves as a target detection model. A lightweight version is deployed on each drone to adapt to the limitations of edge computing resources. The loss function of the target detection model is the cross-entropy loss function, defined as follows: , where is the number of samples, is the input image, is the target label, are the model parameters, is the model output.

9. A multi-UAV collaborative system based on federated learning according to claim 8, characterized in that: When the drone tracks the target part within its field of view, the Kalman filter algorithm is used to predict the target position, and the detection result output by the convolutional neural network is combined to update the target state. The state update equation of the Kalman filter is: , where is the state estimate at time , is the Kalman gain, is the observation value, is the observation matrix.

Images