Unmanned aerial vehicle vision target detection online federated learning method and system based on participant and sample screening
Patent Information
- Application Number
- CN202410003007.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-02
- Publication Date
- 2026-09-11
- Estimated Expiration
- 2044-01-02
AI Technical Summary
一旦无人机距离太远或通信质量较差，数据传输的时间将大幅度延长，速度慢的无人机会减慢联邦学习的训练效率；
[0067] (1) Data sharing among UAVs was achieved through federated learning, which improved the convergence speed of the target detection model;
Smart Images
Figure CN117809156B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent ad hoc network technology for unmanned aerial vehicles (UAVs), and in particular to an online federated learning method and system for visual target detection in UAVs based on participant and sample selection. Background Technology
[0002] Federated Learning (FL) is a novel form of distributed machine learning, first proposed by Google in 2016. FL treats users as clients training models locally, while servers are provided by high-performance computing resources such as cloud servers. Clients are only allowed to transfer their local models to the server, and the server aggregates multiple local models into a global model, thus fusing the training results. Compared to traditional centralized machine learning, Federated Learning employs a multi-node collaborative approach, distributing the high demands of the training dataset and computational power from the server to each client. Federated Learning can achieve data sharing among multiple clients while ensuring individual privacy and security, and can be applied to scenarios with a large number of nodes and dispersed data. Furthermore, Federated Learning allows clients to freely join and leave the system.
[0003] Due to the aforementioned advantages, federated learning is well-suited for unmanned aerial vehicle (UAV) swarms, which feature distributed nodes, heterogeneous local datasets, and limited onboard computing resources. UAVs, characterized by high maneuverability and light weight, are widely used in various fields such as material distribution, target recognition, and fire rescue. Utilizing UAVs for artificial intelligence has become a research hotspot. Federated learning enables UAVs to act as both servers and clients, jointly training a global model with limited computing resources. However, in existing application scenarios, UAV target detection is limited by insufficient model accuracy. When UAV swarms perform target detection in unfamiliar environments, the accuracy is often poor. The main reasons are: a lack of training samples from unfamiliar environments, forcing the model to detect only pre-trained objects; and external interference causing blurred images, such as camera shake or fog. Therefore, online federated learning, which combines online learning with federated learning, has become an effective means to improve model accuracy. Each client UAV can train its local model with continuously supplemented online samples, and then quickly share the knowledge learned by the model with other participants through federated learning.
[0004] Extensive research and experiments have been conducted on the algorithm optimization and object detection applications of federated learning. However, research on online federated learning systems based on UAV object detection is relatively limited, and online federated learning systems still face the following challenges:
[0005] 1. Drone Communication: The transmission channel quality of drones is unstable, and their communication capabilities are limited. If the drone is too far away or the communication quality is poor, the data transmission time will be significantly extended, and slow drones will slow down the training efficiency of federated learning.
[0006] 2. Online Samples: Newly added samples are crucial for improving target detection performance in unfamiliar environments. However, the ever-increasing number of online samples leads to an excessively large local training set on the drone. The excessively long local training time reduces the training efficiency of federated learning. Furthermore, the drone's computing resources are limited, and the number of samples that can be accommodated in a single training session is also limited. Summary of the Invention
[0007] Objective: To address Challenge 1 mentioned in the background, this invention selects a limited number of highly efficient client drones for federated learning. To address Challenge 2, it selects a limited number of important samples from the local training set for local training. For drone target detection, online federated learning based on participant and sample selection is needed to improve model training speed. Therefore, this invention proposes an online federated learning method for drone visual target detection based on participant and sample selection, and further proposes a system for implementing the above method to solve the aforementioned problems in the existing technology.
[0008] Firstly, an online federated learning method for UAV visual target detection based on participant and sample selection is proposed, with the following steps:
[0009] S1. Use a drone pod to collect real-time images and use the collected real-time images as new samples;
[0010] S2. Manually label the new samples to create online samples, and add the online samples to the local dataset of the client drone;
[0011] S3. Send the pre-set initial training model to all client drones;
[0012] S4. The client drone will use the received initial training model as a local model and adjust the number of samples for local training according to the computing performance of its own onboard equipment and the specified local training time.
[0013] S5. Based on the number of samples trained locally, select samples with importance higher than a predetermined value from the local dataset to obtain the training dataset;
[0014] S6. Use the training dataset and the local model to perform local training to obtain real-time training data;
[0015] S7. Combine the local models uploaded by multiple client drones with the real-time training data to form a global model required for the next round of training;
[0016] S8. Calculate the total efficiency of each participant based on the real-time training data, and select the client drones with a total efficiency higher than the predetermined value as participants in the next round of training.
[0017] S9. Send the global model to the client drone selected in step S8;
[0018] S10. Repeat steps S4 to S9 until the global model converges, then end the training.
[0019] In a further embodiment of the first aspect, step S4 adjusts the number of samples for local training based on the computing performance of the onboard equipment and the specified time for local training, including:
[0020] Constructing the Shannon channel to calculate the communication time τ c :
[0021]
[0022]
[0023] In the formula, Indicates the transmission rate, |w i | Indicates the size of the transmitted data; B k and P k The communication bandwidth and transmission power of the client drone k are represented; N0 is the power spectral density of Gaussian noise; h i,k The channel gain between the client drone k and the server drone is represented by the attenuation coefficient o. k and relative distance d i,k This allows for the calculation of the training time that can be allocated to the client drone.
[0024] Therefore, each client drone estimates the number of samples D for local training based on its computing performance. i,k :
[0025]
[0026] Among them, I k C represents the number of local iterations of the client drone k in a round of global training. k f represents the computation cycles required for the GPU of client drone k to compute a single sample. k This represents the computing power of the client drone k, specifically the number of computation cycles it can execute per unit of time.
[0027] In a further embodiment of the first aspect, step S5 involves filtering samples with importance higher than a predetermined value from the local dataset to obtain a training dataset, including:
[0028] Client-side drones continuously replenish online samples Set the current online samples as the highest priority. Local training dataset The combination is represented as:
[0029]
[0030] In the formula, a represents the possible combinations of the selected training samples. i,k Indicates the number of possible combinations; D i,k Indicates the number of samples used for local training; This represents a combination of locally trained datasets;
[0031] For the training samples in the i-th global training iteration, the optimal combination Represented as:
[0032]
[0033] In the formula, loss(*) represents the loss function of machine learning; (x,y) represent the original sample data and the model inference result, respectively.
[0034] In a further embodiment of the first aspect, step S6 further includes:
[0035] For the training dataset and the local model w i,k Perform local training, where i represents the global training round and k represents the drone number;
[0036] It contains K client drones, k = {1, 2, ..., K}; each drone carries a heterogeneous local training dataset. Where (x,y) represents the original image and the target location in the image. This indicates the size of the training dataset.
[0037] In a further embodiment of the first aspect, the training dataset of the client drone is used only locally and does not interact with other client drones;
[0038] During online federated learning, each client drone k is based on its local dataset. Train the model; after multiple local iterations, wait for feedback on a new global model.
[0039] In a further embodiment of the first aspect, step S7 involves uploading local models w from multiple client drones.i,k The real-time training data is combined to form the global model W required for the next round of training. i+1 :
[0040]
[0041] In the formula, This represents the local dataset; K represents the total number of client drones.
[0042] In a further embodiment of the first aspect, the overall efficiency in step S8 includes statistical efficiency U. stat With system efficiency U sys ;
[0043] The statistical efficiency U for client drone k during the i-th global training iteration is... stat The definition is as follows:
[0044]
[0045] In the formula, loss(*) represents the loss function for machine learning; (x d ,y d () represent the original sample data and the model inference results, respectively; Indicates the number of local training datasets;
[0046] The statistical efficiency U of the k client drones in the i-th global training is defined as follows:
[0047]
[0048] In the formula, T represents the specified time for local training; T i,k This represents the actual training time for client k's drones; 1(T <T i,k β is an indicator function, once T i,k Beyond T, the coefficient changes from 1 to β. Significant decrease; T i,k From local training time τ l and communication time τ c Composition; α and β are used as attenuation weights;
[0049] The server-side drones perform the following efficiency evaluation for each customer's drone:
[0050]
[0051]
[0052] Where m i,k This represents the selected client drone, where m i,k=1 indicates that client drone k is selected for the i-th round of training, m i,k =0 indicates that no client drone was selected;
[0053] Before each round of global training begins, the server-side drone selects the M most efficient client-side drones to participate in the training.
[0054] In a second aspect, the present invention proposes an online federated learning system for visual target detection in unmanned aerial vehicles (UAVs). The online federated learning system includes a global model aggregation module, a participant screening module, an online sample collection module, a sample quantity adjustment module, a training sample screening module, a local model training module, and a ground station.
[0055] The online sample acquisition module uses a drone pod to collect real-time images and transmits the collected images back to the ground station as new samples.
[0056] The ground station manually annotates the received real-time images, creates online samples, and returns them to the online sample acquisition module, which then adds them to the local dataset of the client drone.
[0057] The participant screening module sends the initial training model to all client drones, which is received by the sample quantity adjustment module.
[0058] The sample quantity adjustment module of the client drone takes the received initial training model as the local model, adjusts the number of local training samples according to the computing performance of its own airborne equipment and the specified local training time, and passes the relevant parameters to the training sample selection module.
[0059] The training sample selection module selects highly important samples from the local dataset based on a limited number of local training samples to obtain the training dataset, and then passes the training dataset to the local model training module.
[0060] The local model training module uses the selected dataset and local model to perform local training, obtains real-time training data, and sends the trained local model, as well as real-time training data such as training time, communication time, number of local samples, and model accuracy to the global model aggregation module.
[0061] The global model aggregation module receives local models and real-time training data, and aggregates the local models uploaded by multiple client drones into a global model required for the next round of training.
[0062] The participant selection module calculates the total efficiency of each participant based on real-time training data and selects the client drones with high efficiency as participants in the next round of training.
[0063] The participant screening module sends the global model to the selected client drone, which is then received by the sample quantity adjustment module.
[0064] A third aspect of the invention provides an electronic device comprising: a processor and a memory storing computer program instructions; wherein the processor, when executing the computer program instructions, implements the online federated learning method for UAV visual target detection as disclosed in the first aspect and its further embodiments.
[0065] In a fourth aspect, the present invention provides a computer-readable storage medium storing at least one executable instruction that, when executed on an electronic device, causes the electronic device to perform an online federated learning method for UAV visual target detection as disclosed in the first aspect and its further embodiments.
[0066] The present invention has the following beneficial effects:
[0067] (1) Data sharing among UAVs was achieved through federated learning, which improved the convergence speed of the target detection model;
[0068] (2) Online learning enabled the drone to learn new knowledge in an unfamiliar environment, which improved the accuracy of the target detection model;
[0069] (3) Compared with traditional federated learning, online federated learning based on participant and sample screening can supplement new samples in real time during the target detection process, and screen participants and samples based on factors such as communication quality, relative position, computing performance, and sample quality of each node, which further improves the training efficiency and recognition accuracy of UAVs.
[0070] (4) It effectively reduces the time for the UAV target detection model to reach the expected accuracy. Attached Figure Description
[0071] Figure 1 This is a schematic diagram of the structure of online federated learning for UAV visual target detection based on participant and sample screening, as used in an embodiment of the present invention.
[0072] Figure 2(a) is a graph showing the change in accuracy of different functional modules for the target detection model during the model training process of an embodiment of the present invention.
[0073] Figure 2(b) is a graph showing the change in training efficiency of different functional modules for the target detection model during the model training process of an embodiment of the present invention.
[0074] The meanings of the labels in the attached diagram are as follows: 1. Global model aggregation module; 2. Participant screening module; 3. Online sample collection module; 4. Sample quantity adjustment module; 5. Training sample screening module; 6. Local model training module; 7. Ground station. Detailed Implementation
[0075] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.
[0076] Example 1:
[0077] See Figure 1 The diagram shows the structure of an online federated learning system for UAV visual target detection based on participant and sample selection, according to an embodiment of the present invention. The present invention uses a Jetson NX embedded computer mounted on a quadcopter UAV as an edge computing node, and the online federated learning system consists of one server UAV and multiple client UAVs.
[0078] The UAV visual target detection online federated learning system consists of a global model aggregation module 1, a participant screening module 2, an online sample collection module 3, a sample quantity adjustment module 4, a training sample screening module 5, a local model training module 6, and a ground station 7.
[0079] The global model aggregation module 1 and participant selection module 2 are deployed on the server drone. In the federated learning system, they are primarily responsible for receiving local models sent by client drones, aggregating multiple local models into a global model, and returning it to the client drones. They are also responsible for evaluating and selecting client drones.
[0080] The online sample acquisition module 3, sample quantity adjustment module 4, training sample selection module 5, and local model training module 6 are deployed on the client drone. In the federated learning system, they are primarily responsible for receiving the global model sent by the server drone for local training and sending the trained local model back to the server drone. Simultaneously, the online sample acquisition module 3 on the client drone provides new samples for the client drone.
[0081] The global model aggregation module 1, sample quantity adjustment module 4, training sample selection module 5, and local model training module 6 constitute a federated learning system. After the local model training module 6 on the client drone trains the model using the local training set, the local model is sent to the global model aggregation module 1 on the server drone. By weighted averaging the neural network parameters of the local model, a global model is obtained and returned to the sample quantity adjustment module 4 for the next round of global training.
[0082] The online sample acquisition module 3 and the ground station 7 constitute an online sample production system. The online sample acquisition module 3 uses a pod mounted on a UAV to collect real-time environmental images. The images are transmitted from the client UAV to the ground station 7, where they are labeled and made into online learning samples before being returned to the client UAV.
[0083] The sample quantity adjustment module 4 and the training sample selection module 5 constitute the local training set selection system. The sample quantity adjustment module 4 adjusts the sample quantity based on the client drone's computing power and the specified local training time, enabling the client drone to train on as many samples as possible within a limited time. The training sample selection module 5 selects high-quality samples from the local dataset based on the values provided by the sample quantity adjustment module 4.
[0084] The participant selection module 2 deploys a participant selection system. At the start of each round of global training, the server drone can calculate the statistical efficiency of the local training set on different client drones based on the real-time situation of each round of training, calculate the system efficiency of different client drones based on changes in relative position and communication quality, and compare the total efficiency of each client drone to select the client drone with higher efficiency to participate in the next round of global training.
[0085] Example 2:
[0086] Based on Embodiment 1 above, this embodiment discloses an online federated learning method based on participant and sample selection, the steps of which are as follows:
[0087] S01: The online sample acquisition module 3 uses a drone pod to acquire real-time images and transmits the acquired images back to the ground station 7 as new samples;
[0088] S02: Ground station 7 manually annotates the received real-time images, creates online samples, and returns them to the online sample acquisition module 3, which then receives and adds them to the local dataset of the client drone.
[0089] S03: Participant screening module 2 sends the initial training model to all client drones, which is received by sample quantity adjustment module 4;
[0090] S04: The sample quantity adjustment module 4 of the client drone takes the received initial training model as the local model, adjusts the sample quantity of local training according to the computing performance of its own airborne equipment and the specified time of local training, and passes the relevant parameters to the training sample screening module 5.
[0091] S05: The training sample selection module 5 selects highly important samples from the local dataset based on the limited number of local training samples, obtains the training dataset, and passes the training dataset to the local model training module 6.
[0092] S06: Local model training module 6 uses the selected dataset and local model to perform local training, obtains real-time training data, and sends the trained local model, as well as real-time training data such as training time, communication time, number of local samples, and model accuracy to global model aggregation module 1.
[0093] S07: Global model aggregation module 1 receives local models and real-time training data, and aggregates the local models uploaded by multiple client drones into a global model required for the next round of training;
[0094] S08: Participant selection module 2 calculates the total efficiency of each participant based on real-time training data and selects the client drones with high efficiency as participants in the next round of training.
[0095] S09: Participant screening module 2 sends the global model to the selected client drone, which is received by sample quantity adjustment module 4;
[0096] S010: Repeat steps 1-2 and 4-9 respectively to implement online federated learning. Training ends when the global model obtained in step 5 converges.
[0097] This embodiment optimizes the target detection model through online federated learning. A highly accurate target detection model is quickly trained by sharing data between the client drone and the server drone. The main implementation methods are shown in steps 6 and 7.
[0098] In step 6, the client drone's local model training module 6 trains the selected training set and the local model w. i,k Local training was performed, where i represents the global training round and k represents the drone number. There are a total of K client drones, k = {1, 2, ..., K}. Each drone carries a heterogeneous local training dataset. Where (x,y) represents the original image and the target location in the image. This indicates the size of the training dataset. For data security, the training dataset for each client drone is used locally only and does not interact with other client drones. During the online federated learning process, each client drone k is based on its local dataset. Model training. After multiple local iterations, the client drone uploads its trained model to the server drone's global model aggregation module 1, and waits for the server module to return a new global model.
[0099] In step 7, the global model aggregation module 1 of the server drone processes the received local model w. i,k Aggregation yields the global model: After aggregation is complete, the server drone sends the global model to each client drone, and the sample quantity adjustment module 4 performs the next round of global training.
[0100] This embodiment selects high-efficiency participants by calculating the training efficiency (statistical efficiency and system efficiency) of each client drone, thereby accelerating the improvement of model accuracy. The main idea is as shown in step 8.
[0101] In step 8, the participant screening module 2 on the server drone calculates the training efficiency of different client drones, including the statistical efficiency U. stat With system efficiency U sys The goal of this online federated learning system is to reduce the time-to-accuracy required for the global model to reach the desired accuracy. In this embodiment, two main factors influence time-to-accuracy: the number of training samples and the iteration speed. Higher quality and quantity of training samples lead to faster model accuracy improvement; however, this increases the time required for each round of global training. Conversely, reducing the number of training samples to increase iteration speed reduces the time required for each round of global training, but increases the number of rounds required to reach the desired accuracy. To address the trade-off between training samples and iteration speed, this system employs both statistical efficiency and system efficiency to evaluate these factors separately.
[0102] Statistical efficiency: Since the accuracy is improved by taking w i,k The derivative of the loss value, where the loss value originates from the estimation error between the predicted and actual results by the object detection model. A larger loss value leads to a faster improvement in accuracy. Therefore, the statistical efficiency U for the client drone k in the i-th global training iteration is... stat The definition is as follows:
[0103]
[0104] Here, loss(x,y) represents the loss function in machine learning. From the above definition, it can be seen that when the number of local training datasets... The larger the value, or the larger the loss value (loss()), the more likely it is to obtain... The larger the value, the higher the training value of the client drone k.
[0105] System Efficiency: Unlike traditional federated learning, the system efficiency of client drones in drone scenarios lacks stability. This is due to the rapid changes in the relative positions of client drones over time and their extreme sensitivity to communication interference. Therefore, we define the statistical efficiency U of the k client drones in the i-th global training as follows:
[0106]
[0107] Where T represents the specified deadline for global training. i,k This represents the actual time spent by customer k's drones during global training. And 1(T) <T i,k β is an indicator function. Once T i,k Beyond T, the coefficient changes from 1 to β. A significant decrease. Therefore, client-side drones with fast training speeds are more valuable for training. i,k From local training time τ l and communication time τ c Composition. α and β are used as attenuation weights. Considering that communication time is not negligible in the UAV scenario designed in this paper, Considering τ in T c The proportion of [the percentage of training time]. As defined, client-side drones with fast training speeds and short communication times are more valuable for training.
[0108] Therefore, the server drone performs the following efficiency evaluation for each customer drone:
[0109]
[0110]
[0111] Where m i,k This represents the selected client drone, where m i,k =1 indicates that client drone k is selected for the i-th round of training; otherwise, m i,k =0 indicates that no client drones were selected. The server drone will select the M most efficient client drones to participate in the training round before the start of each round of global training.
[0112] This embodiment adjusts the number of local training samples based on the computing performance of the airborne equipment, so that the client drone can train the most samples within a specified time. The main implementation method is shown in step 4.
[0113] In step 4, the sample quantity adjustment module 4 constructs a Shannon channel to more accurately calculate the communication time τ. c :
[0114]
[0115]
[0116] in, Indicates the transmission rate, |w i | Indicates the size of the transmitted data; B k and P k This represents the communication bandwidth and transmission power of the client drone k. N0 is the power spectral density of Gaussian noise. i,k This parameter represents the channel gain between the client drone k and the server drone, and is determined by the attenuation coefficient o. k and relative distance d i,k This constitutes the training time that can be allocated to the client drone.
[0117] Therefore, each client drone can estimate the number of samples for local training based on its computing performance:
[0118]
[0119] Among them, I k C represents the number of local iterations of the client drone k in a round of global training. k f represents the computation cycles required for the GPU of client drone k to compute a single sample. k This represents the computing power of the client drone k, specifically the number of computation cycles it can execute per unit of time.
[0120] This embodiment achieves model convergence speed by implementing sample screening based on the importance of local samples, thereby obtaining high-quality samples. The main implementation method is shown in step 5.
[0121] Based on the sample quantity D provided by sample quantity adjustment module 4 i,k The training sample selection module 5 is responsible for selecting samples for local training. Different samples have varying importance to the model. For example, a partially occluded target carries more information than an unoccluded target, meaning the former is more valuable for training. The training sample selection module 5 uses the sample's loss value (loss(x,y)) as an evaluation metric for importance. Furthermore, as the model continues to train, the importance of initially important samples gradually decreases. Therefore, the training sample selection module 5 needs to adaptively select samples based on the actual situation in each training round, thereby improving the statistical efficiency of the online federated learning system.
[0122] In the online federated learning system, client drones continuously supplement the online samples. Since the newly added online samples are collected in real time in an unknown environment, they are of the highest importance to local training in the current iteration. Therefore, the training sample selection module 5 sets the online samples as the highest priority, and the local training dataset... The possible combinations can be represented as:
[0123]
[0124] in, a represents the possible combinations of the selected training samples. i,k Indicates the number of possible combinations. To extract from... The training dataset with the highest importance is selected. The training sample selection module 5 records and updates the importance of each sample after each global training iteration, using this as a reference for the next round of sample selection. Simultaneously, after completing one round of global iteration, new online samples are reassigned to the old sample dataset. For the training samples in the i-th global training iteration, the optimal combination... It can be represented as:
[0125]
[0126] For example, to prevent low-importance samples from being forgotten in the long term and to enable the client drone to explore more possible combinations to achieve higher accuracy and generality, this embodiment employs an ε-greedy algorithm based on the original sample selection method. In the i-th global training iteration, the client drone k prefers the optimal training combination or explores unknown training combinations as follows:
[0127]
[0128] Specifically, the probability of the training sample selection module 5 selecting the optimal training combination is 1-ε, and the probability of randomly selecting from all combinations is ε. This allows the client-side UAV to strike a balance between utilizing the optimal method and exploring unknown methods.
[0129] This embodiment uses the Visdrone2021 UAV visual dataset as the original training samples and real-time acquired images as online learning samples. The performance improvement is tested by deploying different module algorithms proposed in this embodiment on the traditional federated learning algorithm (FedAvg). The functional modules include: client filtering (PS), sample filtering (SS), and an ε-greedy algorithm. Deploying all three functional modules simultaneously in FedAvg constitutes the method proposed in this invention (FedOL).
[0130] Figure 2(a) shows the accuracy variation curves of different functional modules for the target detection model during the model training process of this embodiment. By deploying the participant screening module, FedAvg achieves a shorter time-to-accuracy and higher average training efficiency than before, because the participant screening module can eliminate lagging client drones, thereby improving system efficiency. However, the performance of FedAvg actually decreases after deploying the sample screening module, because the sample selection module leads to insufficient training of the local model by the client drones. For client drones with poor communication quality or long transmission distances, the local training time is very short, resulting in only a small number of samples being selected for local training. Insufficient local model training hinders the improvement of model accuracy during the aggregation process. Based on this, FedAvg, which uses both client screening and sample screening, can eliminate the above defects and effectively improve the performance of FedAvg. Compared with the other four comparison combinations, FedOL in this embodiment has the best performance, the shortest time-to-accuracy, and the highest final accuracy, because the ε-greedy algorithm further improves the performance of the local model. When the model accuracy approaches the expected value, the ε-greedy algorithm shows a more significant improvement in accuracy because this module takes into account the less important samples that were ignored in the early stages of training.
[0131] Figure 2(b) shows the training efficiency curves of different functional modules for the target detection model during the model training process of this embodiment. In this embodiment, FedOL maintains the highest training efficiency for almost the entire training process. As the number of online samples increases, the training efficiency rises in the early stages of training, but decreases as the model is fully trained. The participant screening module proposed in this embodiment enables the server drone to avoid selecting lagging drones, further improving training efficiency.
[0132] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.
Claims
1. An online federated learning method for UAV visual target detection based on participant and sample selection, characterized in that, Includes the following steps: S1. Use the drone pod to collect real-time images and use the collected real-time images as new samples; S2. Manually label the new samples to create online samples, and add the online samples to the local dataset of the client drone; S3. Send the pre-set initial training model to all client drones; S4. The client drone will use the received initial training model as a local model and adjust the number of samples for local training according to the computing performance of its own onboard equipment and the specified local training time. S5. Based on the number of samples in the local training, samples with importance higher than a predetermined value are selected from the local dataset to obtain the training dataset; the client drone continuously supplements online samples, sets the current online samples as the highest priority, constructs a combination of local training datasets, and uses the loss value of the samples as an evaluation index of importance to select the optimal combination from the training samples of the global training. S6. Use the training dataset and the local model to perform local training to obtain real-time training data; S7. Combine the local models uploaded by multiple client drones with the real-time training data to form a global model required for the next round of training; S8. Calculate the total efficiency of each participant based on the real-time training data, and select client drones with a total efficiency higher than a predetermined value as participants in the next round of training; the total efficiency includes statistical efficiency and system efficiency, wherein the statistical efficiency is used to evaluate the training value of the training samples of the client drone, and the system efficiency is used to evaluate the iteration speed of the client drone. S9. Send the global model to the client drone selected in step S8; S10. Repeat steps S4 to S9 until the global model converges, then end the training.
2. The online federated learning method for UAV visual target detection according to claim 1, characterized in that, Step S4 adjusts the number of samples for local training based on the computing performance of the onboard equipment and the specified local training time, including: Constructing Shannon channels to calculate communication time : In the formula, Indicates the transmission rate. Indicates the size of the transmitted data; and This represents the communication bandwidth and transmission power of the client drone k; The power spectral density of Gaussian noise; The channel gain between the client drone k and the server drone is represented by the attenuation coefficient. and relative distance This allows for the calculation of the training time that can be allocated to the client drone. ; Therefore, each client drone estimates the number of samples for local training based on its computing performance. : in, This represents the number of local iterations performed by the client drone k in a single round of global training. This represents the number of computation cycles required for the GPU of client drone k to compute a single sample. This represents the computing power of the client drone k, specifically the number of computation cycles it can execute per unit of time.
3. The online federated learning method for UAV visual target detection according to claim 2, characterized in that, In step S5, samples with importance higher than a predetermined value are selected from the local dataset to obtain the training dataset, which includes: Client-side drones continuously replenish online samples Set the current online samples as the highest priority. Local training dataset The combination is represented as: In the formula, This represents the possible combinations of the selected training samples. Indicates the number of possible combinations; Indicates the number of samples used for local training; This represents a combination of locally trained datasets; For the training samples in the i-th global training iteration, the optimal combination Represented as: In the formula, Represents the loss function in machine learning. These represent the original sample data and the model inference results, respectively.
4. The online federated learning method for UAV visual target detection according to claim 1, characterized in that, Step S6 further includes: For the training dataset and the local model Perform local training, where i represents the global training round and k represents the drone number; It contains K client drones. Each drone carries a heterogeneous local training dataset. , ,in This represents the original image and the location of the target within the image. This indicates the size of the training dataset.
5. The online federated learning method for UAV visual target detection according to claim 4, characterized in that: The training dataset for client drones is used locally only and does not interact with other client drones; During online federated learning, each client drone k is based on its local dataset. Train the model; after multiple local iterations, wait for feedback on a new global model.
6. The online federated learning method for UAV visual target detection according to claim 5, characterized in that, Step S7 involves uploading local models from multiple client drones. The real-time training data is combined to form the global model required for the next round of training. : In the formula, This represents the local dataset; K represents the total number of client drones.
7. The online federated learning method for UAV visual target detection according to claim 1, characterized in that, The overall efficiency mentioned in step S8 includes statistical efficiency. With system efficiency ; Statistical efficiency of client drone k in the i-th global training session The definition is as follows: In the formula, Represents the loss function in machine learning; These represent the original sample data and the model inference results, respectively. Indicates the number of local training datasets; System efficiency for k client drones during the i-th global training iteration The definition is as follows: In the formula, T represents the specified time for local training; This represents the actual training time for customer k's drone; It is an indicator function, once When the value exceeds T, the coefficient changes from 1 to , Significant decline; From local training time and communication time composition; and Used as a decay weight; The server-side drones perform the following efficiency evaluation for each customer's drone: in Indicates the selected client drone, where This indicates that client drone k is selected for the i-th round of training. This indicates that no client-side drone was selected; Before each round of global training begins, the server-side drone selects the M most efficient client-side drones to participate in the training.
8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the online federated learning method for UAV visual target detection as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the online federated learning method for UAV visual target detection as described in any one of claims 1 to 7.