Autonomous cooperation method and system for low-altitude emergency federated learning client

Abstract

The invention discloses an autonomous cooperation method and system for a low-altitude emergency federated learning client, and aims to solve the problem of training efficiency and resource optimization of an unmanned aerial vehicle group in a complex environment. The method comprises the following steps: firstly, carrying out offline reinforcement learning on a pre-training strategy network through a simulation environment, and implementing a forced participation mechanism at the initial stage of deployment to accumulate real experience; secondly, each unmanned aerial vehicle encodes a local state into a message vector, exchanges and fuses with a neighbor to form a cooperative intention vector, and autonomously determines an action probability of participating in federated learning; then, in a training stage, the selected client executes local training and uploads update through differential gradient compression, and the server performs weighted aggregation according to gradient difference and a precision improvement value. And finally, performing strategy updating of centralized training and distributed execution by using the QMIX hybrid network according to the multi-target reward function, and realizing closed-loop collaborative optimization.

Description

Autonomous Collaborative Methods and Systems for Low-Altitude Emergency Federated Learning Clients Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and emergency disaster relief technology, specifically involving the integration technology of multi-agent reinforcement learning (MARL) and federated learning (FL). It is a method for autonomous collaboration of low-altitude emergency federated learning clients based on multi-agent intention interaction reinforcement learning, which aims to solve the problems of collaborative model training and dynamic resource optimization of UAV swarms in emergency scenarios. Background Technology

[0002] Traditional machine learning methods require centralized data on a central server for model training. This not only incurs high data transmission costs but also poses a serious risk of data privacy breaches due to the cross-domain flow of raw data. Furthermore, this centralized training model is also limited by the data silo problem, making it difficult to achieve collaborative optimization of data from multiple sources.

[0003] To address these challenges, Federated Learning (FL) has emerged as a privacy-preserving distributed machine learning paradigm. Its core mechanism involves clients training models locally using raw data, uploading only model updates (such as gradients or weights) to a central server, while the raw data remains locally. The server aggregates updates from multiple clients to generate a new global model, which is then distributed to clients for the next round of training. This architecture avoids the cross-device flow of raw data, fundamentally reducing the risk of privacy breaches, while fully utilizing the distributed computing power of edge devices to improve training efficiency and scalability.

[0004] Low-altitude drone swarms hold great promise for disaster relief. By collaboratively training global models through federated learning, efficient perception and intelligent decision-making regarding disaster areas can be achieved, enabling core tasks such as disaster assessment, life detection, and rescue planning. These global models collectively constitute the intelligent decision-making hub for post-disaster emergency response.

[0005] However, despite the advantages of distributed training, federated learning's traditional scheduling framework reveals a series of fundamental flaws when facing low-altitude UAV swarm collaboration in emergency scenarios, severely limiting its application effectiveness. Emergency scenarios are characterized by high dynamism, limited resources, and fragile communication; traditional federated learning methods have failed to effectively address these challenges.

[0006] 1. Rigid Client Selection Mechanism: Traditional flyback drone (FL) client selection mechanisms typically rely on random polling or simple static thresholds (such as battery level exceeding a fixed value), completely ignoring the dynamic constraints of the environment on computing power and communication. In actual deployments, the actual available computing power of a drone fluctuates constantly due to sudden obstacle avoidance calculations or high-temperature frequency throttling, causing selected "qualified" clients to fall behind during training due to resource exhaustion, forcing the entire training round to be delayed or even fail.

[0007] 2. Inefficient Synchronous Aggregation: Strictly synchronous aggregation mechanisms require waiting for all selected clients to return updates. This causes individual drones lagging behind due to insufficient computing power or unstable links to drag down the entire cluster, wasting valuable communication windows and severely conflicting with the minute-level timeliness required for emergency response.

[0008] 3. Huge communication overhead: Even for lightweight models, traditional solutions still require complete transmission of gradient updates. In emergency ad hoc networks with limited bandwidth and fragile links, not only is the update delay significant, but the retransmission caused by the high packet loss rate further squeezes out critical flight control signaling, exacerbating the problem of communication constraints.

[0009] 4. Lack of data value perception: Existing methods lack a synergistic perception of data value and resource status. Drones in low-risk areas may repeatedly transmit similar gradients, while nodes carrying high-value data (such as new changes in disaster conditions) or located in critical positions may be overlooked due to poor instantaneous communication, resulting in model training failing to effectively utilize key information.

[0010] In summary, the shortcomings of traditional federated learning methods in client selection, especially in emergency scenarios where there is insufficient comprehensive consideration of multiple factors such as the real-time status of UAVs, communication conditions, and data value, severely restrict their application potential in real-time disaster assessment and rescue decision-making. There is an urgent need for a collaborative innovation solution that can achieve dynamic, collaborative, and efficient client autonomy. Summary of the Invention

[0011] The application scenario of this invention is the training of a collaborative model for emergency scenarios involving low-altitude unmanned aerial vehicle (UAV) swarms. Typical tasks in this scenario include disaster assessment, life detection, and rescue planning. Highly dynamic emergency environments are characterized by extreme dynamism and multiple uncertainties: disasters evolve at minute-level speeds (e.g., fires spreading at speeds exceeding 5 m / s, aftershocks causing secondary collapses), forcing perception models to be updated rapidly; communication environments are highly constrained, with bandwidth fluctuating between 2 and 10 Mbps and packet loss rates reaching 15% to 40%, and multi-hop ad hoc networks introducing delays of 200 ms to 2 seconds and frequent local outages; UAV swarms face stringent resource constraints, with battery life of only 30 to 60 minutes and embedded computing power (≤1 TFLOPS) required to support sudden obstacle avoidance tasks, leading to real-time preemption of computing resources; data value distribution is extremely uneven, with scarce data in high-risk areas coexisting with redundant data in low-risk areas, and sudden changes in disaster conditions causing dramatic shifts in data distribution (e.g., feature drift in new landslide areas).

[0012] The system needs to achieve the following functional requirements under constraints such as limited communication, insufficient computing power, limited power, and strict timeliness: In terms of collaborative perception, it should integrate the status of neighbors within a 500-meter radius in real time and generate a global disaster heat map within minutes; in terms of dynamic scheduling, it should filter clients in real time based on power, risk, and data value, and reduce the consumption of drone resources during training; in terms of efficient training, it should improve the accuracy of the model in key areas and reduce the number of communication rounds through a value perception mechanism; and in terms of robust decision-making, it should automatically enable edge caching to resume transmission when communication is interrupted, ensuring the success rate of training.

[0013] To address the aforementioned background issues, the technical solution adopted by this invention is: a method for autonomous collaborative learning of low-altitude emergency federated learning clients, applied to a federated learning system for UAV swarms in low-altitude emergency scenarios. This method models the joint optimization problem of client selection and model aggregation as a Markov decision process, and achieves autonomous collaborative decision-making of the clients through multi-agent reinforcement learning. The method includes the following steps:

[0014] Initialization and pre-training steps: Construct a training space that closely resembles a real emergency scenario in a simulation environment, model multiple drones as a multi-agent system, pre-train the policy network through offline reinforcement learning, and force all clients to participate in federated learning for at least a preset number of rounds during actual deployment in order to generate an initial global model and accumulate real-world experience.

[0015] Intent interaction decision-making steps: Each UAV encodes its local state information into a message vector, exchanges it with neighboring UAVs within the communication range, and fuses the neighbor messages with its own generated local decision intent to form a cooperative intent vector, outputting the probability of the client participating in federated learning actions;

[0016] Federated learning training steps: Select the client to participate in training based on the action probability. The selected client performs local model training and uploads model updates using differential gradient compression. The server aggregates the local model updates based on the gradient difference and the improvement in model accuracy to generate a new global model.

[0017] Policy update steps: Based on a multi-objective dynamic reward function that includes model accuracy, communication overhead and data value, the QMIX hybrid network is used to perform centralized training and distributed execution updates of the multi-agent policy, and the updated policy parameters are synchronized to each UAV to form a dead loop collaborative optimization.

[0018] In some embodiments, the initialization and pre-training steps further include: a simulation space construction simulation pre-training step: constructing the simulation space containing dynamic environment parameters, resource constraints, and heterogeneous data distribution; a simulation pre-training step: modeling the UAV swarm as a multi-agent system in the simulation environment, defining client selection and model aggregation as Markov decision processes, initializing the state space, action space, and reward function, and using offline reinforcement learning to centrally optimize samples in the simulation experience pool to output a pre-trained policy network; a system deployment and synchronization simulation pre-training step: after the UAV swarm starts and establishes a basic communication link, the server broadcasts the initial global model, each UAV loads the pre-trained policy network, and synchronizes the initial parameters of the QMIX hybrid network; and a field fine-tuning and optimization simulation pre-training step: in the first three rounds of federated learning in actual deployment, all clients are forced to participate in training to cover the multi-dimensional data features of the real environment, and five-tuple experiences including state, action, reward, next state, and communication context are collected and uploaded to the server. The server fine-tunes the policy network and hybrid network parameters based on the global initial experience pool and broadcasts the updated parameters to all UAVs.

[0019] In some embodiments, the intent interaction decision step further includes: a local state acquisition step: each UAV acquires local state information including battery level, communication latency, environmental risk level, and CPU frequency, covering three consecutive time slots; a vector generation and encoding step: the local state is serialized using a gated recurrent unit (GRU) and encoded using a feedforward neural network. The network (FNN) encodes and generates local message vectors and local intent vectors; the neighborhood message fusion step: each UAV exchanges message vectors with neighboring UAVs within its communication range, and concatenates the received neighbor messages with its own local intent vector, and performs weighted aggregation processing through the feedforward neural network to generate a cooperative intent vector; the action sampling and decision step: the cooperative intent vector is input into the policy network, the probability of the client participating in federated learning is calculated and output, and the action probability is sampled in conjunction with dynamic masking rules to finally generate a binary decision action; and the server priority filtering step: the server receives the binary decision actions uploaded by each client, and if the number of participants exceeds a preset limit, it sorts the priorities according to environmental risk, communication quality and remaining power, and filters the final list of clients participating in federated training.

[0020] In some embodiments, the federated learning training steps further include: a local training and gradient compression simulation pre-training step: the clients participating in the training use global model weights and local sensor data to train the model and calculate local gradients, employing differential gradient compression technology to transmit only the changes in the local gradients relative to the previous gradient to the server, thereby reducing communication load; a local performance evaluation simulation pre-training step: after completing training, the clients use a lightweight local test set to evaluate model accuracy, calculate the accuracy improvement value between the current accuracy and the historical average accuracy of participating training, and record the training time cost as an indicator of client computing power; a data value evaluation and weighted aggregation simulation pre-training step: the server collects the local gradient changes of each participating client and calculates the weights of each client according to the data value evaluation system; and a model update simulation pre-training step: the server performs weighted averaging and fusion of the gradients of each client according to the weights, updating and generating a new global model.

[0021] In some embodiments, the weight allocation includes: a gradient difference simulation pre-training step: calculating the cosine similarity between the local gradient and the previous round of global gradient to measure the uniqueness of the gradient, with a higher weight assigned to a larger difference; and a precision improvement value simulation pre-training step: measuring the effectiveness of model evolution based on the validation results of the lightweight local test set.

[0022] In some embodiments, the policy update step includes: a three-dimensional dynamic reward calculation simulation pre-training step: designing a reward function that includes model accuracy reward, communication latency penalty, and gradient difference reward, and generating dynamic weight coefficients in real time through a neural network to dynamically allocate optimization resources among model accuracy, communication overhead, and data value; an experience storage and replay simulation pre-training step: during federated learning, recording quintuple data including the current state, action, real-time reward, next state, and communication context in real time, and storing it in an experience replay pool; a global value evaluation optimization simulation pre-training step: using the QMIX hybrid network architecture, combining supernetwork weights to fuse the local Q values ​​of each client to generate a global Q value, and using a gradient descent algorithm to update the hybrid network parameters and local value network parameters; and a policy update and synchronization simulation pre-training step: based on the guidance of the global Q value, adjusting the policy network parameters by maximizing long-term cumulative rewards, and distributing the updated policy network parameters and local value network parameters to each UAV for parameter coverage, achieving a dead loop optimization of decision consistency among all agents.

[0023] Another technical solution adopted by the present invention is an autonomous collaborative system for a low-altitude emergency federated learning client, which includes: an intent interaction module, a lightweight federated learning module, and a strategy optimization module;

[0024] The intent interaction module is used to encode the local observation data of the UAV into a message vector, and to generate a cooperative decision vector by fusing neighbor node information through the intent combination unit. The intent interaction module compresses the multi-dimensional state into a low-dimensional message vector through the encoding unit, and after receiving the neighbor message, it concatenates it with the local intent vector. Based on the policy parameters fed back by the policy optimization module, it uses a neural network to implicitly weight and generate a cooperative intent vector, and outputs the client's action selection probability by combining dynamic masking rules.

[0025] The lightweight federated learning module is used to dynamically select clients to participate in training based on the action probabilities, perform local model training, and transmit gradient updates using differential gradient compression technology. The lightweight federated learning module only transmits the change in gradient relative to the previous round, and combines the data value evaluation system to perform weighted aggregation of local model updates based on gradient difference and accuracy improvement value, and feeds back the aggregation results and communication status to the policy optimization module.

[0026] The policy optimization module receives feedback data from the lightweight federated learning module to calculate multi-objective dynamic rewards, stores the experience including message interaction data in the experience replay pool, and updates policy parameters through the QMIX hybrid network. The policy optimization module introduces a global communication summary into the hybrid network, uses a super network to dynamically generate a hybrid network weight matrix based on the global state, and integrates the communication context to guide the co-evolution of multi-agent policies.

[0027] In some embodiments, the dynamic masking rules in the intent interaction module include: real-time monitoring of the battery status and communication latency of each drone node, automatically filtering out nodes with battery levels below a preset threshold or communication latency above a preset delay, and inputting the filtered node status matrix into the encoding unit so that it does not participate in the client selection sampling of the current round, thereby avoiding invalid communication and network congestion.

[0028] In some embodiments, the data value assessment system in the lightweight federated learning module includes: setting priority transmission rules for high-risk areas or drone nodes with key data characteristics; in the model aggregation stage, using cosine similarity to calculate the difference between the local gradient and the previous round of global gradient, and combining the accuracy improvement value of the local test set, dynamically adjusting the weighting weight of the node in the global model aggregation, and transmitting the difference and accuracy improvement value as feedback data to the policy optimization module.

[0029] In some embodiments, the multi-objective dynamic reward in the policy optimization module specifically includes: the model accuracy improvement, communication efficiency penalty, and gradient value reward based on the feedback from the lightweight federated learning module, and the weight ratio between each item is adjusted in real time through the weight generation network so that the system achieves a dynamic balance between model convergence speed, energy consumption, and data validity, and thereby guides the QMIX hybrid network to update policy parameters.

[0030] The present invention has the following beneficial effects:

[0031] 1. Significantly improves the efficiency and accuracy of collaborative decision-making in emergency environments. Through the combination of intent interaction mechanism and gated loop unit (GRU), UAVs can integrate the dynamic status and decision intent of neighboring nodes within a 500-meter radius. Compared with traditional single-machine decision-making, this invention breaks through information silos and realizes the generation of global disaster heat maps in minutes, which can effectively cope with rapidly changing emergency scenarios such as fire spread and secondary collapse.

[0032] 2. Minimizing Load Pressure Caused by Communication Constraints: For harsh communication environments with large bandwidth fluctuations (2~10Mbps) and high latency (up to 2s), this invention employs "differential gradient compression technology," transmitting only the change in gradient relative to the previous iteration, fundamentally reducing the transmission of redundant data. Combined with dynamic masking rules and priority transmission mechanisms, it effectively avoids network congestion and ensures the real-time transmission of critical information even in extreme environments.

[0033] 3. Optimizing Dynamic Resource Allocation and Extending Equipment Endurance: This invention incorporates computing power fluctuations, battery life, and unexpected obstacle avoidance tasks into Markov Decision Process (MDP) modeling. Through multi-agent reinforcement learning, it autonomously selects the most suitable client to participate in training. This dynamic scheduling mechanism avoids unnecessary consumption of low-battery or high-load drones, achieving precise resource preemption and rational allocation under computing power constraints (≤1 TFLOPS), ensuring the continuity of disaster relief missions.

[0034] 4. Strengthen data value orientation and improve global model performance. Through a data value assessment system with two dimensions, "gradient difference" and "accuracy improvement value", the system can automatically identify scarce data in high-risk areas. This weighted aggregation mechanism ensures that updates that truly contribute to model evolution receive higher weights, solving the problem of extremely uneven data distribution in emergency scenarios and enabling the model to capture the characteristics of sudden disasters (such as feature drift) more quickly.

[0035] 5. Demonstrates high environmental robustness and the ability to evolve within a loop. Utilizing the QMIX hybrid network for "centralized training and distributed execution," and incorporating a three-dimensional dynamic reward function that includes model accuracy, communication overhead, and data value, the system can automatically adjust its optimization focus based on the evolution of the disaster. When local communication interruptions or sudden disaster changes occur, the system can continue operating by relying on the autonomous optimization capabilities of edge caching and the policy network, achieving loop-based collaborative optimization and significantly improving the system's generalization ability in non-stationary environments.

[0036] 6. Reduce the number of model convergence rounds and improve emergency response time. Through the simulation environment pre-training and the fine-tuning mechanism of mandatory participation in the first 3 rounds of on-site deployment, the system can quickly accumulate real experience in the early stage of actual deployment, shorten the convergence time of the reinforcement learning strategy, thereby reducing the number of communication rounds required to achieve the target accuracy and gaining valuable rescue time. Attached Figure Description

[0037] Embodiments of the present invention will be described with reference to the accompanying drawings, which are briefly described below.

[0038] Figure 1 is a schematic diagram of the initialization and pre-training steps of the collaborative method of the present invention.

[0039] Figure 2 is a schematic diagram of the collaborative method of the present invention and its intended interactive decision-making process.

[0040] Figure 3 is a schematic diagram of the federated learning training steps of the collaborative method of the present invention.

[0041] Figure 4 is a schematic diagram of the strategy update steps of the collaborative method of the present invention.

[0042] Figure 5 is a schematic diagram of the autonomous collaborative system of the low-altitude emergency federated learning client of the present invention.

[0043] In the accompanying drawings, the same reference numerals generally denote the same, functionally similar and / or structurally similar components.

[0044] Figure labeling: Intent Interaction Module - 10; Lightweight Federated Learning Module - 20; Policy Optimization Module - 30; Client - 40. Detailed Implementation

[0045] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate preferred embodiments of the application. However, this application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0047] Embodiments of this application are described herein with reference to cross-sectional views, which are preferred embodiments of this application. Variations in the illustrated shapes due to, for example, manufacturing techniques and / or tolerances are expected. Therefore, embodiments of this application should not be limited to the specific shapes of the regions shown herein, but include shape deviations due to, for example, manufacturing techniques. The regions shown in the figures are substantially schematic, and their shapes do not represent the actual shapes of regions of the device, nor do they limit the scope of embodiments of this application.

[0048] Please refer to Figures 1 to 4. Figure 1 is a schematic diagram of the initialization and pre-training steps of the collaborative method of the present invention; Figure 2 is a schematic diagram of the intention interaction decision-making process of the collaborative method of the present invention; Figure 3 is a schematic diagram of the federated learning training steps of the collaborative method of the present invention; and Figure 4 is a schematic diagram of the policy update steps of the collaborative method of the present invention.

[0049] This invention provides an autonomous collaborative method for low-altitude emergency federated learning clients, applied to a UAV swarm federated learning system in low-altitude emergency scenarios. It models the joint optimization problem of client selection and model aggregation as a Markov decision process, and achieves autonomous collaborative decision-making by the clients through multi-agent reinforcement learning. The method includes the following steps:

[0050] Example 1

[0051] As shown in Figure 1, the initialization and pre-training step S1 involves constructing a training space that closely resembles a real emergency scenario in a simulation environment. Multiple drones are modeled as a multi-agent system. The policy network is pre-trained through offline reinforcement learning, and during actual deployment, all clients are forced to participate in at least a preset number of federated learning rounds to generate an initial global model and accumulate real-world experience. The initialization and pre-training step S1 further includes:

[0052] Step S1.1 of constructing the simulation space: Based on the common characteristics of emergency scenarios, such as urban ruins, forest fires, and flood-stricken areas, a simulation space containing dynamic environmental parameters is constructed. In terms of environmental dynamics, bandwidth fluctuations (2~10Mbps), packet loss rate (15%~40%), and communication latency (200ms~2s) are simulated. In terms of resource constraints, the battery life of drones (30~60 minutes of flight time) and computing power fluctuations (≤1 TFLOPS, and considering the resource competition caused by sudden obstacle avoidance tasks) are simulated. In terms of data distribution, heterogeneous data of high-risk areas (such as collapse points and fire sources) and low-risk areas are generated to simulate the imbalance of data value. In terms of task scenarios, the model training requirements of typical tasks such as disaster assessment, life detection, and path planning are reproduced.

[0053] Simulation pre-training step S1.2: In the simulation environment, the UAV swarm is modeled as a multi-agent system, using "client selection-model aggregation" as the Markov decision process (MDP), and the state space, action space, and reward function are initialized. After client selection and training, a large number of "state-action-reward" samples are generated and stored in the simulation experience pool. Offline reinforcement learning is used for centralized optimization, iteratively optimizing the policy network parameters, enabling the agents to have preliminary collaborative decision-making capabilities in the simulation and outputting the pre-trained policy network (denoted as...). ).

[0054] System deployment and synchronization step S1.3: The drone swarm starts and establishes a basic communication link; the server broadcasts the initial global model, denoted as W0, to all drones; the drones load the pre-trained policy network. Initialize the local experience pool and synchronize the initial parameters of the QMIX hybrid network. ).

[0055] In the field, fine-tuning and optimization step S1.4 involves all clients participating in the first three rounds of federated learning. This ensures that the initial global model covers the multi-data features of the real environment and meets the needs of subsequent intent interactions for temporal state data. After local training, gradient transfer, and server aggregation, each drone uploads its "state-action-reward-next state-communication context" (no intent interaction occurs during the initialization phase) from the three training rounds to the server. The server aggregates this data to form a global initial experience pool. The server then optimizes the policy network based on this global experience pool. Then, the fine-tuned strategy network and hybrid network parameters are broadcast to all drones to complete the initialization operation.

[0056] Example 2

[0057] As shown in Figure 2, the intent interaction decision step S2 involves each UAV encoding its local state information into a message vector, exchanging it with neighboring UAVs within its communication range, and fusing the neighbor messages with its own generated local decision intent to form a collaborative intent vector. This vector outputs the probability of the client participating in federated learning. The intent interaction decision step S2 further includes:

[0058] Local state acquisition step S2.1: The central server broadcasts the global model to all drones. The drones collect information on drone battery level, communication latency, environmental risk level, and CPU frequency, and acquire local state Sn={S} for three consecutive time slots. n,t-2 ,S n,t-1 ,S n,t}

[0059] Vector generation and encoding step S2.2: A single-layer GRU (Gated Recurrent Unit) is used to process the three time-slot sequences. A 64-dimensional feature vector is calculated for each time slot t'. Initial state :

[0060] .

[0061] Next, the features are compressed to 32 dimensions using a single-layer FNN, and the ReLU activation function is used to filter out negative weight features, reducing communication overhead.

[0062] .

[0063] Neighborhood message fusion step S2.3: Share the GRU structure of the coding module, input S n,t Generate intent hidden state:

[0064] .

[0065] The Intent FNN maps the hidden state to a 16-dimensional intent vector:

[0066] .

[0067] 16-dimensional intent vector h n It provides a baseline for decision-making based on purely local information, independent of communication messages.

[0068] Action sampling and decision-making step S2.4: Each UAV n generates and broadcasts its local message m. n And receive neighbor messages: M n ={m1,m2,m3}

[0069] Concatenate the neighbor message with the intent vector:

[0070] .

[0071] Next, two layers of FNN with implicit weighting are used, the first layer being 112-dimensional to 64-dimensional:

[0072] ;

[0073] The second layer consists of 64-dimensional to 32-dimensional collaborative intent vectors:

[0074] ;

[0075] h i This indicates the collaborative decision-making intention of the drone after comprehensively considering its own decision-making tendencies and information about the surrounding environment.

[0076] Server-side priority filtering step S2.5: Collaboration intent vector h i Inputting a policy network (FNN) and using Softmax to output action probabilities:

[0077] ;

[0078] The dynamic masking rules are adjusted. If the battery level is too low or the communication latency is too high, the action value is forcibly set to 0. If the participation probability is greater than 80%, it is forcibly set to 1. Finally, the probability p is used to determine the action value. i Sampling generates binary actions: a i ∈{0,1}. After the server receives the actions from all clients, if the number of participating clients does not exceed the preset limit, it passes directly; otherwise, it prioritizes them according to environmental risk, communication quality, and remaining battery power, and selects the final list K of clients participating in federated training.

[0079] Example 3

[0080] As shown in Figure 3, federated learning training step S3: Based on the action probabilities, select the clients to participate in training. The selected clients perform local model training and upload model updates using differential gradient compression. The server aggregates the local model updates based on the gradient difference and the improvement in model accuracy, generating a new global model. The federated learning training step S3 includes:

[0081] Local training and gradient compression step S3.1: The central server will process the current deep learning global model w from the t-th round. t-1 The broadcast is sent to participating drone clients K={1,2,...,K}, and the drones utilize the received global model weights w. t-1The model is trained on a local dataset using local sensor data. The training process aims to minimize the cross-entropy loss function.

[0082] ;

[0083] Where B represents each local data batch, It is a local model for samples The predicted output; This represents the true label of the j-th sample, which is usually 0 or 1.

[0084] Update the weights of the local model using the gradient descent algorithm:

[0085] .

[0086] After E local iterations, the final local gradient is calculated, and differential gradient compression is used to transmit only the change of the local gradient relative to the previous round, thereby significantly reducing the communication load.

[0087] .

[0088] Local performance evaluation step S3.2: The UAV maintains a lightweight local test dataset, which is executed immediately after local training is completed: evaluate the new model using the test set to obtain accuracy. Subtract the average accuracy from each previous training session. Obtain the value of precision change Meanwhile, the time cost for each client in this process is defined as... Client computing power can be expressed as This affects the model training time.

[0089] Data value assessment and weighted aggregation step S3.3: Next, the agent calculates the gradient difference between the local gradient and the previous round's global gradient:

[0090] ;

[0091] Model update step S3.4: The server first collects the local gradients {ΔW} from all participating clients. i Subsequently, based on the data value assessment system, and combining two core factors—gradient difference and client accuracy improvement—each client was assigned a corresponding weight:

[0092] .

[0093] Among them, gradient dissimilarity measures the uniqueness of local gradients by calculating the cosine similarity between local and global gradients. Gradients with greater dissimilarity are considered to have a higher potential contribution to the global model. Accuracy improvement value verifies the effectiveness of model evolution based on a lightweight local test set, while ensuring the data voice in key areas and dynamically selecting effective updates that truly improve model performance.

[0094] Finally, the system merges the gradients from each client using a weighted average:

[0095] .

[0096] Example 4

[0097] As shown in Figure 4, the policy update step S4 involves: based on a multi-objective dynamic reward function that includes model accuracy, communication overhead, and data value, using a QMIX hybrid network to perform centralized training and distributed execution updates of the multi-agent policy, and synchronizing the updated policy parameters to each UAV to form a loop-based collaborative optimization. The policy update step S4 further includes:

[0098] Three-dimensional dynamic reward calculation step S4.1: This invention defines the joint optimization of client selection and model aggregation in emergency scenarios as a Markov Decision Process (MDP). Formally, a Markov Decision Process (MDP) is defined as a unary tuple.<S,A,P,R,γ> Let S represent the state space, A be the finite action space, P be the state transition probability function, R be the return value after taking a specific action to reach the next state, and γ be the discount factor for future rewards, where γ∈(0,1). We formulate the joint optimization of client selection and model aggregation as a control decision problem under a Markov decision process model. Here, we mainly give the specific definitions of S, A, and R.

[0099] State space: Define the state of all clients in round j as follows Each state It is a four-dimensional vector ,in This indicates the size of the data sample owned by the client; This indicates the risk level, which is related to the drone's location. Each drone maintains a lightweight local test dataset, which is used immediately after local training to evaluate the new model and obtain accuracy. Subtract the average accuracy from each previous training session. Obtain the value of precision change Meanwhile, the time cost for each client in this process is defined as... Client computing power can be expressed as This affects the model training time; It is the communication delay of user i in the previous round of model update.

[0100] Action Space: At the start of round j, the agent on each drone uses a policy network to output action probabilities p based on the fused messages and local intentions. i Ultimately, with probability p i Sampling generates binary actions: a i ∈{0,1}.

[0101] Reward: In the j-th round, the agent achieves the following in the current environmental state s j Next, execute action a j The environment will be in action a j The following generates a new state s j+1 At the same time, the intelligent agent will also receive r returned from the environment. j+1 According to the formula, our goal is to reduce the number of convergence epochs and training time while maintaining a given target accuracy. Therefore, the reward function in the j-th epoch is defined as follows:

[0102] ;

[0103] The definitions of each item are as follows:

[0104] 1. Model accuracy bonus (ΔAccuracy)

[0105] Calculation method: acc(t+1) - acc(t);

[0106] Features: It uses relative increases rather than absolute values ​​to eliminate the impact of initial model performance.

[0107] 2. Communication efficiency penalty item

[0108] Calculation method: exp(-Latency / τ), where τ=100ms is the time constant;

[0109] Total latency for client-side training (computation + communication);

[0110] Features: Exponential form smooths delayed mutations.

[0111] 3. Tiered Value Reward Items

[0112] Calculation method: ;

[0113] q∈[0,1] represents the risk factor for unmanned aerial vehicles (UAVs);

[0114] Features: Introducing risk factor q to enhance drone performance in high-risk areas;

[0115] Therefore, the total cumulative reward for an episode is defined as:

[0116] Where γ∈(0,1] is the discount factor for future rewards.

[0117] Experience storage and replay step S4.2: Update the policy network using an optimization method based on the QMIX hybrid network. During each round of federated learning, the agent performs client selection, local model training, and gradient compression and transfer according to the current policy, while simultaneously recording state-action transition data (including the current state s) in real time. t Action a t Instant rewards t Next state s t+1 and communication context M n This process continues until the current training episode ends.

[0118] Global value assessment optimization step S4.3: After the entire training round is completed, the local value network uses the current local value network parameters θ local Calculate the local Q-value for each action in the sampled data:

[0119] ;

[0120] The system utilizes a target hybrid network combined with the global state s global Generate a global Q-value Q from the local Q-values ​​of each agent. total :

[0121] ;

[0122] The target Q-value is generated by combining the Bellman equation with a target hybrid network (parameter delayed update):

[0123] ;

[0124] The hybrid network loss minimizes the mean square error between the predicted global Q-value and the target Q-value.

[0125] ;

[0126] The local value network loss is achieved by optimizing the consistency between the local Q-value and the global objective through backpropagation of the hybrid network.

[0127] ;

[0128] Among them, Q target,i It is determined by the discounting of local rewards and global Q-value.

[0129] Gradient descent optimization: The optimizer is used to calculate the gradient of the loss function and update the hybrid network parameters θ.mixer and local value network parameters θ local :

[0130] ;

[0131] ;

[0132] Where α is the learning rate.

[0133] Soft update mechanism: target network parameters θ target The main network parameters are smoothly synchronized using soft update coefficients τ:

[0134] ;

[0135] The policy network parameters are adjusted to maximize long-term cumulative returns, and the gradient is guided by the global Q-value.

[0136] ;

[0137] The parameter update formula is:

[0138] ;

[0139] Where β is the learning rate of the policy network.

[0140] Strategy update and synchronization step S4.4: The server distributes the updated policy network parameters to each drone. After receiving the parameters, the drone overwrites the old parameters on its local network, ensuring policy consistency across all agents. The drone then uses the updated policy network parameters to generate action decisions for the next round, forming a closed-loop optimization.

[0141] Example 5

[0142] Please refer to Figure 5, which is a schematic diagram of the autonomous collaborative system of the low-altitude emergency federated learning client of the present invention.

[0143] As shown in Figure 5, the present invention provides an autonomous collaborative system for a low-altitude emergency federated learning client, which includes: an intent interaction module 10, a lightweight federated learning module 20, and a strategy optimization module 30.

[0144] First, the intent interaction module 10 encodes the UAV's local observation data into message vectors and generates a collaborative decision vector by fusing neighboring node information through an intent combination unit. The intent interaction module 10 compresses multi-dimensional states into low-dimensional message vectors through an encoding unit, and after receiving neighbor messages, concatenates them with the local intent vector. Based on the policy parameters fed back by the policy optimization module 30, it uses a neural network to implicitly weight and generate a collaborative intent vector, combining dynamic masking rules to output the probability of the client 40 selecting an action. The dynamic masking rules in the intent interaction module 10 include: real-time monitoring of the battery status and communication latency of each UAV node, automatically filtering out nodes with battery levels below a preset threshold or communication latency exceeding a preset delay, and inputting the filtered node state matrix into the encoding unit to prevent them from participating in the current round of client 40 selection sampling, thereby avoiding invalid communication and network congestion.

[0145] Secondly, the lightweight federated learning module 20 dynamically selects clients 40 to participate in training based on the action probabilities, performs local model training, and transmits gradient updates using differential gradient compression technology. The lightweight federated learning module 20 only transmits the change in gradient relative to the previous round, and, in conjunction with a data value assessment system, performs weighted aggregation of local model updates based on gradient difference and accuracy improvement, feeding back the aggregation results and communication status to the policy optimization module 30. The data value assessment system in the lightweight federated learning module 20 includes: setting priority transmission rules for drone nodes in high-risk areas or with key data characteristics; during the model aggregation stage, using cosine similarity to calculate the difference between the local gradient and the previous round's global gradient, and combining this with the accuracy improvement value of the local test set, dynamically adjusting the weighting of the node in the global model aggregation, and transmitting the difference and accuracy improvement value as feedback data to the policy optimization module 30.

[0146] Furthermore, the policy optimization module 30 receives feedback data from the lightweight federated learning module to calculate multi-objective dynamic rewards, stores the experience including message interaction data in the experience replay pool, and updates policy parameters through the QMIX hybrid network. The policy optimization module 30 introduces a global communication summary into the hybrid network, uses a supernetwork to dynamically generate a hybrid network weight matrix based on the global state, and integrates communication context to guide the collaborative evolution of multi-agent policies. The multi-objective dynamic rewards in the policy optimization module 30 specifically include: model accuracy improvement, communication efficiency penalty, and gradient value reward based on feedback from the lightweight federated learning module 20. The weight ratios between these items are adjusted in real-time through a weight generation network to achieve a dynamic balance between model convergence speed, energy consumption, and data validity, thereby guiding the QMIX hybrid network to update policy parameters.

[0147] While various embodiments of the invention have been described above, it should be understood that these are presented by way of example only and not as limiting. Therefore, the scope of the invention should not be limited to the above embodiments, but should be defined solely by the claims and their equivalents.

Claims

1. A collaborative method for autonomous low-altitude emergency federated learning clients, characterized in that, A federated learning system for UAV swarms applied to low-altitude emergency scenarios includes the following steps: Initialization and pre-training step: Constructing a training space closely resembling a real emergency scenario in a simulation environment, modeling multiple UAVs as a multi-agent system, pre-training the policy network through offline reinforcement learning, and forcing all clients to participate in at least a preset number of federated learning rounds during actual deployment to generate an initial global model and accumulate real-world experience; Intent interaction and decision-making step: Each UAV encodes its local state information into a message vector, exchanges it with neighboring UAVs within its communication range, and fuses the neighbor messages with its own generated local decision intent to form a cooperative intent. The vector outputs the action probability of the client participating in federated learning; the federated learning training steps are as follows: the client participating in training is selected according to the action probability, the selected client performs local model training and uploads the model update using differential gradient compression, the server aggregates the local model update based on the gradient difference and the improvement value of model accuracy, and generates a new global model; the policy update steps are as follows: based on a multi-objective dynamic reward function that includes model accuracy, communication overhead and data value, the QMIX hybrid network is used to perform centralized training and distributed execution updates of the multi-agent policy, and the updated policy parameters are synchronized to each UAV to form a dead loop collaborative optimization.

2. The collaborative method according to claim 1, characterized in that, The initialization and pre-training steps further include: a simulation space construction step: constructing a simulation space containing dynamic environmental parameters, resource constraints, and heterogeneous data distribution; a simulation pre-training step: modeling the UAV swarm as a multi-agent system in the simulation environment, defining client selection and model aggregation as Markov decision processes, initializing the state space, action space, and reward function, and using offline reinforcement learning to centrally optimize the samples in the simulation experience pool, outputting the pre-trained policy network; a system deployment and synchronization step: after the UAV swarm starts and establishes a basic communication link, the server broadcasts the initial global model, each UAV loads the pre-trained policy network, and synchronizes the initial parameters of the QMIX hybrid network; and a field fine-tuning and optimization step: in the first three rounds of federated learning in actual deployment, all clients are forced to participate in training to cover the multi-dimensional data features of the real environment, and five-tuple experiences containing state, action, reward, next state, and communication context are collected and uploaded to the server. The server fine-tunes the parameters of the policy network and hybrid network based on the global initial experience pool and broadcasts the updated parameters to all UAVs.

3. The collaborative method according to claim 1, characterized in that, The intent interaction decision-making steps further include: a local state acquisition step: each UAV acquires local state information including battery level, communication latency, environmental risk level, and CPU frequency, covering three consecutive time slots; a vector generation and encoding step: the local state is serialized using a gated recurrent unit, and a local message vector and a local intent vector are generated by encoding through a feedforward neural network; a neighborhood message fusion step: each UAV exchanges message vectors with neighboring UAVs within its communication range, and concatenates the received neighbor messages with its own local intent vector, performing weighted aggregation processing through the feedforward neural network to generate a collaborative intent vector; an action sampling and decision-making step: the collaborative intent vector is input into the policy network, the probability of the client participating in federated learning is calculated and output, and the action probability is sampled using dynamic masking rules to finally generate a binary decision action; and a server-side priority filtering step: the server receives the binary decision actions uploaded by each client, and if the number of participants exceeds a preset limit, it sorts the clients according to environmental risk, communication quality, and remaining battery power to filter the final list of clients participating in federated training.

4. The collaborative method according to claim 1, characterized in that, The federated learning training steps further include: a local training and gradient compression step: the participating clients use global model weights and local sensor data to train the model and calculate local gradients. Differential gradient compression technology is used to transmit only the change of the local gradient relative to the previous gradient to the server, thereby reducing communication load; a local performance evaluation step: after completing training, the client uses a lightweight local test set to evaluate the model accuracy, calculates the accuracy improvement value between the current accuracy and the historical average accuracy of participating training, and records the training time cost as an indicator of the client's computing power; a data value evaluation and weighted aggregation step: the server collects the local gradient changes of each participating client and calculates the weight of each client according to the data value evaluation system; and a model update step: the server performs a weighted average fusion of the gradients of each client according to the weights, and updates and generates a new global model.

5. The collaborative method according to claim 4, characterized in that, The weight allocation includes: a gradient difference step: calculating the cosine similarity between the local gradient and the previous global gradient to measure the uniqueness of the gradient; the greater the difference, the higher the weight is assigned; and a precision improvement step: measuring the effectiveness of model evolution based on the validation results of the lightweight local test set.

6. The collaborative method according to claim 1, characterized in that, The policy update steps include: a three-dimensional dynamic reward calculation step: designing a reward function that includes model accuracy reward, communication latency penalty, and gradient difference reward, and generating dynamic weight coefficients in real time through a neural network to dynamically allocate optimization resources among model accuracy, communication overhead, and data value; an experience storage and replay step: during federated learning, recording five-tuple data containing the current state, action, real-time reward, next state, and communication context in real time, and storing it in an experience replay pool; a global value evaluation and optimization step: using the QMIX hybrid network architecture, combining supernetwork weights to fuse the local Q values ​​of each client to generate a global Q value, and using a gradient descent algorithm to update the hybrid network parameters and local value network parameters; and a policy update and synchronization step: based on the guidance of the global Q value, adjusting the policy network parameters by maximizing long-term cumulative rewards, and distributing the updated policy network parameters and local value network parameters to each drone for parameter coverage, achieving a loop optimization for consistent decision-making among all agents.

7. A low-altitude emergency federated learning client autonomous collaborative system, characterized in that, include: The module includes an intent interaction module, a lightweight federated learning module, and a policy optimization module. The intent interaction module encodes local UAV observation data into message vectors and fuses neighboring node information through an intent combination unit to generate a collaborative decision vector. The intent interaction module compresses multi-dimensional states into low-dimensional message vectors through an encoding unit, and after receiving neighbor messages, concatenates them with the local intent vector. Based on the policy parameters fed back by the policy optimization module, it uses a neural network to implicitly weight and generate a collaborative intent vector, and outputs the client's action selection probability using dynamic masking rules. The lightweight federated learning module dynamically selects clients to participate in training based on the action probabilities, performs local model training, and transmits gradient updates using differential gradient compression technology. The lightweight federated learning module only transmits the change in gradient relative to the previous round, and combines a data value evaluation system to perform weighted aggregation of local model updates based on gradient difference and accuracy improvement, feeding back the aggregation results and communication status to the policy optimization module. The policy optimization module is used to receive feedback data from the lightweight federated learning module to calculate multi-objective dynamic rewards, store the experience containing message interaction data in the experience replay pool, and update policy parameters through the QMIX hybrid network. The policy optimization module introduces a global communication summary in the hybrid network, uses the super network to dynamically generate a hybrid network weight matrix based on the global state, and integrates the communication context to guide the co-evolution of multi-agent policies.

8. The collaborative system according to claim 7, characterized in that, The dynamic masking rules in the intent interaction module include: real-time monitoring of the battery status and communication latency of each drone node, automatically filtering out nodes with battery levels below a preset threshold or communication latency above a preset delay, and inputting the filtered node status matrix into the encoding unit so that it does not participate in the client selection sampling of the current round, thereby avoiding invalid communication and network congestion.

9. The collaborative system according to claim 7, characterized in that, The data value assessment system in the lightweight federated learning module includes: setting priority transmission rules for drone nodes in high-risk areas or with key data features; in the model aggregation stage, using cosine similarity to calculate the difference between the local gradient and the previous round of global gradient, and combining the accuracy improvement value of the local test set, dynamically adjusting the weighting weight of the node in the global model aggregation, and transmitting the difference and accuracy improvement value as feedback data to the policy optimization module.

10. The collaborative system according to claim 7, characterized in that, The multi-objective dynamic reward in the policy optimization module specifically includes: the improvement in model accuracy, the communication efficiency penalty, and the gradient value reward based on the feedback from the lightweight federated learning module. The weight ratio between these items is adjusted in real time through the weight generation network to achieve a dynamic balance between model convergence speed, energy consumption, and data validity, thereby guiding the QMIX hybrid network to update policy parameters.

Images