Old perception single federal learning method for solving heterogeneity

Abstract

The invention relates to the technical field of privacy computing, in particular to an old perception single federal learning method for solving heterogeneity. After receiving the global model issued by the server, the client carries out model training locally, maintains a model pool, adds the trained model into the model pool, aggregates the models in the model pool through an index perception algorithm after the training of the model pool is completed, and uploads the aggregated model to a road side unit; in a certain time slice, the road side unit aggregates the models uploaded by all the clients and received in the time slice through an old perception algorithm, and sends the aggregated model to the server; and after receiving the aggregation models uploaded by the road side units, the server performs weighted average on the aggregation models according to the data volume of each road side unit to obtain a final global model. According to the method, the interference of data heterogeneity and system heterogeneity on the accuracy of the federal learning model is effectively relieved, the communication overhead is effectively reduced, and the accuracy and robustness of the model are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of privacy computing technology, specifically to a single-batch federated learning method. Background Technology

[0002] Federated Learning (FL) enables collaborative training of highly available global models across clients, provided their data remains within the same domain. However, in certain scenarios, such as autonomous driving, directly applying FL faces key challenges including data heterogeneity, system heterogeneity, and frequent exits. One-Shot Federated Learning effectively addresses the high communication overhead of traditional FL, where the client and server communicate only once to complete global model training. While this significantly reduces the training cycle of traditional FL, in scenarios with highly heterogeneous data distribution and systems, the global model trained through one-shot federated learning exhibits poor generalization ability, severely limiting its practical applicability and operational reliability. One-shot federated learning faces the dual challenges of data and system heterogeneity in practical deployments, prompting existing research to explore multi-faceted technological evolution paths. To address the issue of data heterogeneity, mainstream solutions primarily focus on optimizing objective design and innovating training mechanisms: introducing proximal regularization constraints into the local objective function effectively limits the tendency of client models to deviate from the global optimum; for one-time federated learning scenarios, open-set voting and adversarial outlier augmentation techniques significantly improve the model's discriminative ability under skewed label distributions; ensemble learning methods enhance the system's adaptability to non-independent and identically distributed data by constructing diverse model pools and designing distance metric criteria; furthermore, personalized frameworks based on clustering and multi-task learning further enable differentiated modeling among clients. Regarding system heterogeneity, technological development mainly focuses on resource adaptation and efficiency improvement: asynchronous learning frameworks alleviate participation problems caused by uneven device resources by designing age-aware weight aggregation and adaptive scheduling strategies; model compression techniques provide scalable model deployment solutions for devices with different capabilities through dynamic pruning, knowledge distillation, and low-rank decomposition; and the combination of quantized communication and edge association mechanisms significantly reduces communication overhead while ensuring accuracy.

[0003] However, the aforementioned studies mostly focus on single-dimensional heterogeneity challenges, such as optimizing data distribution (non-independent and identically distributed) or client resource differences. While these solutions improve model performance and usability in their respective areas, they struggle to address the combined challenges of data heterogeneity, system heterogeneity, and the feasibility of federated learning in highly dynamic vehicular networking scenarios. Especially in environments with limited communication and significant resource differences, existing methods often fail to balance model accuracy and convergence efficiency with practical deployment and robustness. Therefore, a federated learning solution capable of simultaneously handling multiple heterogeneities and adapting to complex scenarios is urgently needed. Summary of the Invention

[0004] The purpose of this invention is to provide a stale perception-based single-pass federated learning method that addresses heterogeneity. This method is applicable to real-world scenarios such as autonomous driving, effectively mitigating the interference of non-independent identically distributed data and system heterogeneity on the accuracy of federated learning models, reducing communication overhead, and significantly improving the accuracy and robustness of the model.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] An outdated perception-based single-pass federated learning method for addressing heterogeneity, the technique comprising:

[0007] Step 1: After receiving the global model from the server, the client maintains a model pool locally, performs multiple model training sessions, and adds the trained models to the model pool.

[0008] Step 2: The client aggregates models in the model pool using the index-aware algorithm and uploads the aggregated models to the roadside unit.

[0009] Step 3: Within a time slice, the roadside unit aggregates all client-uploaded models received in that time slice using the outdated sensing algorithm based on the received client-uploaded models, and sends the aggregated model to the server.

[0010] Step 4: After receiving the aggregated models uploaded by the roadside units, the server performs a weighted average of these aggregated models according to the amount of data in the roadside units to obtain the final global model.

[0011] The beneficial effects of this solution are: it effectively alleviates the interference of non-independent identically distributed data and system heterogeneity on the accuracy of federated learning models, reduces communication overhead, significantly improves the accuracy and robustness of the model, and is suitable for real-world scenarios such as autonomous driving.

[0012] Furthermore, in step 1, the client locally maintains a model pool consisting of three parts: model initialization, loss function design, and model training.

[0013] Furthermore, the model initialization is done by the client. Received the global model from the aggregation server Then, based on the model pool The weighted average of existing models is used to obtain the initial model, and the calculation formula is as follows:

[0014] ,

[0015] in, This represents the index of an existing model in the model pool. This represents the total number of existing models in the model pool.

[0016] Furthermore, the loss function is designed to explore model diversity and prevent local models from deviating too much from the global model; to enhance local model diversity during training and prevent the currently trained model from being too similar to existing models in the model pool, a constraint term is introduced. The calculation formula is as follows:

[0017] ,

[0018] in, Indicates the L2 norm distance between model parameters;

[0019] Meanwhile, to prevent the current trained model from deviating too much from the global model, regularization constraints are introduced. To constrain the current model and the initial global model The distance between them is calculated using the following formula:

[0020] ;

[0021] This constraint ensures that the current model does not deviate excessively from the initial global model during local training, which helps to mitigate client drift caused by non-independent and identically distributed data and adapt to system heterogeneity;

[0022] Ultimately, the client's overall local training loss function The calculation formula is as follows:

[0023] ,

[0024] in, It is the original loss function. It is a client Local dataset, and These are the hyperparameters that balance the strength of diversity constraints and global constraints, respectively.

[0025] Furthermore, the model training is done on the client side. After the local model is initialized, each model The same parameters will be used. Each training cycle includes hyperparameters such as learning rate and batch size. After training, the model will be added to the client. Local model pool middle.

[0026] The beneficial effects of the above-mentioned further solutions are as follows: designing loss functions can explore the diversity of models and ensure model convergence, thereby improving the accuracy and robustness of the global model; the model pooling mechanism can ensure that the client can train offline, ensuring the stability of the federated learning training process and further improving the accuracy and robustness of the global model.

[0027] Furthermore, in step 2, the client aggregates models from the model pool using an index-aware algorithm and uploads the aggregated models to the roadside unit, specifically as follows:

[0028] Client Model pool After all models in the pool have been trained, the index-aware algorithm is used to aggregate all models in the model pool. The calculation formula is as follows:

[0029] ,

[0030] The resulting aggregated local model Federated learning rounds participated in by the client Send to its corresponding roadside unit This averaging strategy is based on the assumption that model accuracy and overall performance will improve as the number of local training rounds increases.

[0031] The beneficial effects of the above-mentioned further scheme are: by using the index-aware algorithm, the interference of the low-precision training model in the early stage on the global model is reduced, thus ensuring the convergence of the global model.

[0032] Furthermore, in step 3, within a time slice, the roadside unit aggregates all client-uploaded models received in that time slice using an outdated sensing algorithm based on the received client-uploaded models, and sends the aggregated models to the server, specifically as follows:

[0033] The roadside unit defines a fixed aggregation time window. Within this window, the roadside unit performs weighted aggregation on all received local aggregation models uploaded by clients, using the following calculation formula:

[0034] ,

[0035] in Indicates the roadside unit within the time window The total number of local models received internally. This represents the total number of global aggregations performed in federated learning. It's important to note that this method does not require roadside units to wait for all clients to upload their models; aggregations are performed at the end of the window, effectively reducing idle waiting time for computationally efficient clients and improving overall system training efficiency.

[0036] Finally, the roadside unit uploads the aggregation model. This allows the server to perform subsequent global model updates.

[0037] The beneficial effects of the above further scheme are: the roadside unit is set to a fixed time slot. Aggregation is performed only on models received within the specified time slice, eliminating the need to wait for all clients to upload their aggregated models. This significantly reduces the waiting time for high-performance clients while ensuring that low-performance clients participate in aggregation as much as possible, thus guaranteeing the accuracy of the global model. By assigning a small weight to client models that participate in fewer rounds of the global model using the staleness-aware algorithm, the impact of these models on the accuracy of the global model is reduced, ensuring the accuracy of the global model.

[0038] Furthermore, in step 4, after receiving the aggregated models uploaded by the roadside units, the server performs a weighted average of these aggregated models according to the data volume of the roadside units to obtain the final global model, specifically as follows:

[0039] The server receives the aggregated model from the roadside unit. Then, a global aggregation operation is performed, and the calculation formula is as follows:

[0040] ,

[0041] in This indicates the total number of roadside units involved in the current aggregation. Indicates roadside unit The sum of data from all clients in the process. .

[0042] The beneficial effects of the above-mentioned further scheme are: weighted averaging based on the amount of data reduces the interference of the model trained on a small amount of data on the global model, thus ensuring the convergence of the global model. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating a stale perception single-pass federated learning method for addressing heterogeneity, provided by an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of local model maintenance for a stale perception single federated learning method for solving heterogeneity provided in an embodiment of the present invention;

[0046] Figure 3 This is a system model diagram of an outdated perception-based single-pass federated learning method for solving heterogeneity, provided by an embodiment of the present invention.

[0047] Figure 4 This is an experimental graph showing the effect of different hyperparameter values ​​on accuracy during ablation in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0049] like Figure 1 As shown in this embodiment, a stale perception single-pass federated learning method for addressing heterogeneity includes the following steps:

[0050] Step 1: After receiving the global model from the server, the client maintains a model pool locally, performs multiple model training sessions, and adds the trained models to the model pool.

[0051] In this step, the client locally maintains a model pool consisting of three parts: model initialization, loss function design, and model training.

[0052] Model initialization is client-side Received the global model from the aggregation server Then, based on the model pool The weighted average of existing models is used to obtain the initial model, and the calculation formula is as follows:

[0053] ,

[0054] in, This represents the index of an existing model in the model pool. This represents the total number of existing models in the model pool.

[0055] The loss function is designed to explore model diversity and prevent the local model from deviating too much from the global model. To enhance local model diversity during training and prevent the currently trained model from being too similar to existing models in the model pool, a constraint term is introduced. The calculation formula is as follows:

[0056] ,

[0057] in, Indicates the L2 norm distance between model parameters;

[0058] Meanwhile, to prevent the current trained model from deviating too much from the global model, regularization constraints are introduced. To constrain the current model and the initial global model The distance between them is calculated using the following formula:

[0059] ;

[0060] This constraint ensures that the current model does not deviate excessively from the initial global model during local training, which helps to mitigate client drift caused by non-independent and identically distributed data and adapt to system heterogeneity;

[0061] Ultimately, the client's overall local training loss function The calculation formula is as follows:

[0062] ,

[0063] in, It is the original loss function. It is a client Local dataset, and These are the hyperparameters that balance the strength of diversity constraints and global constraints, respectively.

[0064] Model training is on the client side. After the local model is initialized, each model The same parameters will be used. Each training cycle includes hyperparameters such as learning rate and batch size. After training, the model will be added to the client. Local model pool middle.

[0065] An implementation method for maintaining a local model pool, such as... Figure 2 As shown, the client First, train the first model in the model pool based on the global model. And add the model to the model pool; the second model Initialize to And train locally using the same hyperparameters, then add the model to the model pool after training is complete; the third model Initialize to And train locally using the same hyperparameters, then add the model to the pool after training is complete... the | A model Initialize to The model is then trained locally using the same hyperparameters, and added to the model pool after training. This completes the client-side setup. The local model pool is used for training, and the training process for other clients is the same.

[0066] Step 2: The client aggregates models in the model pool using the index-aware algorithm and uploads the aggregated models to the roadside unit.

[0067] In this step, the client Model pool After all models in the pool have been trained, the index-aware algorithm is used to aggregate all models in the model pool. The calculation formula is as follows:

[0068] ,

[0069] The resulting aggregated local model Federated learning rounds participated in by the client Send to its corresponding roadside unit This averaging strategy is based on the assumption that model accuracy and overall performance will improve as the number of local training rounds increases.

[0070] Step 3: Within a time slice, the roadside unit aggregates all client-uploaded models received in that time slice using the outdated sensing algorithm based on the received client-uploaded models, and sends the aggregated model to the server.

[0071] In this step, the roadside unit defines a fixed aggregation time window. Within this window, the roadside unit performs weighted aggregation on all received local aggregation models uploaded by clients, using the following calculation formula:

[0072] ,

[0073] in Indicates the roadside unit within the time window The total number of local models received internally. This represents the total number of global aggregations performed in federated learning. It's important to note that this method does not require roadside units to wait for all clients to upload their models; aggregations are performed at the end of the window, effectively reducing idle waiting time for computationally efficient clients and improving overall system training efficiency.

[0074] Finally, the roadside unit uploads the aggregation model. This allows the server to perform subsequent global model updates.

[0075] Step 4: After receiving the aggregated models uploaded by the roadside units, the server performs a weighted average of these aggregated models according to the amount of data in the roadside units to obtain the final global model.

[0076] In this step, the server receives the aggregated model from the roadside unit. Then, a global aggregation operation is performed, and the calculation formula is as follows:

[0077] ,

[0078] in This indicates the total number of roadside units involved in the current aggregation. Indicates roadside unit The sum of data from all clients in the process. .

[0079] An implementation of an outdated perception-based single-pass federated learning method to address heterogeneity, such as... Figure 3 As shown, the client first trains a model pool locally. After the model pool is trained, it aggregates all models in the model pool according to the index-aware algorithm and uploads the aggregated model to the roadside unit. The roadside unit aggregates the models uploaded by the client using the outdated awareness algorithm and sends the aggregated model to the server. The server performs a weighted average of the aggregated models uploaded by the roadside unit according to the amount of data to obtain the final aggregated model, thus completing one communication process of the method.

[0080] Experimental verification of an outdated perception-based single-pass federated learning method for addressing heterogeneity in this invention:

[0081] 1. Experimental Environment

[0082] The experimental platform built in this invention was mainly conducted on the Windows 10 operating system. The equipment configuration information is as follows: CPU 2.30GHz, RAM 16.0GB, GPU NVIDIA GeForce GTX 3060, CUDA version 13.0, programming language Python 3.10, and PyTorch framework for experiments.

[0083] 2. Experimental verification

[0084] Accuracy Analysis: 1) One Federated Scenario: To verify the effectiveness of this method, tests were conducted on four datasets with 100 and 200 local training epochs for two scenarios: label offset (CIFAR-10, Tiny-ImageNet) and domain offset (PACS, Office-Caltech-10). The experimental results are shown in Table 1. The results show that this method consistently outperforms existing approaches on all four datasets. Specifically, on the CIFAR-10 dataset, this method improves accuracy by more than 15% compared to state-of-the-art federated learning methods. Furthermore, it also achieves a stable gain of approximately 2% on the PACS and Office-Caltech-10 datasets, consistently demonstrating the effectiveness of the proposed method. It is worth noting that on the Tiny ImageNet dataset, which contains 200 classes, almost all methods (including this method) struggle to provide satisfactory performance. This is mainly due to the extremely imbalanced data distribution between different classes, making it difficult for existing federated learning schemes to handle such tasks. However, even in this case, this method still achieves approximately 1% higher accuracy than other baselines.

[0085] Table 1. Accuracy comparison of different schemes on different datasets in a single federated learning scenario (accuracy) (Standard deviation %)

[0086]

[0087] 2) Few-order Federated Learning: Although this method is specifically designed for single-order federated learning, its scalability in few-order federated learning settings was further evaluated. The experimental results are shown in Table 2. The results show that in few-order federated learning settings, this method consistently outperforms the baseline method, further confirming its effectiveness and scalability. However, it should be noted that the model pooling mechanism introduced in this method's design increases computational and training costs in few-order federated learning.

[0088] Table 2. Accuracy comparison on the PACS dataset in a few-time federated learning scenario (%)

[0089]

[0090] 3) Different Communication Rounds Settings: To evaluate the effectiveness of the legacy perceptual aggregation algorithm, experiments were conducted on the CIFAR-10 dataset. The total number of global federated learning rounds was set to 10, with each client participating in different rounds. The number of training rounds was 100 and 200 to simulate a typical asynchronous participation scenario, and the experimental results are shown in Table 3. The results show that, with the number of local training rounds being 100 and 200 respectively, this method outperforms other baseline methods. This is mainly attributed to the stale perception algorithm in this method, which effectively mitigates the impact of differences in the number of client participation rounds, thereby reducing its interference with the convergence and stability of the global model.

[0091] Table 3. Accuracy comparison on the CIFAR-10 dataset (%)

[0092]

[0093] Communication overhead analysis: This method requires each client to train | There are 1 model, therefore the total training time for each client is 1. , and the total training time is Compared to traditional federated learning schemes, this method has a higher computational cost. However, due to the outdated sensing asynchronous aggregation algorithm on roadside units, even if some clients are not within the time window... Training of the local model pool is completed internally, and the entire federated learning process remains unaffected. Furthermore, as shown in Table 1, this method achieves optimal model performance on the mentioned datasets without incurring additional communication overhead. Therefore, this method achieves an effective trade-off between computational cost and performance gain.

[0094] Ablation experiment analysis: 1) The impact of setting hyperparameters on accuracy: The constraint terms were evaluated. , and model pool size The impact of the experimental results on this method is shown in Table 4. The results indicate that, compared to using only the model pool... In contrast, introducing constraint terms separately or This can improve performance, and using both constraints simultaneously will further improve accuracy. It's important to note that even without... and Only use the model pool Even after training, this method still outperforms most baseline methods.

[0095] Table 4. The impact of setting hyperparameters on accuracy (%)

[0096]

[0097] 2) The impact of different hyperparameter values ​​on accuracy: The sensitivity of this method to different hyperparameter values ​​was further investigated. The experimental results are as follows: Figure 4 As shown. Experimental results show that this method is almost unaffected by the hyperparameter values, meaning it is robust to hyperparameter variations; furthermore, from Figure 4 As can be seen from (3), blindly increasing the size of the model pool will not bring significant performance improvement to this method.

[0098] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A stale perception-based single-pass federated learning method for addressing heterogeneity, characterized in that, The method includes: Step 1: After receiving the global model from the server, the client maintains a model pool locally, performs multiple model training sessions, and adds the trained models to the model pool. Step 2: The client aggregates models in the model pool using the index-aware algorithm and uploads the aggregated models to the roadside unit. Step 3: Within a time slice, the roadside unit aggregates all client-uploaded models received in that time slice using the outdated sensing algorithm based on the received client-uploaded models, and sends the aggregated model to the server. Step 4: After receiving the aggregated models uploaded by the roadside units, the server performs a weighted average of these aggregated models according to the amount of data in the roadside units to obtain the final global model.

2. The stale perception single-pass federated learning method for solving heterogeneity according to claim 1, characterized in that, In step 1, the client locally maintains a model pool consisting of three parts: model initialization, loss function design, and model training.

3. The stale perception single-pass federated learning method for solving heterogeneity according to claim 2, characterized in that, The model initialization is on the client side. Received the global model from the aggregation server Then, based on the model pool The weighted average of existing models is used to obtain the initial model, and the calculation formula is as follows: ， in, This represents the index of an existing model in the model pool. This represents the total number of existing models in the model pool.

4. The stale perception single-pass federated learning method for solving heterogeneity according to claim 2, characterized in that, The loss function is designed to explore model diversity and prevent local models from deviating too much from the global model. To enhance local model diversity during training and prevent the currently trained model from being too similar to existing models in the model pool, a constraint term is introduced. The calculation formula is as follows: ， in, Indicates the L2 norm distance between model parameters; Meanwhile, to prevent the current trained model from deviating too much from the global model, regularization constraints are introduced. To constrain the current model and the initial global model The distance between them is calculated using the following formula: ； This constraint ensures that the current model does not deviate excessively from the initial global model during local training, which helps to mitigate client drift caused by non-independent and identically distributed data and adapt to system heterogeneity; Ultimately, the client's overall local training loss function The calculation formula is as follows: ， in, It is the original loss function. It is a client Local dataset, and These are the hyperparameters that balance the strength of diversity constraints and global constraints, respectively.

5. The stale perception single-pass federated learning method for solving heterogeneity according to claim 2, characterized in that, The model training is on the client side. After the local model is initialized, each model The same parameters will be used. Each training cycle includes hyperparameters such as learning rate and batch size. After training, the model will be added to the client. Local model pool middle.

6. The stale perception single-pass federated learning method for solving heterogeneity according to claim 1, characterized in that, In step 2, the client aggregates models from the model pool using an index-aware algorithm and uploads the aggregated models to the roadside unit, specifically as follows: Client Model pool After all models in the pool have been trained, the index-aware algorithm is used to aggregate all models in the model pool. The calculation formula is as follows: ， The resulting aggregated local model Federated learning rounds participated in by the client Send to its corresponding roadside unit This averaging strategy is based on the assumption that model accuracy and overall performance will improve as the number of local training rounds increases.

7. The stale perception single-pass federated learning method for solving heterogeneity according to claim 1, characterized in that, In step 3, within a time slice, the roadside unit aggregates all client-uploaded models received in that time slice using an outdated sensing algorithm based on the received client-uploaded models, and sends the aggregated models to the server. Specifically: The roadside unit defines a fixed aggregation time window. Within this window, the roadside unit performs weighted aggregation on all received local aggregation models uploaded by clients, using the following calculation formula: ， in Indicates the roadside unit within the time window The total number of local models received internally. This represents the total number of global aggregations performed in federated learning. It is important to note that this method does not require roadside units to wait for all clients to upload their models; aggregation is performed at the end of the window, which effectively reduces the idle waiting time of high-computational-performance clients and improves the overall system training efficiency. Finally, the roadside unit uploads the aggregation model. This allows the server to perform subsequent global model updates.

8. The stale perception single-pass federated learning method for solving heterogeneity according to claim 1, characterized in that, In step 4, after receiving the aggregated models uploaded by the roadside units, the server performs a weighted average of these aggregated models according to the data volume of the roadside units to obtain the final global model, specifically as follows: The server receives the aggregated model from the roadside unit. Then, a global aggregation operation is performed, and the calculation formula is as follows: ， in This indicates the total number of roadside units involved in the current aggregation. Indicates roadside unit The sum of data from all clients in the process. .

Images