Vehicle-road cooperative heterogeneous scene data optimization method based on sparse perturbation federated learning

Abstract

The application discloses a vehicle-road cooperation heterogeneous scene data optimization method based on sparse disturbance federated learning, which comprises the following steps: a central server initializes a federated learning setting; the central server randomly selects a group of edge nodes as participants of this round of training and sends the current global model to the edge nodes; the edge nodes participating in this round of training use their own training data to train the model, during which sparse disturbance is performed on the model parameters to realize lightweight sharpness-aware minimization, and the local model of the next iteration is updated through parameter correction; the local model parameters trained by each edge node are uploaded to the central server; the central server aggregates the received model parameters, generates a new global model and distributes it, until the training of all rounds is completed. The application can improve the flatness of the global model in the vehicle-road cooperation data heterogeneous scene, enhance the generalization ability and robustness of the model, and effectively control the increase in the calculation cost caused by the adoption of sharpness-aware minimization.

Description

Technical Field

[0001] This invention relates to the field of vehicle-road cooperative data processing technology, and in particular to a method for optimizing heterogeneous vehicle-road cooperative scenario data based on sparse perturbation federated learning. Background Technology

[0002] With the rapid development of intelligent transportation and vehicle-road cooperative technologies, a large amount of data is distributed across edge intelligent nodes such as intelligent connected vehicles and intelligent roadside computing units. This data is highly dispersed and heterogeneous, including images from intelligent vehicle cameras and roadside monitoring video frames. Traditional centralized machine learning methods typically require uploading this data to a central server for unified processing. However, this approach faces significant challenges in terms of data privacy, transmission costs, and bandwidth consumption, especially when sensitive information involving intelligent connected vehicles is involved, where data privacy and security risks are particularly prominent. Against this backdrop, federated learning, as an emerging distributed machine learning paradigm, provides an effective solution for privacy protection and cross-node collaboration of vehicle-road cooperative data. Federated learning allows each participant to train its model locally on its data, avoiding centralized storage and processing of data, thereby reducing the risk of data leakage and communication bandwidth consumption. This distributed modeling method is highly compatible with the data distribution and privacy protection requirements in vehicle-road cooperative scenarios, enabling multiple edge nodes with computing capabilities to jointly model data without directly sharing data, thus promoting the development of intelligent transportation systems.

[0003] However, data in vehicle-road cooperative scenarios is typically highly heterogeneous, with significant differences in feature distribution, sample size, and annotation quality across different edge nodes. This data heterogeneity causes trained local models to converge to a single point with low loss, thus falling into local overfitting. When the central server aggregates these models, the resulting global model often performs poorly, potentially even worse than a model trained on a single edge node.

[0004] To address the aforementioned data heterogeneity issue, existing research has proposed Sharpness-Aware Minimization (SAM), which improves the generalization ability and robustness of models by exploring the flat regions of the loss function. For example, the paper [Qu Z, Li X, Duan R, et al. Generalized federated learning via sharpness-aware minimization[C] / / International Conference on Machine Learning.PMLR,2022:18250-18280.] introduces sharpness-aware minimization into federated learning to alleviate the performance degradation of models in heterogeneous data environments. The paper [Dai R, Yang X, Sun Y, et al. Fedgamma: Federated learning with global sharpness-aware minimization[J].IEEE Transactions on Neural Networks and Learning Systems,2023.] further mitigates the gap between local and global models through variance reduction techniques, achieving certain optimization effects. However, these methods still have two significant shortcomings: First, the data distribution of each edge node participating in federated training may differ, making it impossible to guarantee that the low-loss region of the model trained on each edge node is the same, which directly affects the flatness and stability of the global model; Second, after introducing sharpness-aware minimization, local training requires an additional gradient ascent and descent process, which significantly increases the computational burden and slows down the convergence speed of the algorithm.

[0005] Therefore, how to ensure the stability and generalization ability of the global model of federated learning, while effectively controlling the computational burden and convergence delay caused by minimizing sharpness perception, and achieving efficient, high-precision, and privacy-preserving processing of vehicle-road cooperative scenario data has become a key issue that urgently needs to be addressed. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a data optimization method for heterogeneous vehicle-road cooperative scenarios based on sparse perturbation federated learning. On one hand, it aligns the low-loss regions of each edge node participating in training; on the other hand, it minimizes the sharpness perception of sparsity through dynamic sparsity rate, thereby improving the flatness of the global model under heterogeneous data, enhancing the model's generalization ability and robustness, reducing the computational overhead caused by additional gradient ascent, optimizing computational efficiency, and thus improving its effective processing of heterogeneous vehicle-road cooperative scenario data.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] The first technical solution of this invention provides a method for optimizing vehicle-road cooperative heterogeneous scenario data based on sparse perturbation federated learning, including:

[0009] S1, The central server initializes the federated learning parameters and prepares to start federated learning training;

[0010] S2, the central server randomly selects a group of edge nodes as participants in this round of training and sends the current global model to these edge nodes;

[0011] S3: Edge nodes participating in this round of training use their own data to train the model using the current global model as the local model. The sharpness perception of sparsity is minimized by dynamic sparsity rate to improve computational efficiency.

[0012] The expression for the sparsity ratio is: α r =max(α) 0 ·e -λr ,α min )

[0013] Where, α r α is the sparsity of the r-th round of communication, used to generate the mask; 0 α is the initial sparsity, λ is the decay rate, and α is the initial sparsity. min It is the minimum sparsity rate, and the sparsity rate α is dynamically adjusted during training. r Optimize model performance; based on the current sparsity α r Generate mask:

[0014]

[0015] Where, m l It is the l-th element of the binary mask m; n l This represents the local model parameters of edge node i after the k-th iteration in the r-th round of communication. A random number between 0 and 1 is generated for each parameter position l;

[0016] pass Calculate the model parameters after adding perturbations.

[0017] in, Let be the gradient of edge node i after the k-th iteration in the r-th round of communication; ρ is the perturbation rate;

[0018] Then, based on the model parameters after adding perturbations... Further calculations were performed to determine the gradient after adding sparse perturbations. gradient with added sparse perturbation To perform the local model update for the next iteration;

[0019] Each edge node performs K local training iterations, where the number of local iterations is k = 0, ..., K-1.

[0020] S4. Upload the local model parameters trained at each edge node in step S3 to the central server.

[0021] S5: The central server aggregates the received model parameters, generates a new global model, and starts the next round of training until all training rounds are completed.

[0022] Furthermore, in step S3, the update formula for the local model update in the next iteration is:

[0023]

[0024] Where η is the local learning rate and β is the momentum coefficient. c is the local gradient control factor for edge node i in the r-th round. r It is the global gradient control factor in the r-th round, g r Let be the global gradient change estimate for the r-th round; This represents the local model parameters of edge node i after the (k+1)th iteration in the r-th round of communication;

[0025] Introducing a global gradient control factor c r and the local gradient control factor of edge node i To coordinate global and local objectives; when updating the model at edge nodes, an estimate of the change in global gradient from the previous training round, g, is introduced. r .

[0026] The second technical solution, according to this invention, provides a method for optimizing vehicle-road cooperative heterogeneous scenario data based on sparse perturbation federated learning, including:

[0027] S1, The central server initializes the federated learning parameters and prepares to start federated learning training;

[0028] S2, the central server randomly selects a group of edge nodes as participants in this round of training and sends the current global model to these edge nodes;

[0029] S3, the edge nodes participating in this training round use their own data to train the model using the current global model as the local model. The sparsity of the model parameters is minimized through sparsity perceptualization. The update parameters of the local model are corrected using the global gradient control factor, the local gradient control factor, and the global gradient change estimate. Therefore, the update formula for the local model is:

[0030]

[0031] Where η is the local learning rate and β is the momentum coefficient. c is the local gradient control factor for edge node i in the r-th round. r It is the global gradient control factor in the r-th round, g r Let be the global gradient change estimate for the r-th round; This represents the local model parameters of edge node i after the (k+1)th iteration in the r-th round of communication;

[0032] Introducing a global gradient control factor c r and the local gradient control factor of edge node i To coordinate global and local objectives; when updating the model at edge nodes, an estimate of the change in global gradient from the previous training round, g, is introduced. r It aggregates the gradient changes of each edge node participating in training, helps each edge node to consider the overall optimization goal when updating in order to achieve global penetration, and promotes the alignment of low-loss regions between different edge nodes.

[0033] S4. Upload the local model parameters trained at each edge node in step S3 to the central server.

[0034] S5: The central server aggregates the received model parameters, generates a new global model, and starts the next round of training until all training rounds are completed.

[0035] Furthermore, the local gradient control factor for edge node i in the next round The update formula is:

[0036]

[0037] Among them, for edge nodes S r represents the edge nodes participating in the r-th round of federated learning training; K represents the total number of iterations when the edge nodes are performing local training.

[0038] Furthermore, in step S5, the process of the central server aggregating the received model parameters includes:

[0039] Update and generate a new estimate of the global gradient change g. r+1 :

[0040]

[0041] Among them, S r This represents the edge node involved in the r-th round of federated learning training; w represents the local model parameters of edge node i after the Kth iteration in the r-th round of communication; r This is the global model in the r-th round of communication;

[0042] Based on the new round of global gradient change estimate g r+1 Generate a new global model w r+1 :

[0043] w r+1 =w r -γg r+1

[0044] Where γ is the global learning rate;

[0045] Finally, the central server generates a new round of global gradient control factor c. r+1 :

[0046]

[0047] in Let be the local gradient control factor for edge node i in round r+1.

[0048] Furthermore, the initialization of federated learning parameters in step S1 includes: initializing the global model w 0 Global gradient change estimator g 0 Total number of communication rounds R, number of local iterations K for each edge node, and initial gradient control factor c. 0 and Local learning rate η, global learning rate γ, momentum coefficient β, and perturbation rate ρ; N represents the number of edge nodes.

[0049] Furthermore, the objectives of federal learning training are:

[0050]

[0051] Among them, F i (·) represents the local empirical loss of edge node i. Let n represent the loss function. i ξ represents the number of training samples at edge node i. i This represents the n elements possessed by edge node i. i There are 1 training sample; j represents the training sample index number of edge node i, and w represents the parameters of the global model.

[0052] Furthermore, in step S2, a set of edge nodes is randomly selected: in the r-th round of communication, the central server samples a subset S of edge nodes. r And the global model w trained in this round r Distribute it to them.

[0053] Further, step S4 includes uploading the local model parameters trained by each edge node in the r-th round of federated learning training in step S3 to the central server.

[0054] Furthermore, this invention also protects a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, performs the following:

[0055] It features a central server and multiple computing-capable clients, each communicating with the central server via a network connection. Based on federated learning, the central server initializes a global model and loads local models for each client. During local model training, sharpness-aware minimization of model parameters is achieved through dynamic sparsity ratio.

[0056] The expression for the sparsity ratio is: α r =max(α) 0 ·e -λr ,α min )

[0057] Where, α r α is the sparsity of the r-th round of communication, used to generate the mask; 0 α is the initial sparsity, λ is the decay rate, and α is the initial sparsity. min It is the minimum sparsity rate, and the sparsity rate α is dynamically adjusted during training. r Optimize model performance; based on the current sparsity α r Generate mask:

[0058]

[0059] Where, m l It is the l-th element of the binary mask m; n l This represents the local model parameters of client i after the k-th iteration in the r-th round of communication. A random number between 0 and 1 is generated for each parameter position l;

[0060] pass Calculate the model parameters after adding perturbations.

[0061] in, Let ρ be the gradient of client i after the k-th iteration in the r-th round of communication; ρ is the perturbation rate.

[0062] Then, based on the model parameters after adding perturbations... Further calculations were performed to determine the gradient after adding sparse perturbations. gradient with added sparse perturbation To perform the local model update for the next iteration;

[0063] The update formula for the local model update in the next iteration is:

[0064]

[0065] Where η is the local learning rate and β is the momentum coefficient. It is the local gradient control factor for client i in the r-th round, c r It is the global gradient control factor in the r-th round, g r Let be the global gradient change estimate for the r-th round; This represents the local model parameters of client i after the (k+1)th iteration in the r-th round of communication.

[0066] More efficient models can be obtained on the aforementioned storage media, achieving faster training speeds while maintaining the same model accuracy, thus improving hardware training efficiency.

[0067] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0068] This invention presents a data optimization method for heterogeneous vehicle-road cooperative scenarios based on sparse perturbation federated learning. During the local model training phase at edge nodes, the sharpness perception of model parameters is minimized by dynamically sparsity rate. Sparsity reduces computational complexity and improves computational efficiency for optimizing heterogeneous vehicle-road cooperative scenario data.

[0069] This invention modifies model parameters through parameter control factors, aligning them as closely as possible with the low-loss regions of each edge node. This allows for a better solution space and a flatter low-loss region from a global perspective, making the global model more flat and effectively mitigating the performance degradation caused by data heterogeneity. This, in turn, improves the accuracy, generalization ability, and robustness of the global model. Specifically, this manifests as follows:

[0070] (1) Compared with six state-of-the-art methods (FedAvg, Scaffold, FedCM, FedSAM, FedGAMMA and FedSpeed), the accuracy of this invention is significantly improved.

[0071] (2) Compared with the six methods mentioned in (1) above, the present invention requires fewer federated learning rounds to achieve the same accuracy, which means that the present method converges faster and is more computationally efficient. Attached Figure Description

[0072] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0073] Figure 1This is an architecture diagram of a vehicle-road cooperative heterogeneous scenario data optimization method based on sparse perturbation federated learning provided by an embodiment of the present invention;

[0074] Figure 2 This is a flowchart of a method for optimizing heterogeneous scenario data in vehicle-road cooperative systems based on sparse perturbation federated learning, provided by an embodiment of the present invention.

[0075] Figure 3 This is the test accuracy curve of FedGF of the present invention under CIFAR-10, CIFAR-100 and IID.

[0076] Figure 4 This is the test accuracy curve of FedGF of the present invention under CIFAR-10, CIFAR-100 and DIR(0.6).

[0077] Figure 5 This is the test accuracy curve of FedGF of the present invention under CIFAR-10, CIFAR-100 and DIR(0.1). Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0079] This invention provides a method for optimizing heterogeneous scenario data in vehicle-road cooperative systems based on sparse perturbation federated learning. To illustrate this method, a schematic diagram of the architecture supporting its implementation is first provided. Figure 1 This is a schematic diagram of the architecture of a vehicle-road cooperative heterogeneous scene data optimization method based on sparse perturbation federated learning provided by an embodiment of the present invention, as shown below. Figure 1 As shown, the architecture includes a central server and multiple edge nodes with computing capabilities. Each edge node communicates with the central server via a network connection. These edge nodes can be, but are not limited to, intelligent connected vehicles, intelligent roadside monitoring equipment, and intelligent roadside computing units.

[0080] The central server acts as the coordination center, responsible for global model initialization, model updates, aggregation, and model version management. In this embodiment, each edge node is denoted as variable i, where i ∈ [N], and N represents the number of edge nodes. Edge nodes i ∈ [N] possess a property that follows D. i Distributed local dataset (X) i ,Y i ), where X iY represents the data characteristics at edge node i. i Indicates the corresponding label, D i This represents the data distribution. Due to the heterogeneity of the data, the data distribution differs across different edge nodes. Furthermore, edge node i also possesses n... i From D i Training samples ξ drawn from the distribution i Used for local training of the model. Typically, the goal of federated learning algorithms in vehicle-to-infrastructure (V2I) scenarios is to learn a global model w that can adapt to the dataset of all edge nodes by minimizing the empirical loss function at each edge node. The training objective is:

[0081]

[0082] Among them, F i (·) represents the local empirical loss of edge node i. Let n represent the loss function. i represents the number of training samples at edge node i, j represents the index number of the training sample at edge node i, and w represents the parameters of the global model.

[0083] As mentioned above, existing methods incorporate sharpness-aware minimization into federated learning. This method adds a perturbation during the local training phase, performing an additional gradient ascent and descent operation. Specifically, it optimizes the objective function by obtaining a perturbation δ within the perturbation radius ρ, minimizing the loss at w+δ to find a low-loss region. The training objective can be expressed as:

[0084]

[0085] Among them, the model parameters after perturbation The following formula can be used to derive it:

[0086] For each edge node i, first check F i (w+δ i Perform a first-order Taylor expansion at w to transform it into a linearly constrained optimization problem:

[0087]

[0088] Since ρ is very small, the higher-order term O(ρ) is ignored. 2 At the same time, F i (w) can be considered a constant term because it depends only on w and not on δ. i Therefore, it has no impact on the result during the optimization process, so we can obtain:

[0089]

[0090] in, It's about δ i A linear function, It is the gradient of the model parameter w, δ i Satisfy ||δ i The constraint condition is ||≤ρ. This problem can be transformed into finding a point δ within the constraint set (a sphere centered at the origin). i , so that the objective function The maximum value is found at the boundary of the constraint set, which is a sphere with norm ρ. Since the objective function is linear, its maximum value always occurs on the boundary of the constraint set. Therefore, to satisfy ||δ... i ||=ρ, representing the solution δ i It must be on a sphere with radius ρ.

[0091] maximize The essence is to let and The direction should be as consistent as possible with the direction of the optimal solution. Therefore, the direction of the optimal solution should be consistent with the direction of the optimal solution. Same, that is:

[0092]

[0093] Where α represents a proportionality coefficient. Let ||δ i ||=ρ and Substituting the constraints, we get:

[0094]

[0095] Therefore, the proportionality constant can be solved. Finally, α will be substituted back. From this, we can obtain:

[0096]

[0097] This step calculates the perturbation δ that causes the greatest loss in the neighborhood of w. i And obtain the updated model parameters after perturbation:

[0098]

[0099] However, due to the differences in data distribution among edge nodes, it cannot be guaranteed that the low-loss regions of each edge node are the same, which directly affects the flatness and stability of the global model. Introducing sharpness-aware minimization requires adding an extra gradient ascent during the local training phase, increasing the computational burden.

[0100] Therefore, based on the above architecture and optimization issues, Figure 2This is a flowchart of a data optimization method for heterogeneous vehicle-road cooperative scenarios based on sparse perturbation federated learning, provided by an embodiment of the present invention. Under heterogeneous data conditions, this method can not only align low-loss regions and improve the flatness of the global model, but also effectively control the increase in computational cost caused by sharpness-aware minimization. The method includes the following steps:

[0101] S1, the central server initializes the federated learning settings and configures the necessary model training parameters, including the initial global model w. 0 Global gradient change estimator g 0 Total number of communication rounds R, number of local iterations K for each edge node, gradient control factor c 0 and The local learning rate η, global learning rate γ, momentum coefficient β, and perturbation rate ρ represent the R rounds of federated training coordinated by the central server across all edge nodes, with each edge node performing K rounds of local training. This step primarily prepares for the federated learning training.

[0102] S2, the central server randomly selects a set of edge nodes as participants in this round of training. Specifically, in the r-th round of communication, the central server samples a subset S of edge nodes. r And the global model w trained in this round r Distribute it to them;

[0103] S3, the edge nodes i∈S participating in this round of training. r Using its own training samples ξ i Model training involves sparse perturbation of model parameters to achieve lightweight sharpness-aware minimization, and parameter correction to promote low-loss region alignment between different edge nodes. This includes the following steps:

[0104] (1) For each edge node i∈S r The following training is performed in parallel;

[0105] (2) Each edge node i receives the global model w sent by the central server. r Then, initialize the local model: This means that edge nodes use the global model distributed by the central server as the initial model for local training. This represents the local model parameters of edge node i after the k-th iteration in the r-th round of communication. Let be the local model parameters of edge node i after the 0th iteration in the r-th round of communication.

[0106] (3) Each edge node calculates the gradient after adding sparse perturbation. For local iteration numbers k = 0, ... K-1, edge node i performs the following operation:

[0107] pass Perform local training and calculate gradients. Then through Calculate the model parameters after adding perturbations.

[0108] Where j represents the index of the training sample of edge node i in the k-th iteration of the r-th round of communication, and m is a random binary mask used to achieve sparsity-aware minimization, for example, randomly selecting some elements as 1 and the rest as 0. This method uses a dynamic sparsity rate to generate the random mask:

[0109] α r =max(α) 0 ·e -λr ,α min )

[0110] Where, α r α is the sparsity of the r-th round of communication. 0 α is the initial sparsity, λ is the decay rate, and α is the initial sparsity. min It is the minimum sparsity rate, and the sparsity rate α is dynamically adjusted during training. r Model performance can be optimized. A higher sparsity ratio can be used initially to accelerate convergence, and then the sparsity ratio can be gradually reduced to refine the model. Then, for... For each parameter position l, generate a random number n. l :

[0111] n l ~U(0,1)

[0112] Then, based on the current sparsity α r Generate mask m:

[0113]

[0114] Where, m l It is the l-th element of the binary mask m, and then... Calculate the gradient after adding sparse perturbations

[0115] (4) Update the model at each edge node, correct the parameters, and align with the low-loss region:

[0116]

[0117] Where η is the local learning rate and β is the momentum coefficient. c is the local gradient control factor for edge node r in the r-th round. r It is the global gradient control factor in the r-th round, g rLet be the global gradient change estimate for the r-th round; Let represent the local model parameters of edge node i after the (k+1)th iteration in the r-th round of communication. A global gradient control factor c is introduced. r and local gradient control factor To coordinate global and local goals. c represents the iterative deviation of the local gradient of edge node i in the global optimization problem. r This represents the average bias of all edge nodes participating in the training in the global optimization problem. It is a centralized quantity that reflects the bias correction of the global optimization objective. In this method, through... To adjust the local gradient, that is, to subtract the local gradient control factor specific to each edge node. The iteration bias was subtracted from the local gradient, and the global gradient control factor c was added. r The global average bias is incorporated into the update. Furthermore, when updating the model at edge nodes, an estimate of the global gradient change g from the previous training iteration is introduced. r It aggregates the gradient changes of each edge node participating in training, and helps each edge node to consider the overall optimization goal during updates in a manner similar to momentum terms, so as to achieve global penetration.

[0118] Subsequently, edge node i updates the local gradient control factor for the next round:

[0119]

[0120] Among them, for edge nodes S r Let K represent the edge nodes participating in the r-th round of federated learning training, K represent the total number of iterations when the edge nodes are trained locally, and k represent the number of the current iteration.

[0121] S4 uploads the local model parameters trained at each edge node in S3 to the central server.

[0122] S5. After receiving the model parameters uploaded by each edge node, the central server updates and generates a new round of global gradient change estimate g. r+1 :

[0123]

[0124] And this generates a new global model w. r+1 :

[0125] w r+1 =w r -γg r+1

[0126] Where γ is the global learning rate. Finally, the central server generates a new round of global gradient control factors c. r+1 :

[0127]

[0128] This completes the current round of federated learning training. Next, the central server will sample a new set of edge nodes and generate a new global model w. r+1 The training is then distributed to these edge nodes until all training rounds are completed.

[0129] In this embodiment of the invention, the low-loss regions on the loss function found during the training of each edge node are aligned as much as possible through control factors. From a global perspective, the low-loss regions and better solution spaces are explored, which further smooths the model update process. At the same time, the sparse perturbation is performed through dynamic sparsity rate, which alleviates the computational burden brought about by the introduction of sharpness-aware minimization.

[0130] To further illustrate the effectiveness of the present invention, experiments were conducted to verify the vehicle-road cooperative heterogeneous scene data optimization method based on sparse perturbation federated learning.

[0131] 1. Experimental Environment

[0132] The experimental platform built for this invention was mainly conducted on the Ubuntu operating system (version 20.04). The equipment configuration information is as follows: AMD Ryzen 9 5950X 16-Core Processor CPU, 64GB memory, NVIDIA RTX 3090Ti GPU with 24GB of video memory, programming language is Python 3.8, and the experimental framework is PyTorch.

[0133] 2. Experimental verification

[0134] Experiments were conducted on two classic datasets commonly used in federated learning, CIFAR-10 and CIFAR-100, to validate the results.

[0135] (1) Dataset Description

[0136] CIFAR-10 is a small dataset containing 10 classes of natural images, with images evenly distributed across each class. This dataset consists of 60,000 32×32 pixel color images, with 50,000 used for training and 10,000 for testing. The classes include common objects and animals such as airplanes, cars, birds, and cats, exhibiting a uniform and balanced distribution. The CIFAR-100 dataset is an extended version of CIFAR-10, also consisting of 60,000 32×32 pixel color images, with 50,000 used for training and 10,000 for testing. It contains 100 classes, with a relatively smaller number of images per class. Compared to CIFAR-10, CIFAR-100 has more refined categories, making the classification task more challenging, especially in heterogeneous data scenarios, placing higher demands on the robustness of federated learning algorithms.

[0137] (2) Model

[0138] The model used in the experiment was the classic ResNet-18 model.

[0139] (3) Construction of heterogeneous datasets

[0140] To simulate heterogeneous data scenarios, this experiment introduces two different heterogeneities into the CIFAR-10 and CIFAR-100 datasets using Dirichlet distributions (with parameters of 0.6 and 0.1): DIR(0.6) and DIR(0.1). The Dirichlet distribution is affected by the parameter α, where a smaller α indicates a higher degree of data heterogeneity and a more uneven data distribution, while a larger α indicates a lower degree of data heterogeneity and a more uniform data distribution.

[0141] (4) Comparison of algorithms and parameter settings

[0142] The method of this invention (denoted as FedGF) was compared with six state-of-the-art federated learning algorithms: FedAvg, Scaffold, FedCM, FedSAM, FedGAMMA, and FedSpeed. These algorithms share the same focus as FedGF in this invention: optimizing the local training phase of the model. In this experiment, the number of communication rounds was set to 1000, the total number of clients was set to 100, the client sampling ratio was 0.1, the number of local iterations was set to 5, and the local training data batch size was set to 50. Furthermore, the local learning rate and global learning rate were set to 0.1 and 1.0, respectively. The local learning rate decayed by 0.998 after each communication round, the momentum coefficient was set to 0.1, and the perturbation rate was set to 0.01.

[0143] (5) Experimental Results and Analysis

[0144] Comparative experiments were conducted on two datasets, CIFAR-10 and CIFAR-100, and under three different levels of data heterogeneity (IID, DIR(0.6), DIR(0.1)).

[0145] Table 1 shows the accuracy performance of the FedGF algorithm of this invention compared to the six algorithms mentioned above after 1000 rounds of communication. In the table, "IID" indicates a uniform distribution of training samples on each client; "DIR(0.6)" indicates a non-independent and identically distributed case introducing general data heterogeneity using a Dirichlet distribution (parameter 0.6); and "DIR(0.1)" indicates a non-independent and identically distributed case introducing high data heterogeneity using a Dirichlet distribution (parameter 0.1). The results in the table show that the FedGF algorithm of this invention achieved the highest accuracy on both the CIFAR-10 and CIFAR-100 datasets and under three different data distributions. Specifically, on the CIFAR-10 dataset, the FedGF algorithm of this invention outperformed the second-ranked algorithm by 1.84%, 2.51%, and 2.84% in accuracy, respectively, under the three different data distributions; and on the CIFAR-100 dataset, it outperformed the second-ranked algorithm by 0.89%, 0.94%, and 2.23% in accuracy, respectively. This demonstrates that the present invention has significant advantages in addressing data heterogeneity issues. Furthermore, it can be seen that on both datasets, as data heterogeneity increases, the accuracy of the FedGF invention, which surpasses the second-ranked algorithm, improves. This indicates that when data distribution is more biased towards certain categories or exhibits greater imbalance, the FedGF invention can more effectively reduce the negative impact of data imbalance on model performance and improve overall accuracy. This advantage reflects the adaptability of FedGF to data heterogeneity.

[0146] Table 1. Accuracy (%) of the present invention in tests on two datasets and under different heterogeneous data settings.

[0147]

[0148] Figure 3 , Figure 4 , Figure 5The results show the trends of global test accuracy of the proposed FedGF algorithm and six other algorithms on the CIFAR-10 and CIFAR-100 datasets under IID, DIR(0.6), and DIR(0.1) distributions as the number of communication rounds increases. The results demonstrate that the proposed FedGF algorithm consistently performs well in most communication rounds, ultimately achieving the highest test accuracy, significantly outperforming the other comparison algorithms. Overall, the curve of the proposed FedGF algorithm is steeper in the six graphs, indicating faster convergence in the early stages and a rapid improvement in model performance. In later communication rounds (e.g., after 500 rounds), the test accuracy of FedGF is more stable and continuously improves, demonstrating excellent stability and convergence. The other six comparison algorithms either have fast convergence in the early stages but fail to achieve a high and stable accuracy in the later stages, or their overall convergence speed and final accuracy are both weaker than the proposed FedGF algorithm.

[0149] Table 2 shows the number of communication rounds required for FedGF (based on this invention) and six other algorithms to achieve the predetermined accuracy. The predetermined accuracy for CIFAR-10 is 80%, and for CIFAR-100 it is 45%. N / A in the table indicates that the algorithm does not reach the corresponding predetermined accuracy. The data shows that FedGF exhibits strong performance. On the CIFAR-10 dataset, FedGF requires 82 communication rounds in the IID (80% accuracy) scenario, demonstrating a significant efficiency advantage, far superior to other algorithms such as FedAvg and Scaffold, which require 330 and 293 rounds respectively. In a moderately heterogeneous data environment with DIR (0.6), FedGF requires 112 communication rounds, still maintaining its lead and showing strong adaptability. However, in a highly heterogeneous data environment with DIR (0.1), the number of communication rounds for FedGF increases to 540, which, although relatively high, is still more efficient than FedAvg and Scaffold. For the CIFAR-100 dataset, the FedGF algorithm of this invention requires 92 rounds of communication in the IID scenario, which is significantly better than FedAvg and Scaffold. It also requires 134 rounds in the DIR(0.6) scenario, still performing well. However, in the highly heterogeneous environment of DIR(0.1), the number of rounds increases to 213. Despite this, the FedGF algorithm of this invention is still more efficient than other algorithms. Overall, the FedGF algorithm of this invention demonstrates a significant advantage in convergence speed.

[0150] Table 2. Number of communication rounds required to achieve the predetermined accuracy rate.

[0151]

[0152] Finally, it should be noted that the above description is only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention. Any aspects not covered in this invention are applicable to the prior art.

Claims

1. A vehicle-road cooperation heterogeneous scene data optimization method based on sparse perturbation federated learning, characterized in that, Comprise: S1, the central server initializes the federated learning parameter, prepares to start the federated learning training; S2, the central server randomly selects a group of edge nodes as the participant of this round of training, and sends the current global model to these edge nodes; S3, the edge node participating in this round of training uses its own data to use the current global model as the local model for model training, and the sharpness-aware minimization of the sparsity of the model parameters is improved; The sparsity is expressed as: a r = max(a 0 ·e -λr , a min ) wherein, α r is the sparsity of the rth communication round, used to generate the mask; α 0 is the initial sparsity, λ is the decay rate, α min is the minimum sparsity, and during the training process, the sparsity α r is dynamically adjusted to optimize the model performance; and according to the current sparsity α r the mask is generated: wherein m l is the lth element of the binary mask m; n l denotes the local model parameters of edge node i after the kth iteration in the rth round of communication a random number between 0 and 1 generated by each parameter position l of By Computing model parameters after adding perturbations wherein, is the gradient of the edge node i after the kth iteration in the rth round of communication; p is the perturbation rate; Then the local model update for the next iteration is performed using the gradient with added sparse perturbation Further, the gradient with added sparse perturbation is computed The local model update for the next iteration is performed using the gradient with added sparse perturbation ​ Each edge node performs K times of local training, for local iteration number k=0,...,K-1; S4, the local model parameters trained in step S3 are uploaded to the central server; S5, the central server aggregates the received model parameters to generate a new global model, and starts the next round of training until all training rounds are completed.

2. The method of claim 1, wherein, In step S3, the update formula of the local model update of the next iteration is: where η is the local learning rate, and β is the momentum coefficient, is the local gradient control factor of the edge node i in the rth round, c r is the global gradient control factor in the rth round, g r is the global gradient change estimation in the rth round; denotes the local model parameters of the edge node i after the k+1th iteration in the rth round of communication.

3. A vehicle-road cooperation heterogeneous scene data optimization method based on sparse perturbation federated learning, characterized in that, Comprise: S1, the central server initializes the federated learning parameter, prepares to start the federated learning training; S2, the central server randomly selects a group of edge nodes as the participant of this round of training, and sends the current global model to these edge nodes; S3, the edge node participating in this round of training uses its own data to use the current global model as the local model for model training, and the sharpness-aware minimization of the sparsity of the model parameters is improved; where η is the local learning rate, and β is the momentum coefficient, is the local gradient control factor of the edge node i in the rth round, c r is the global gradient control factor in the rth round, g r is the global gradient change estimation in the rth round; denotes the local model parameters of the edge node i after the k+1th iteration in the rth round of communication; Introducing a global gradient control factor c r and a local gradient control factor of the edge node i to coordinate global and local objectives; introducing the change estimation g of the global gradient in the last round of training when updating the model at the edge node r , which aggregates the gradient changes of each edge node participating in training, helps each edge node consider the overall optimization goal when updating to achieve global penetration, and promotes low-loss regional alignment between different edge nodes; S4, the local model parameters trained in step S3 are uploaded to the central server; S5, the central server aggregates the received model parameters to generate a new global model, and starts the next round of training until all training rounds are completed.

4. The method according to claim 2 or 3, characterized in that, Edge node i next round local gradient control factor The update formula is: wherein, for edge nodes S r denotes the edge nodes participating in the rth round of federated learning training; K denotes the total number of iterations when the edge nodes perform local training.

5. The method according to claim 2 or 3, characterized in that, In step S5, the process of the central server aggregating the received model parameters comprises: updating and generating a new round of global gradient change estimators g r+1 : wherein S r denotes the edge node participating in the rth round of federated learning training; denotes the local model parameters of the edge node i after the Kth iteration in the rth round of communication; w r is the global model in the rth round of communication; According to a new round of global gradient change estimator g r+1 Generate a new global model w r+1 : w r+1 = w r - γg r+1 Wherein, γ is the global learning rate; Finally, the central server generates a new round of global gradient control factor c r+1 : wherein is the local gradient control factor for edge node i in the r+1th round.

6. The method according to claim 1 or 3, characterized in that, The initialization of the federated learning parameters in step S1 includes: an initial global model w 0 , a global gradient change estimator g 0 , a total number of communication rounds R, a local iteration number K of each edge node, an initial gradient control factor c 0 , and a local learning rate η, a global learning rate γ, a momentum coefficient β, and a perturbation rate ρ; N represents the number of edge nodes.

7. The method according to claim 1 or 3, characterized in that, The federated learning training target is: where F i (·) denotes the local empirical loss of edge node i, denotes the loss function, n i denotes the number of training samples on edge node i, ξ i denotes the n i training samples owned by edge node i; j denotes the training sample index number of edge node i, and w denotes the parameters of the global model.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, the following contents are realized: There is a central server and multiple clients with computing power, each client is connected with the central server for communication through the network;Based on federated learning, the central server initializes the global model, and loads the local model for the client;The sharpness-aware minimization of the sparsity of the model parameters is improved when the local model training is performed through the dynamic sparsity, The expression of the sparsity is: a r = max(a 0 · e -λ r, a min ) wherein α r is the sparsity of the rth communication round, used to generate the mask; α 0 is the initial sparsity, λ is the decay rate, α min is the minimum sparsity, and during the training process, the sparsity α r is dynamically adjusted to optimize the model performance; and according to the current sparsity α r the mask is generated: where m l is the lth element of the binary mask m; n l denotes the local model parameters of client i after the kth iteration in the rth round of communication a random number generated by each parameter position l of the model By Computing model parameters after adding perturbations wherein, is the gradient of client i after the kth iteration in the rth round of communication; p is the perturbation rate; Then the local model update for the next iteration is performed using the gradient with added sparse perturbation Further, the gradient with added sparse perturbation is computed The local model update for the next iteration is performed using the gradient with added sparse perturbation Further, the gradient with added sparse perturbation is computed The update formula of the local model update of the next iteration is: where η is the local learning rate, and β is the momentum coefficient, is the local gradient control factor of the client i in the rth round, c r is the global gradient control factor in the rth round, g r is the global gradient change estimation in the rth round; denotes the local model parameters of the client i after the k+1th iteration in the rth round of communication.

Images