Vehicle infrastructure collaborative heterogeneous scene data optimization method based on sparse disturbance federated learning

Abstract

The invention discloses a vehicle infrastructure collaborative heterogeneous scene data optimization method based on sparse disturbance federated learning. The method comprises the following steps that a central server initializes federated learning setting; the central server randomly selects a group of edge nodes as training participants of the round, and sends a current global model to the edge nodes; performing model training on the edge nodes participating in the current round of training by using self training data, performing sparse disturbance on model parameters during the period to realize lightweight sharpness perception minimization, and updating a local model of the next iteration through parameter correction; uploading the trained local model parameters of each edge node to a central server; and the central server aggregates the received model parameters, generates a new global model and issues the new global model until all rounds of training are completed. According to the method, the flatness of the global model can be improved in a vehicle-road collaborative data heterogeneous scene, the generalization ability and robustness of the model are enhanced, and the calculation cost increase caused by adopting acutance perception minimization is effectively controlled.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle-road collaboration data processing, and in particular to a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning. Background Art

[0002] With the rapid development of intelligent transportation and vehicle-road collaboration technologies, a large amount of data is distributed in edge intelligent nodes such as intelligent connected vehicles and intelligent roadside computing units. These data are highly decentralized and heterogeneous, and include various forms such as intelligent vehicle camera images and roadside monitoring video frames. Traditional centralized machine learning methods usually require that these data be uploaded to a central server for unified processing. However, this method faces huge challenges in terms of data privacy, transmission costs, and bandwidth consumption. Especially when it comes to sensitive information of intelligent connected vehicles, data privacy and security risks are particularly prominent. In this context, federated learning, as an emerging distributed machine learning paradigm, provides an effective solution for privacy protection and cross-node collaboration of vehicle-road collaboration data. Federated learning allows each participant to train models on their data locally, avoiding centralized storage and processing of data, thereby reducing the risk of data leakage and the occupation of communication bandwidth. This distributed modeling method is highly consistent with the data distribution and privacy protection requirements in vehicle-road collaboration scenarios. It can enable multiple edge nodes with computing capabilities to jointly model without directly sharing data, promoting the development of intelligent transportation systems.

[0003] However, data in vehicle-road collaboration scenarios is usually highly heterogeneous, and data from different edge nodes vary significantly in terms of feature distribution, sample size, and annotation quality. This data heterogeneity causes the trained local models to converge to a single point with low loss, thus falling into local overfitting. When the central server aggregates these models, the performance of the generated global model is often poor, and may even be worse than the model trained by a single edge node alone.

[0004] In response to the above-mentioned data heterogeneity problem, some studies have proposed sharpness-aware minimization (SAM), which improves the generalization ability and robustness of the model by exploring the flat area of ​​the loss function. For example, the literature [Qu Z, Li X, Duan R, et al. Generalized federated learning via sharpness aware minimization [C] / / International Conference on Machine Learning. PMLR, 2022: 18250-18280.] introduces sharpness-aware minimization into federated learning to alleviate the performance degradation of the model in a data heterogeneous environment. The literature [Dai R, Yang X, Sun Y, et al. Fedgamma: Federated learning with global sharpness-aware minimization [J]. IEEE Transactions on Neural Networks and Learning Systems, 2023.] further alleviates the gap between the local model and the global model through variance reduction technology, and achieves certain optimization effects. However, these methods still have two significant shortcomings: first, the data distribution of each edge node participating in federated training may be different, and it is impossible to ensure that the low-loss area of ​​the model trained by each edge node is the same, which directly affects the flatness and stability of the global model; second, after the introduction of sharpness-aware minimization, local training needs to introduce an additional gradient ascent and descent process, which significantly increases the computational burden and slows down the convergence of the algorithm.

[0005] Therefore, how to ensure the stability and generalization ability of the global model of federated learning, while effectively controlling the computational burden and convergence delay brought about by minimizing sharpness perception, and achieve efficient and high-precision privacy processing of vehicle-road collaborative scenario data has become a key issue that needs to be solved urgently. Summary of the invention

[0006] In response to the above problems, the present invention provides a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning. On the one hand, it aligns the low-loss areas of each edge node participating in the training, and on the other hand, it minimizes the sharpness perception of sparseness through dynamic sparsity rate, so as to improve the flatness of the global model under data heterogeneity, enhance the generalization ability and robustness of the model, reduce the computational overhead brought by the additional gradient ascent, optimize the computational efficiency, and thereby improve its effective processing of vehicle-road collaboration heterogeneous scenario data.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0008] The first technical solution, the present invention provides a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning, comprising:

[0009] S1, the central server initializes the federated learning parameters and prepares to start federated learning training;

[0010] S2, the central server randomly selects a group of edge nodes as participants in this round of training and sends the current global model to these edge nodes;

[0011] S3, the edge nodes participating in this round of training use their own data to train the model with the current global model as the local model, and minimize the sharpness perception of sparsifying the model parameters through dynamic sparsity to improve computing efficiency;

[0012] The expression of the sparseness rate is: α r =max(α 0 ·e -λr ,α min )

[0013] Among them, α r is the sparsity rate of the rth communication round, used to generate the mask; α 0 is the initial sparsity rate, λ is the decay rate, α min is the minimum sparse rate. During the training process, the sparse rate α is dynamically adjusted. r Optimize model performance; according to the current sparsity rate α r Generate the mask:

[0014]

[0015] Among them, m l is the lth element of the binary mask m; n l represents the local model parameters of edge node i after the kth iteration in the rth round of communication A random number from 0 to 1 is generated for each parameter position l;

[0016] pass Calculate the model parameters after adding the perturbation

[0017] in, is the gradient of edge node i after the kth iteration in the rth round of communication; ρ is the perturbation rate;

[0018] Then according to the model parameters after adding the disturbance Further calculate the gradient after adding sparse perturbations With the gradient after adding sparse perturbation To update the local model for the next iteration;

[0019] Each edge node performs K local trainings, for local iterations k = 0, ... K-1;

[0020] S4, uploading the local model parameters trained by each edge node in step S3 to the central server;

[0021] S5, the central server aggregates the received model parameters, generates a new global model, and starts the next round of training until all training rounds are completed.

[0022] Furthermore, in step S3, the update formula for the local model update in the next iteration is:

[0023]

[0024] Among them, η is the local learning rate, β is the momentum coefficient, is the local gradient control factor of edge node i in round r, c r is the global gradient control factor of the rth round, g r is the global gradient change estimator of the rth round; represents the local model parameters of edge node i after the k+1th iteration in the rth round of communication;

[0025] Introducing the global gradient control factor c r and the local gradient control factor of edge node i To coordinate global and local goals; when updating the model at the edge node, the change estimate of the global gradient in the previous round of training is introduced r .

[0026] The second technical solution, the present invention provides a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning, comprising:

[0027] S1, the central server initializes the federated learning parameters and prepares to start federated learning training;

[0028] S2, the central server randomly selects a group of edge nodes as participants in this round of training and sends the current global model to these edge nodes;

[0029] S3, the edge nodes participating in this round of training use their own data to train the model with the current global model as the local model, minimize the sharpness perception of sparse model parameters through the sparsity rate, and use the global gradient control factor, local gradient control factor and global gradient change estimator to correct the update parameters of the local model. The update formula of the local model is:

[0030]

[0031] Among them, η is the local learning rate, β is the momentum coefficient, is the local gradient control factor of edge node i in round r, c r is the global gradient control factor of the rth round, g r is the global gradient change estimator of the rth round; represents the local model parameters of edge node i after the k+1th iteration in the rth round of communication;

[0032] Introducing the global gradient control factor c r and the local gradient control factor of edge node i To coordinate global and local goals; when updating the model at the edge node, the change estimate of the global gradient in the previous round of training is introduced r , aggregates the gradient changes of each edge node participating in the training, helps each edge node consider the overall optimization goal when updating to achieve global penetration, and promotes the alignment of low-loss areas between different edge nodes;

[0033] S4, uploading the local model parameters trained by each edge node in step S3 to the central server;

[0034] S5, the central server aggregates the received model parameters, generates a new global model, and starts the next round of training until all training rounds are completed.

[0035] Furthermore, the local gradient control factor of edge node i in the next round is The update formula is:

[0036]

[0037] Among them, for edge nodes S r represents the edge nodes participating in the rth round of federated learning training; K represents the total number of iterations of the edge nodes when performing local training.

[0038] Furthermore, in step S5, the process of the central server aggregating the received model parameters includes:

[0039] Update and generate a new round of global gradient change estimator g r+1 :

[0040]

[0041] Among them, S r represents the edge nodes participating in the rth round of federated learning training; represents the local model parameters of edge node i after the Kth iteration in the rth round of communication; w r is the global model in the rth round of communication;

[0042] According to the new round of global gradient change estimator g r+1 Generate a new global model w r+1 :

[0043] w r+1 =w r -γg r+1

[0044] Among them, γ is the global learning rate;

[0045] Finally, the central server generates a new round of global gradient control factor c r+1 :

[0046]

[0047] in is the local gradient control factor of edge node i in the r+1th round.

[0048] Furthermore, the initialization of federated learning parameters in step S1 includes: the initial global model w 0 , global gradient change estimator g 0 , the total number of communication rounds R, the number of local iterations of each edge node K, the initial gradient control factor c 0 and local learning rate η, global learning rate γ, momentum coefficient β and perturbation rate ρ; N represents the number of edge nodes.

[0049] Furthermore, the training objectives of federated learning are:

[0050]

[0051] Among them, F i (·) represents the local experience loss of edge node i, represents the loss function, n i represents the number of training samples on edge node i, ξ i represents the number of n owned by edge node i i training samples; j represents the training sample index number of edge node i, and w represents the parameters of the global model.

[0052] Furthermore, in step S2, a set of edge nodes is randomly selected: in the rth round of communication, the central server samples a subset of edge nodes S r , and the global model w of this round of training r Send it to them.

[0053] Furthermore, the step S4 includes: uploading the local model parameters trained by each edge node in the rth round of federated learning training in the step S3 to the central server.

[0054] In addition, the present invention also protects a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following:

[0055] There is a central server and multiple clients with computing power, each of which communicates with the central server through a network connection. Based on federated learning, the central server initializes the global model and loads the local model for the client. When training the local model, the model parameters are sparsely spaced by dynamic sparsity rate to minimize the sharpness perception.

[0056] The expression of the sparseness rate is: α r =max(α 0 ·e -λr ,α min )

[0057] Among them, α r is the sparsity rate of the rth communication round, used to generate the mask; α 0 is the initial sparsity rate, λ is the decay rate, α min is the minimum sparse rate. During the training process, the sparse rate α is dynamically adjusted. r Optimize model performance; according to the current sparsity rate α r Generate the mask:

[0058]

[0059] Among them, m l is the lth element of the binary mask m; n l represents the local model parameters of client i after the kth iteration in the rth round of communication A random number from 0 to 1 is generated for each parameter position l;

[0060] pass Calculate the model parameters after adding the perturbation

[0061] in, is the gradient of client i after the kth iteration in the rth round of communication; ρ is the perturbation rate;

[0062] Then according to the model parameters after adding the disturbance Further calculate the gradient after adding sparse perturbations With the gradient after adding sparse perturbation To update the local model for the next iteration;

[0063] The update formula for the local model update in the next iteration is:

[0064]

[0065] Among them, η is the local learning rate, β is the momentum coefficient, is the local gradient control factor of client i in round r, c r is the global gradient control factor of the rth round, g r is the global gradient change estimator of the rth round; represents the local model parameters of client i after the k+1th iteration in the rth round of communication.

[0066] A more efficient model can be obtained on the above storage medium, which has a faster speed while training the same model accuracy, thereby improving the hardware training efficiency.

[0067] Compared with the prior art, the beneficial effects of the present invention are:

[0068] The present invention is a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning. In the local model training stage of the edge node, the sharpness perception is minimized by sparsifying the model parameters through a dynamic sparsity rate. The computational complexity is reduced by sparsification, thereby improving the computational efficiency of vehicle-road collaboration heterogeneous scenario data optimization.

[0069] The present invention modifies the model parameters through parameter control factors, aligns the low-loss areas of each edge node as much as possible, and thus finds a better solution space and a flat low-loss area from a global perspective, making the global model flat, effectively alleviating the problem of model performance degradation caused by data heterogeneity, thereby improving the accuracy, generalization ability and robustness of the global model. Specifically, it is manifested as:

[0070] (1) Compared with the six most advanced methods (FedAvg, Scaffold, FedCM, FedSAM, FedGAMMA and FedSpeed), the accuracy of the present invention is greatly improved.

[0071] (2) Compared with the six methods mentioned in (1) above, the present invention requires fewer rounds of federated learning to achieve the same accuracy, which means that the present method converges faster and has higher computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0073] Figure 1It is an architecture diagram of a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning provided by an embodiment of the present invention;

[0074] Figure 2 It is a flowchart of a method for optimizing vehicle-road collaboration heterogeneous scenario data based on sparse perturbation federated learning provided by an embodiment of the present invention.

[0075] Figure 3 It is the test accuracy curve of FedGF of the present invention under CIFAR-10, CIFAR-100 and IID.

[0076] Figure 4 It is the test accuracy curve of FedGF of the present invention under CIFAR-10, CIFAR-100 and DIR (0.6).

[0077] Figure 5 It is the test accuracy curve of FedGF of the present invention under CIFAR-10 and CIFAR-100 and DIR(0.1). DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0079] An embodiment of the present invention provides a method for optimizing vehicle-road collaboration heterogeneous scenario data based on sparse perturbation federated learning. In order to illustrate the method, a schematic diagram of an architecture supporting the implementation of the method is first described. Figure 1 is a schematic diagram of the architecture of a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning provided by an embodiment of the present invention, such as Figure 1 As shown in FIG. 1 , the architecture includes a central server and multiple edge nodes with computing capabilities. Each edge node communicates with the central server through a network connection. These edge nodes can be, but are not limited to, intelligent connected vehicles, intelligent roadside monitoring equipment, and intelligent roadside computing units.

[0080] The central server acts as a coordination center and is responsible for the initialization, model update, aggregation, and model version management of the global model. In this embodiment, each edge node is denoted as a variable i and i∈[N], where N represents the number of edge nodes. Edge node i∈[N] has a D i Distributed local datasets (X i ,Y i ), where X irepresents the data features on edge node i, Y i Indicates the corresponding label, D i represents data distribution. Due to the heterogeneity of data, the data distribution of different edge nodes is different. In addition, edge node i also has n i From D i The training samples ξ are drawn from the distribution i Used for local training of the model. Generally speaking, the goal of the federated learning algorithm in the vehicle-road collaboration scenario is to learn a global model w that can adapt to all edge node data sets by minimizing the empirical loss function on each edge node. The training goal is:

[0081]

[0082] Among them, F i (·) represents the local experience loss of edge node i, represents the loss function, n i represents the number of training samples on edge node i, j represents the training sample index number of edge node i, and w represents the parameters of the global model.

[0083] As mentioned above, there are existing methods that introduce sharpness perception minimization into federated learning. This method adds a perturbation in the local training phase and performs an additional gradient ascent and descent operation, that is, by optimizing the objective function by obtaining a perturbation δ within the perturbation radius ρ, so that the loss is minimized at w+δ to find a low loss area. The training objective can be expressed as:

[0084]

[0085] Among them, the model parameters after disturbance are It can be derived by the following formula:

[0086] For each edge node i, first perform F i (w+δ i ) Perform a first-order Taylor expansion at w and transform it into a linear constrained optimization problem:

[0087]

[0088] Since ρ is very small, we ignore the higher-order terms O(ρ 2 ). At the same time, F i (w) can be considered a constant term because it depends only on w and not on δ i , so it has no effect on the results during the optimization process, so we can get:

[0089]

[0090] in, It is about δ i A linear function of is the gradient of the model parameter w, δ i Satisfy ||δ i ||≤ρ. This problem can be transformed into finding a point δ in the constraint set (the sphere centered at the origin). i , so that the objective function Since the objective function is a linear function, its maximum value always appears on the boundary of the constraint set. Here, the boundary is a sphere with a norm of ρ. Therefore, ||δ i ||=ρ, indicating the solution δ i It must be on a sphere with radius ρ.

[0091] maximize The essence of and Therefore, the direction of the optimal solution should be consistent with The same, that is:

[0092]

[0093] Among them, α represents a proportional coefficient. i ||=ρ and Substituting the constraints, we can get:

[0094]

[0095] Therefore, the proportionality coefficient can be solved Finally, substitute α back You can get:

[0096]

[0097] This step calculates the perturbation δ that causes the largest loss in the neighborhood of w i , and get the model parameter update after disturbance:

[0098]

[0099] However, due to the differences in data distribution of each edge node, it is impossible to ensure that the low-loss area of ​​each edge node is the same, which directly affects the flatness and stability of the global model. After the introduction of sharpness-aware minimization, an additional gradient ascent is added to the local training phase, which increases the computational burden.

[0100] Therefore, based on the above architecture and optimization issues, Figure 2This is a flowchart of a vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning provided by an embodiment of the present invention. Under the condition of data heterogeneity, this method can not only align low-loss areas and improve the flatness of the global model, but also effectively control the increase in computing cost caused by minimizing sharpness perception, including the following steps:

[0101] S1, the central server initializes the federated learning settings and configures the necessary model training parameters, where the necessary model training parameters include the initial global model w 0 , the global gradient change estimator g 0 , the total number of communication rounds R, the number of local iterations of each edge node K, the gradient control factor c 0 and Local learning rate η, global learning rate γ, momentum coefficient β, perturbation rate ρ, that is, the central server coordinates each edge node to conduct R rounds of federated training, and each edge node conducts K rounds of local training. This step is mainly to prepare for the implementation of federated learning training;

[0102] S2, the central server randomly selects a group of edge nodes as participants in this round of training. Specifically, in the rth round of communication, the central server samples a subset of edge nodes S r , and the global model w of this round of training r Send it to them;

[0103] S3, edge nodes i∈S participating in this round of training r Use your own training sample ξ i During the model training, the model parameters are sparsely perturbed to achieve lightweight sharpness perception minimization, and the parameters are corrected to promote the alignment of low-loss areas between different edge nodes. The specific steps include:

[0104] (1) For each edge node i∈S r The following training is carried out in parallel;

[0105] (2) Each edge node i receives the global model w sent by the central server r After that, initialize the local model: That is, the edge node uses the global model sent by the central server as the initial model for local training. represents the local model parameters of edge node i after the kth iteration in the rth round of communication, where is the local model parameter of edge node i after the 0th iteration in the rth round of communication.

[0106] (3) Each edge node calculates the gradient after adding the sparse perturbation. For the local iteration number k = 0, ... K-1, edge node i performs the following operations:

[0107] pass Perform local training and calculate gradients Then pass Calculate the model parameters after adding the perturbation

[0108] Where j represents the index of the training sample of edge node i at the kth iteration in the rth round of communication, and m is a random binary mask used to minimize the sharpness perception of sparsification, such as randomly selecting some elements to be 1 and the rest to be 0. This method uses a dynamic sparsity rate to generate a random mask:

[0109] α r =max(α 0 ·e -λr ,α min )

[0110] Among them, α r is the sparsity rate of the rth round of communication, α 0 is the initial sparsity rate, λ is the decay rate, α min is the minimum sparse rate. During the training process, the sparse rate α is dynamically adjusted. r The model performance can be optimized. In the initial stage, a higher sparsity rate can be used to accelerate convergence, and the sparsity rate can be gradually reduced in the later stage to refine the model. For each parameter position l, generate a random number n l :

[0111] n l ~U(0,1)

[0112] Then according to the current sparse rate α r Generate mask m:

[0113]

[0114] Among them, m l is the lth element of the binary mask m, then Compute the gradient after adding sparse perturbations

[0115] (4) Each edge node updates the model, performs parameter correction, and aligns the low-loss area:

[0116]

[0117] Among them, η is the local learning rate, β is the momentum coefficient, is the local gradient control factor of edge node r in round r, c r is the global gradient control factor of the rth round, g ris the global gradient change estimator of the rth round; represents the local model parameters of edge node i after the k+1th iteration in the rth round of communication. Introducing the global gradient control factor c r and the local gradient control factor To coordinate global and local goals. represents the iterative deviation of the local gradient of edge node i in the global optimization problem, c r represents the average deviation of all edge nodes participating in the training in the global optimization problem. It is a centralized quantity that reflects the deviation correction of the global optimization objective. To adjust the local gradient, that is, to subtract the local gradient control factor specific to each edge node The iteration bias is subtracted from the local gradient, and the global gradient control factor c is added r , adding the global average deviation to the update. In addition, the global gradient change estimate g in the previous round of training is introduced when the edge node performs model update r , which aggregates the gradient changes of each edge node participating in the training, and helps each edge node consider the overall optimization goal when updating to achieve global penetration in a manner similar to the momentum term.

[0118] Subsequently, edge node i updates the next round of local gradient control factor:

[0119]

[0120] Among them, for edge nodes S r represents the edge nodes participating in the rth round of federated learning training, K represents the total number of iterations of the edge nodes during local training, and k represents the number of the current iteration.

[0121] S4, upload the local model parameters trained by each edge node in S3 to the central server;

[0122] S5, after receiving the model parameters uploaded by each edge node, the central server updates and generates a new round of global gradient change estimation g r+1 :

[0123]

[0124] And thus generate a new global model w r+1 :

[0125] w r+1 =w r -γg r+1

[0126] Among them, γ is the global learning rate. Finally, the central server generates a new round of global gradient control factor c r+1 :

[0127]

[0128] At this point, this round of federated learning training is completed. Next, the central server will sample a new set of edge nodes and send the new round of global model w r+1 It is sent to these edge nodes for training until all rounds of training are completed.

[0129] In an embodiment of the present invention, the low-loss areas on the loss function found during the training of each edge node are aligned as much as possible through control factors, and low-loss areas and better solution spaces are explored from a global perspective, further smoothing the model update process. At the same time, sparse perturbations are performed through dynamic sparsity rates, alleviating the computational burden brought about by the introduction of sharpness perception minimization.

[0130] In order to further illustrate the effect of the present invention, the vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning of the present invention is experimentally verified:

[0131] 1. Experimental environment

[0132] The experimental platform built by the present invention is mainly carried out on the Ubuntu operating system (version 20.04). The configuration information of the equipment is as follows: AMD Ryzen 9 5950X 16-Core Processor CPU, 64GB memory, NVIDIA RTX 3090TiGPU 24G video memory, the programming language is Python3.8, and the experimental framework is Pytorch.

[0133] 2. Experimental verification

[0134] Experimental verification is carried out on two classic datasets commonly used in federated learning, CIFAR-10 and CIFAR-100.

[0135] (1) Dataset description

[0136] CIFAR-10 is a small dataset containing 10 categories of natural images, with images in each category evenly distributed. The dataset consists of 60,000 32×32 pixel color images, of which 50,000 are used for training and 10,000 for testing. Each category includes common objects and animals, such as airplanes, cars, birds, cats, etc., with the characteristics of uniform categories and balanced distribution. The CIFAR-100 dataset is an extended version of CIFAR-10, also consisting of 60,000 32×32 pixel color images, of which 50,000 are used for training and 10,000 are used for testing. It contains 100 categories, and the number of images in each category is relatively small. Compared with CIFAR-10, CIFAR-100 has more detailed categories and more challenging classification tasks, especially in data heterogeneity scenarios, which puts higher requirements on the robustness of federated learning algorithms.

[0137] (2) Model

[0138] The model used in the experiment is the classic ResNet-18 model.

[0139] (3) Construction of heterogeneous datasets

[0140] To simulate data heterogeneity scenarios, this experiment introduces two different heterogeneities, DIR(0.6) and DIR(0.1), to the CIFAR-10 and CIFAR-100 datasets through Dirichlet distribution (with parameters of 0.6 and 0.1). Dirichlet distribution is affected by parameter α, where the smaller α is, the higher the degree of data heterogeneity and the more uneven the data distribution is. Conversely, the lower the degree of data heterogeneity, the more even the data distribution is.

[0141] (4) Comparison of algorithms and parameter settings

[0142] The method of the present invention (denoted as FedGF) is compared with six state-of-the-art federated learning algorithms. These algorithms are FedAvg, Scaffold, FedCM, FedSAM, FedGAMMA and FedSpeed, which have the same focus as FedGF of the present invention, and all optimize the local training phase of the model. In this experiment, the number of communication rounds is set to 1000, the total number of clients is set to 100, the client sampling ratio is 0.1, the number of local iterations is set to 5, and the local training data batch size is set to 50. In addition, the local learning rate and the global learning rate are set to 0.1 and 1.0 respectively, the local learning rate is decayed by 0.998 after each round of communication, the momentum coefficient is set to 0.1, and the perturbation rate is set to 0.01.

[0143] (5) Experimental results and analysis

[0144] Comparative experiments were carried out on two datasets, CIFAR-10 and CIFAR-100, and three different degrees of data heterogeneity (IID, DIR(0.6), and DIR(0.1)).

[0145] Table 1 shows the test accuracy performance of the FedGF of the present invention and the above six algorithms after 1000 rounds of communication. The "IID" in the table indicates the uniform distribution of training samples on each client; "DIR (0.6)" indicates the non-independent and identically distributed situation of introducing general data heterogeneity by using the Dirichlet distribution (parameter is 0.6); "DIR (0.1)" indicates the non-independent and identically distributed situation of introducing high data heterogeneity by using the Dirichlet distribution (parameter is 0.1). It can be seen from the results in the table that the FedGF of the present invention has achieved the highest accuracy in both CIFAR-10 and CIFAR-100 data sets and in three different data distributions. Specifically, the accuracy of the FedGF of the present invention in the three different data distributions on the CIFAR-10 data set is 1.84%, 2.51% and 2.84% higher than that of the second-ranked algorithm, and the accuracy of the FedGF on the CIFAR-100 data set is 0.89%, 0.94% and 2.23% higher than that of the second-ranked algorithm. This shows that the present invention has significant advantages in dealing with data heterogeneity. In addition, it can be seen that on both data sets, as data heterogeneity increases, the accuracy of the FedGF of the present invention is higher than that of the second-ranked algorithm, which shows that when the data distribution is more biased towards certain categories or there is a greater imbalance, the FedGF of the present invention can more effectively reduce the negative impact of data imbalance on model performance and improve the overall accuracy. This advantage reflects the adaptability of FedGF to data heterogeneity.

[0146] Table 1 Test accuracy of the present invention under two data sets and different data heterogeneity settings (%)

[0147]

[0148] Figure 3 , Figure 4 , Figure 5The global test accuracy of FedGF of the present invention and other six algorithms under IID distribution, DIR (0.6) distribution and DIR (0.1) distribution on CIFAR-10 and CIFAR-100 data sets is shown as the change trend of the communication rounds. It can be seen from the results that FedGF of the present invention always performs well in most communication rounds, and finally achieves the highest test accuracy, which is significantly better than other comparison algorithms. In general, in the six figures, the curve of FedGF of the present invention is steeper, indicating that it converges faster in the early stage and can quickly improve the model performance. In the later communication rounds (such as after 500 rounds), the test accuracy of FedGF is more stable and continues to improve, showing excellent stability and convergence. The other six comparison algorithms either converge faster in the early stage but cannot achieve a higher and stable accuracy in the later stage, or the overall convergence speed and final accuracy are all weaker than FedGF of the present invention.

[0149] Table 2 shows the number of communication rounds required for FedGF of the present invention and other six algorithms to achieve the predetermined accuracy, where the predetermined accuracy of CIFAR-10 is 80% and that of CIFAR-100 is 45%. N / A in the table indicates that the algorithm cannot reach the corresponding predetermined accuracy. It can be seen from the data that FedGF of the present invention exhibits a strong performance. On the CIFAR-10 dataset, FedGF of the present invention requires 82 rounds of communication in the IID (80% accuracy) scenario, showing a significant efficiency advantage, far superior to other algorithms, such as FedAvg and Scaffold, which require 330 and 293 rounds, respectively. In the moderate data heterogeneity environment of DIR (0.6), FedGF of the present invention requires 112 rounds of communication, still maintaining its lead, showing strong adaptability. However, in the highly heterogeneous data environment of DIR (0.1), the number of communication rounds of FedGF of the present invention increases to 540 rounds, which is relatively high, but still more efficient than FedAvg and Scaffold. For the CIFAR-100 dataset, the FedGF of the present invention requires 92 rounds of communication in the IID scenario, which is significantly better than FedAvg and Scaffold, and requires 134 rounds in the DIR(0.6) scenario, which still performs well. However, in the highly heterogeneous environment of DIR(0.1), the number of rounds increases to 213 rounds. Despite this, the FedGF of the present invention is still more efficient than other algorithms. In general, the FedGF of the present invention shows great advantages in convergence speed.

[0150] Table 2 Number of communication rounds required to achieve the desired accuracy (times)

[0151]

[0152] Finally, it should be noted that the above is only used to illustrate the technical solution of the present invention, rather than to limit it; although the present invention is described in detail with reference to the above embodiment, a person skilled in the art should understand that the technical solution described in the above embodiment can still be modified, or some or all of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the technical solution of the embodiment of the present invention. Anything not described in the present invention is applicable to the prior art.

Claims

1. A vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning, characterized in that: include: S1, the central server initializes the federated learning parameters and prepares to start federated learning training; S2, the central server randomly selects a group of edge nodes as participants in this round of training and sends the current global model to these edge nodes; S3, the edge nodes participating in this round of training use their own data to train the model with the current global model as the local model, and minimize the sharpness perception of sparsifying the model parameters through dynamic sparsity to improve computing efficiency; The expression of the sparseness rate is: α r =max(α 0 ·e -λr , α min ) Among them, α r is the sparsity rate of the rth communication round, used to generate the mask; α 0 is the initial sparse rate, λ is the decay rate, α min is the minimum sparse rate. During the training process, the sparse rate α is dynamically adjusted. r Optimize model performance; according to the current sparsity rate α r Generate the mask: Among them, m l is the lth element of the binary mask m; n l represents the local model parameters of edge node i after the kth iteration in the rth round of communication A random number of O~1 is generated for each parameter position l; pass Calculate the model parameters after adding the perturbation in, is the gradient of edge node i after the kth iteration in the rth round of communication; ρ is the perturbation rate; Then according to the model parameters after adding the disturbance Further calculate the gradient after adding sparse perturbations With the gradient after adding sparse perturbation To update the local model for the next iteration; Each edge node performs K local trainings, for local iterations k=0,...K-1; S4, uploading the local model parameters trained by each edge node in step S3 to the central server; S5, the central server aggregates the received model parameters, generates a new global model, and starts the next round of training until all training rounds are completed.

2. The method according to claim 1, characterized in that In step S3, the update formula for the local model update in the next iteration is: Among them, η is the local learning rate, β is the momentum coefficient, is the local gradient control factor of edge node i in round r, c r is the global gradient control factor of the rth round, g r is the global gradient change estimator of the rth round; represents the local model parameters of edge node i after the k+1th iteration in the rth round of communication.

3. A vehicle-road collaboration heterogeneous scenario data optimization method based on sparse perturbation federated learning, characterized in that: include: S1, the central server initializes the federated learning parameters and prepares to start federated learning training; S2, the central server randomly selects a group of edge nodes as participants in this round of training and sends the current global model to these edge nodes; S3, the edge nodes participating in this round of training use their own data to train the model with the current global model as the local model, minimize the sharpness perception of sparse model parameters through the sparsity rate, and use the global gradient control factor, local gradient control factor and global gradient change estimator to correct the update parameters of the local model. The update formula of the local model is: Among them, η is the local learning rate, β is the momentum coefficient, is the local gradient control factor of edge node i in round r, c r is the global gradient control factor of the rth round, g r is the global gradient change estimator of the rth round; represents the local model parameters of edge node i after the k+1th iteration in the rth round of communication; Introducing the global gradient control factor c r and the local gradient control factor of edge node i To coordinate global and local goals; when updating the model at the edge node, the change estimate of the global gradient in the previous round of training is introduced r , aggregates the gradient changes of each edge node participating in the training, helps each edge node consider the overall optimization goal when updating to achieve global penetration, and promotes the alignment of low-loss areas between different edge nodes; S4, uploading the local model parameters trained by each edge node in step S3 to the central server; S5, the central server aggregates the received model parameters, generates a new global model, and starts the next round of training until all training rounds are completed.

4. The method according to claim 2 or 3, characterized in that: The local gradient control factor of edge node i in the next round The update formula is: Among them, for edge nodes S r represents the edge nodes participating in the rth round of federated learning training; K represents the total number of iterations of the edge nodes when performing local training.

5. The method according to claim 2 or 3, characterized in that: In step S5, the process of the central server aggregating the received model parameters includes: Update and generate a new round of global gradient change estimator g r+1 : Among them, S r represents the edge nodes participating in the rth round of federated learning training; represents the local model parameters of edge node i after the Kth iteration in the rth round of communication; w r is the global model in the rth round of communication; According to the new round of global gradient change estimator g r+1 Generate a new global model w r+1 : In r+1 =in r -γg r+1 Among them, γ is the global learning rate; Finally, the central server generates a new round of global gradient control factor c r+1 : in is the local gradient control factor of edge node i in the r+1th round.

6. The method according to claim 1 or 3, characterized in that: Initialization of federated learning parameters in step S1 includes: initial global model w 0 , global gradient change estimator g 0 , the total number of communication rounds R, the number of local iterations of each edge node K, the initial gradient control factor c 0 and local learning rate η, global learning rate γ, momentum coefficient β and perturbation rate ρ; N represents the number of edge nodes.

7. The method according to claim 1 or 3, characterized in that: The training objectives of federated learning are: Among them, F i (·) represents the local experience loss of edge node i, represents the loss function, n i represents the number of training samples on edge node i, ξ i represents the number of n owned by edge node i i training samples; j represents the training sample index number of edge node i, and w represents the parameters of the global model.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it achieves the following: There is a central server and multiple clients with computing power, each of which communicates with the central server through a network connection. Based on federated learning, the central server initializes the global model and loads the local model for the client. When training the local model, the model parameters are sparsely spaced by dynamic sparsity rate to minimize the sharpness perception. The expression of the sparseness rate is: α r =max(α 0 ·e -λ r, α min ) Among them, α r is the sparsity rate of the rth communication round, used to generate the mask; α 0 is the initial sparse rate, λ is the decay rate, α min is the minimum sparse rate. During the training process, the sparse rate α is dynamically adjusted. r Optimize model performance; according to the current sparsity rate α r Generate the mask: Among them, m l is the lth element of the binary mask m; n l represents the local model parameters of client i after the kth iteration in the rth round of communication A random number from O to I is generated for each parameter position l; pass Calculate the model parameters after adding the perturbation in, is the gradient of client i after the kth iteration in the rth round of communication; ρ is the perturbation rate; Then according to the model parameters after adding the disturbance Further calculate the gradient after adding sparse perturbations With the gradient after adding sparse perturbation To update the local model for the next iteration; The update formula for the local model update in the next iteration is: Among them, η is the local learning rate, β is the momentum coefficient, is the local gradient control factor of client i in round r, c r is the global gradient control factor of the rth round, g r is the global gradient change estimator of the rth round; represents the local model parameters of client i after the k+1th iteration in the rth round of communication.

Images