A narrow asynchronous personalized federated learning adaptive aggregation method

Abstract

The present invention discloses a narrow sense asynchronous personalized federated learning adaptive aggregation method, which belongs to the field of vehicle networking; first, for a vehicle client that newly enters the RSU service range at a certain moment, the current model is downloaded from the federated server as a local model, and is trained using a local data set and then uploaded to the server; the server calculates the Euclidean distance to represent the difference between models; then, the difference between models, global model accuracy and model loss are used to represent the current state, and the action to be taken is adaptively obtained through the A2C algorithm, which is the aggregation weight. The weight is used to perform global aggregation to obtain a new global model, and the reward obtained by executing this action is calculated based on the improvement in model accuracy and the attenuation of model loss, and the quality of the result of this action is quantified. Learning is performed through the A2C value network until all vehicle clients complete the personalized federated learning process. The present invention can quickly achieve the convergence of the global model with high accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of Internet of Vehicles, and specifically relates to a narrow sense asynchronous personalized federated learning adaptive aggregation method. Background Art

[0002] Federated learning is a distributed machine learning method in which clients train a model locally and then upload only the model parameters to a server for global aggregation. Because clients don't need to share raw data, federated learning reduces communication overhead while protecting client data privacy.

[0003] The Internet of Vehicles (IoV) is an important application scenario for federated learning. Vehicles typically hold a large amount of private data and have limited communication, computing, and storage resources. In IoV, roadside units (RSUs), deployed as road infrastructure, can act as servers and communicate with several vehicles acting as clients within their coverage area to conduct federated learning and meet application requirements.

[0004] However, federated learning in the Internet of Vehicles scenario still faces special challenges: on the one hand, the high mobility of vehicles can cause them to immediately leave the coverage of the roadside unit (RSU), causing them to fall behind or exit, and on the other hand, can cause communication to be offline or interrupted due to unstable communication. Exit and interruption during the learning and communication process will affect the overall latency and model accuracy of federated learning. On the other hand, some Internet of Vehicles applications mainly focus on road traffic characteristics, network characteristics and local correlations in different regions, that is, vehicle data has local characteristics. This heterogeneous phenomenon of non-independent identically distributed (non-IID) data will affect the aggregation speed and model convergence of federated learning.

[0005] The narrow asynchronous federated learning method means that after each vehicle client completes local training and uploads the model parameters to the RSU, the server will immediately perform a model aggregation, instead of waiting for several models to arrive before performing aggregation as in the traditional method. This method alleviates the negative effects of high vehicle mobility because exiting or interrupted clients will not affect the aggregation process of other clients. At the same time, for the phenomenon of non-IID highly heterogeneous data, the widespread use of personalized federated learning can solve the problem of poor convergence. Therefore, it is of great significance to use a narrow asynchronous method for personalized federated learning in the Internet of Vehicles scenario.

[0006] Existing solutions to mobility problems can be divided into three categories:

[0007] The first approach involves predicting the mobility of all client vehicles at the beginning of each round of federated learning. The server then selects vehicles that can complete this round of federated learning before leaving the server's RSU coverage area, minimizing the impact of mobility. However, this approach, which relies on predicting vehicle mobility over a period of time, suffers from low accuracy and requires the RSU server to collect all vehicle mobility information before making a selection decision. This poses a privacy risk.

[0008] The second approach involves the server collecting only a portion of the model parameters while waiting for the client to upload them, and then performing global aggregation. Collecting only a portion of the client parameters inevitably discards model parameters that haven't arrived in a while. While this avoids the server having to wait too long (i.e., the discrete effect), it also addresses the impact of high mobility to a certain extent. However, the core challenge of this approach lies in determining the threshold for waiting for aggregation. Furthermore, this approach still involves a global server waiting process, which introduces certain unreliability.

[0009] The third approach is narrow asynchronous federated learning. This approach eliminates the need for the server to consider the movement of all clients or wait for the arrival of a certain number of models before performing global aggregation. A client's vehicle exit or communication interruption will not affect the learning process of other clients. Furthermore, narrow asynchronous approaches can enable more client participation in federated learning than other approaches.

[0010] Existing federated learning solutions for data heterogeneity problems are all set in the scenarios of synchronous or generalized asynchronous federated learning. When it comes to model aggregation weights, factors such as local dataset size and local model characteristics are usually considered. The multi-agent aggregation weight decision methods used include sliding average weights, linear weights, square weights, and differential weights.

[0011] For example, in existing document 1: a federated learning load forecasting method based on dynamic weighted aggregation with application number CN202210506246.0, for all local models participating in training, the similarity between their parameter change vectors is calculated; then the similarity matrix of this round is generated, and the consistency vector between each local model is calculated; based on the accuracy of the local model on the server verification set and the consistency between each local model, weighted aggregation processing is performed on the server to obtain a global model.

[0012] Although the global server considers model consistency when evaluating local models before aggregation, this invention uses cosine similarity for measurement; the accuracy of local models on the server validation set needs to be considered separately; in a synchronous federated learning setting, the normalized aggregation weights of several local models are considered during global aggregation;

[0013] Document 2: Application number CN202210754970.5 discloses a personalized federated learning method and apparatus for processing imbalanced data. The central server establishes a parameter dictionary for each client, stores the model parameters uploaded by each client, and updates the parameter dictionary based on the latest received data in each iteration. The central server constructs a similarity matrix dictionary to store the similarity values ​​between the clients, and calculates the similarity value between each two clients participating in the training using the cosine similarity formula based on the basic coding layer model parameters of each client participating in the training. The similarity value is then updated to the similarity matrix dictionary as a weight coefficient. However, this method has the following disadvantages: the invention uses a normalized matrix dictionary of cosine similarity for measurement; a client model is compared with each other client model for similarity, and then the client local model is updated.

[0014] Document 3: Application number CN202111450243.1 is a robot collaborative control algorithm based on federated learning. For each master-slave robot system client, asynchronous transmission updates are used. After a fixed number of local training rounds, each neural network parameter is sent to the parameter server, and the latest neural network parameters are waited for. After the server receives the parameters, it uses a sliding average method to aggregate them into new weights. This invention uses the mean square error loss function as the evaluation criterion, and uses a sliding average method to aggregate the original model on the server and the model transmitted to the server from the master-slave robot system client to obtain a new model for the server;

[0015] Document 4: Application number CN202110616360.4 describes an efficient federated learning method for the Internet of Vehicles (IoV) scenario. Once a roadside unit receives a local training model uploaded by a vehicle, it calculates the weight of the local training model in real time. It then weights and aggregates the local training model uploaded by the vehicle into a global model in real time, generates the current model parameters for the training task, and returns them to each vehicle in real time. This invention sets a certification level for each vehicle. Before each aggregation, the vehicle's rating is assessed by comprehensively considering the vehicle's historical performance and the total number of vehicles participating in the current federated learning task. The levels are then normalized to form the aggregation weight.

[0016] In summary, it can be seen that few methods in existing federated learning technologies consider the negative impact of vehicle mobility on overall latency and model accuracy; when performing global aggregation in federated learning, the solution to the imbalanced data problem is usually based on model similarity using relatively simple aggregation weight decision methods such as sliding average weights, linear weights, square weights, and differential weights, which lacks consideration of the overall situation of the system. The present invention aims to reduce the impact of vehicle mobility problems on learning latency and model accuracy to a greater extent by improving the process of the federated learning system; by considering the current model system state during global aggregation to adaptively determine the appropriate aggregation weight, the problem of poor convergence caused by imbalanced data is solved, and training is ultimately achieved to obtain a global model with better accuracy and loss performance. Summary of the Invention

[0017] To address the above problems, the present invention proposes an adaptive aggregation method for narrow asynchronous personalized federated learning. It uses Euclidean distance to represent the feature differences between models, considers factors such as model differences, model accuracy, and model loss, and uses the Advantage Actor-Critic (A2C) algorithm in the field of reinforcement learning to adaptively derive personalized weights during narrow asynchronous federated learning aggregation.

[0018] The narrow sense asynchronous personalized federated learning adaptive aggregation method has the following specific steps:

[0019] Step 1: For a roadside unit RSU, the client of vehicle m that enters the service range of the RSU at time t The current global model is downloaded from the federation server at the RSU as the local model of the vehicle client.

[0020] Step 2: Use the local dataset to train the local model of vehicle m locally, and use the trained local model Upload to the RSU's federated server;

[0021] Step 3: The federated server receives the local model Calculate the Euclidean distance between it and the current global model to indicate the feature differences between the models.

[0022] Step 4: Feature Differences Between Federated Server Usage Models Global Model Accuracy and losses To indicate the current state The action to be taken is obtained through the A2C algorithm adaptation, which is the aggregation weight

[0023] Step 5: Federated Server Using Aggregate Weight Perform global aggregation to obtain a new global model The accuracy of this new global model loss And the new global model Compared with the original global model characteristic differences Constitute the next state

[0024] Step 6: Calculate the reward for executing this action based on the improvement in model accuracy and loss attenuation Quantify the quality of the results of this action.

[0025] Step 7: Based on the current state of vehicle m Next status and rewards Learning through the A2C value network.

[0026] Step 8. Select the next vehicle client and repeat the above steps to implement the narrow asynchronous adaptive weight aggregation personalized federated learning process for each vehicle client, and finally obtain a global model with good accuracy performance.

[0027] The advantages of the present invention are:

[0028] Compared to existing technologies, this invention enables more vehicle clients to participate in the global aggregation process of federated learning, achieves global model convergence faster, and achieves higher model accuracy. Data verification shows that the global model accuracy is close to 80%. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A diagram of the Internet of Vehicles scenario for the narrow sense asynchronous personalized federated learning adaptive aggregation built for the present invention;

[0030] Figure 2 This is a flowchart of a narrow sense asynchronous personalized federated learning adaptive aggregation method of the present invention;

[0031] Figure 3 This is a comparison chart of the number of clients participating in federated learning in the narrow asynchronous and broad asynchronous processes of the present invention;

[0032] Figure 4 Schematic diagram of global model accuracy in the federated learning process of the present invention;

[0033] Figure 5 Schematic diagram of global model loss in the federated learning process of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described below with reference to the following embodiments and accompanying drawings. It should be understood that these embodiments are intended only to illustrate the technical solutions and implementations of the present invention and are not intended to be exhaustive. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without making any inventive changes to the embodiments shall fall within the scope of protection of the present invention.

[0035] The present invention discloses a narrow asynchronous personalized federated learning adaptive aggregation method, which uses a narrow asynchronous approach in the Internet of Vehicles (IoV) to reduce the impact of vehicle mobility on falling behind and dropping out of federated learning. In narrow asynchronous aggregation, the present invention addresses the problem of poor model convergence caused by highly heterogeneous non-IID client data. This method considers the feature differences between models represented by Euclidean distance, model accuracy, and model loss, and utilizes the A2C algorithm to derive the global aggregation weight of the vehicle client, ultimately resulting in a global model with faster convergence and higher accuracy.

[0036] In such Figure 1 In the narrow asynchronous federated learning in the Internet of Vehicles scenario shown, the local model of the vehicle client will be uploaded to the global server at the RSU for aggregation. Due to the problem of highly heterogeneous non-IID data on the client side, the determination of the aggregation weight will take into account the difference between the local model and the global model, that is, whether the distribution of the local data of the vehicle client is consistent with the local characteristics of the data within the RSU range. The purpose of aggregation is to ultimately obtain a global model with higher accuracy and lower loss in a shorter time.

[0037] Therefore, the present invention comprehensively considers the differences between models, the improvement of model accuracy, and the attenuation of model loss, so as to obtain the global aggregation weight during the narrow asynchronous aggregation of the client local model.

[0038] like Figure 2 As shown in Figure 2, the specific steps of the narrow asynchronous personalized federated learning adaptive aggregation method are as follows:

[0039] Step 1: For a roadside unit (RSU), a vehicle client that newly enters the service range of the RSU at time t downloads the current global model from the federation server at the RSU as the local model of each vehicle client.

[0040] The federated server deployed on the RSU is s, and the set of all vehicle clients that newly enter the service range of server s at time t is

[0041] Note that for any two different times t i and t j (i≠j), Because a vehicle client only enters the service range at a certain moment and remains in the service range until it leaves.

[0042] Step 2: Client for vehicle m Use the local dataset to train the local model locally and use the trained local model Upload to the RSU's federated server;

[0043] Client for vehicle m At time t, the dataset on vehicle m is To express, Represents the local model of vehicle m at time t. The time overhead of local training and model upload of vehicle m client is To express.

[0044] The local model uploaded after training the vehicle m using the local dataset; this model will be uploaded to the server s.

[0045] Step 3: The federated server receives the local model Calculate the Euclidean distance between it and the current global model to indicate the feature differences between the models.

[0046] The Euclidean distance between the local model and the global model of vehicle m is calculated as:

[0047]

[0048] Represents the global model before aggregation on the federated server s.

[0049] Step 4: The federated server uses the feature differences between models, global model accuracy, and model loss to represent the current state of vehicle m. The action to be taken is obtained through the adaptive A2C algorithm, which is the aggregation weight.

[0050] After the server receives the local model and related information uploaded by client m, it needs to adaptively determine its aggregation weight according to the model situation. The current state of vehicle m Including pre-aggregation global model The precision value Global model before aggregation The loss value Feature differences between local and global models

[0051] The training objective is defined as maximizing the accuracy improvement and loss attenuation of the global model:

[0052]

[0053] in Indicates the improvement in global model accuracy, Represents the attenuation value of the global model loss; α and β are fixed parameters.

[0054] Through the A2C algorithm's Actor strategy network π, the weight of the vehicle m local model used for aggregation is adaptively obtained The calculation process is:

[0055]

[0056] Step 5: Federated Server Using Aggregate Weight Perform global aggregation to obtain a new global model This new global model Accuracy Global Model loss And the new global model Compared with the original global model characteristic differences Constitute the next state

[0057] In Advanced Actor-Critic (A2C) reinforcement learning, the state is the model state at that moment (feature difference, model accuracy, model loss). The action output by the actor network is the local model aggregation weight. The optimization target is the reward. The next state is the global model state after aggregation, that is:

[0058] state:

[0059] action:

[0060] reward function:

[0061] next state:

[0062] in It represents the degree of feature difference between the new global model and the original global model after global aggregation, that is:

[0063]

[0064] Because the Actor uses the policy network to determine the appropriate weight for this aggregation based on the current model state, the model state changes after aggregation using this weight. The Critic uses the state before the action, the current state after the action, and the action taken to quantify the quality of the action's outcome based on a reward function, learning through the value network. Each time a local model is uploaded to each vehicle client, the above steps are repeated to implement machine learning, ultimately resulting in a global model with high accuracy.

[0065] Step 6: Calculate the reward for executing this action based on the improvement in model accuracy and loss attenuation Quantify the quality of the results of this action.

[0066] Step 7: Based on the current state of vehicle m Next status and rewards Learning through the A2C value network.

[0067] Step 8. Return to step 2 to select the next vehicle client and repeat the narrow asynchronous adaptive weight aggregation personalized federated learning process for each vehicle client to eventually obtain a global model with good accuracy performance.

[0068] Example:

[0069] At time t, the vehicle client When a new client enters the service range of server s, it downloads the global model w from the server. t As a local model Using a local dataset Perform model training. After local training is completed, the client will use the new local model Upload to server s.

[0070] After receiving the data, the server s calculates the feature difference between the local model of the vehicle client and the global model. Leveraging global model accuracy Global model loss Model feature differences Represents the current state Aggregation weights are obtained through adaptive Actor components of the A2C network

[0071] Server s uses this weight Perform global aggregation to obtain a new global model. The accuracy of the global model is The global model loss is The characteristic difference between the new global model and the original global model is That is, the status at this time is

[0072] Next, further calculate the reward for this action, combining the two states before and after the action and Learning is performed through the Critic component of the A2C network.

[0073] This invention solves the problem of client falling behind and exiting due to vehicle mobility in federated learning in the Internet of Vehicles scenario. For highly heterogeneous data in federated learning, the adaptive aggregation weight is obtained using the A2C algorithm, taking into account the factors of the degree of difference between models, model accuracy, and model loss.

[0074] The present invention uses the real mobile trajectory dataset inD, uses image classification as the machine learning task, selects ResNet-18 as the machine learning model, and verifies the feasibility of the method by using the cifar-10 image dataset processed by the Dirichlet non-IID method.

[0075] The final data showed that Figure 3 As shown in , among all 160 mobile trajectories within the selected time end, the broad asynchronous federated learning setting makes only 143 vehicle clients of the trajectory participate in the global aggregation, while the narrow asynchronous method has a participation number of 159. And at the same time, Figure 4 and Figure 5 As shown, compared with the generalized asynchronous method (gen_asy_FedSA) and other federated learning methods (Scaffold, PerFedAvg) that consider highly heterogeneous data problems, the narrow asynchronous adaptive aggregation method (spe_asy_FedSA) of the present invention makes the global model converge faster, and the global model accuracy and loss performance are better. The final model accuracy reaches 79.28%, and the model loss is as low as 0.545, which is similar to the effect under the ideal case of IID data.

Claims

1. A narrow sense asynchronous personalized federated learning adaptive aggregation method, characterized by: The specific steps are as follows: Step 1: For a roadside unit (RSU), each vehicle client that enters the service range of the RSU at time t downloads the current global model from the federation server at the RSU as the local model of each vehicle client. Step 2: Client for vehicle m Use the local dataset to train the local model locally and use the trained local model Upload to the RSU's federated server; represents the time overhead of local training and model uploading for vehicle m; Step 3: The federated server receives the local model Calculate the Euclidean distance between it and the current global model to indicate the feature differences between the models; The Euclidean distance between the local model and the global model of vehicle m is calculated as: represents the global model before aggregation on the federated server s; Step 4: Feature Differences Between Federated Server Usage Models Global Model Accuracy and losses To indicate the current state The action to be taken is obtained through the A2C algorithm adaptation, which is the aggregation weight Step 5: Federated Server Using Aggregate Weight Perform global aggregation to obtain a new global model This new global model Accuracy Global Model loss And the new global model Compared with the original global model characteristic differences Constitute the next state Step 6: Calculate the reward for executing this action based on the improvement in model accuracy and loss attenuation Quantify the quality of the results of this action; Step 7: Based on the current state of vehicle m Next status and rewards Learning through the A2C value network; Step 8. Return to step 2 to traverse each vehicle client and repeat the narrow asynchronous adaptive weight aggregation personalized federated learning process for each vehicle client to finally obtain a global model with good accuracy performance.

2. The method for adaptive aggregation of narrow asynchronous personalized federated learning according to claim 1, characterized in that: The federated server deployed on the RSU in step 1 is s, and the set of all vehicle clients that newly enter the service range of server s at time t is 3. The narrow sense asynchronous personalized federated learning adaptive aggregation method according to claim 1, characterized in that: The aggregation weight in step 4 The calculation process is: π is the Actor strategy network of the A2C algorithm, the current state of vehicle m Including pre-aggregation global model The precision value Global model before aggregation The loss value Feature differences between local and global models 4. The method for adaptive aggregation of narrow asynchronous personalized federated learning according to claim 1, characterized in that: In step 5, the degree of feature difference between the new global model and the original global model is calculated as follows: It is the new global model after global aggregation.

5. The narrow sense asynchronous personalized federated learning adaptive aggregation method according to claim 1, characterized in that: In step 6, the calculation formula for the accuracy improvement and loss attenuation of the global model is: in Indicates the improvement in global model accuracy, Represents the attenuation value of the global model loss; α and β are fixed parameters.

Images