A vehicle federated learning method and system based on clustering and computation offloading

By adopting a vehicle federated learning method based on clustering and computational offloading in the intelligent transportation system, clustering and selecting cluster head vehicles, implementing stratified federated learning and performing computational offloading in the cluster, communication blockage in vehicle federated learning and The problem of high communication costs improves efficiency and model accuracy and protects user privacy.

CN118714537BActive Publication Date: 2024-11-19SHANDONG NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411184249.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-19
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

In intelligent transportation systems, communication blockage and high communication costs of vehicle federated learning are inefficient in training due to scarce wireless resources and non-independent and homogeneous data distributed, model accuracy is reduced, and user privacy protection is difficult to achieve.

Method used

Using a vehicle federated learning method based on clustering and computational offloading, the vehicles are clustered and clustered through the improved K-means algorithm, layered federated learning is implemented, and computational offloading is performed in the cluster, and the task completion time is optimized. Reduce communication overhead.

Benefits of technology

It effectively reduces task completion time, reduces communication overhead, improves the efficiency and model accuracy of vehicle federated learning, and protects user privacy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118714537B_ABST
    Figure CN118714537B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of wireless communication technology for vehicle networks, and provides a method and system for vehicle federated learning based on clustering and computation offloading, including the following steps: constructing a system network model for vehicle federated learning including multiple vehicles and a base station with a server; clustering the vehicles based on an improved K-means algorithm, and selecting the vehicle with the highest score in each cluster as the cluster head vehicle through comprehensive scoring; establishing a computation offloading model and a delay model for the local computation and communication conditions of each cluster after clustering; defining the optimization problem as minimizing the time to complete a task; and optimizing the offloading decision to minimize the time to complete the task. The present invention clusters the vehicles, and after clustering, the vehicles do not need to frequently transmit model parameters to a distant base station, thereby reducing communication overhead; partially offloading the computational tasks of vehicles with slow model training to other vehicles with fast model training in the cluster for computation, thereby reducing the time to complete the task.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology for vehicle networks, and in particular to a vehicle federated learning method and system based on clustering and computation offloading. Background Art

[0002] As an important emerging technology in the 6G era, Intelligent Transportation System (ITS) combines artificial intelligence (AI) and vehicle edge computing (VEC) to give birth to future vehicle networks that can support autonomous driving and some vehicle applications, further expanding the potential value of vehicle data. Autonomous vehicles collect a large amount of important data by equipping them with lidar sensors, camera sensors, and radar sensors, and share them with vehicle networks. Artificial intelligence (AI) can use this data to achieve a variety of applications such as object detection and tracking, traffic sign classification, lane flow estimation, trajectory route planning, and high-definition map generation. However, in recent years, with the enhancement of personal privacy protection awareness, vehicle users are reluctant to expose their information to other users, nor are they willing to upload data to edge computing (MEC) servers or the cloud, because this may lead to malicious interception and abuse of vehicle user information.

[0003] Federated learning (FL), as an emerging distributed machine learning paradigm, allows vehicle users to train local models on their private datasets and upload the trained models to the base station (central server) for global aggregation. The aggregated global model is broadcast to users for the next iteration of FL training, and the iteration process continues until it converges to a certain global federated learning model accuracy. Unlike uploading raw data, FL allows distributed users to train global models by uploading local models to a central server, thereby reducing communication overhead and protecting user privacy.

[0004] In intelligent transportation systems, some Internet of Vehicles tasks involve vehicle safety control and protection, and vehicles need to respond and process tasks accurately and quickly. However, due to the scarcity of wireless resources, a large number of vehicle users participating in federated learning (FL) training at the same time will cause communication congestion, resulting in low FL training efficiency. Moreover, the distance between vehicle users and base stations is far, and frequent uploading and downloading of models will sharply increase the overall communication overhead. In addition, the current FL algorithm also faces the challenge of non-independent and identically distributed (Non-IID) data. The higher the degree of Non-IID in the data, the slower the convergence speed of the FL algorithm, resulting in high communication costs and reduced model accuracy. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a vehicle federated learning method and system based on clustering and computation offloading. The improved K-means method is used to divide vehicle users into multiple clusters and select cluster head vehicles to perform hierarchical federated learning (HFL). In order to speed up the completion of tasks, an intra-cluster computation offloading strategy is introduced to offload part of the tasks of vehicles that complete tasks slowly to other vehicles in the cluster that complete tasks quickly, thereby increasing the speed at which the slowest vehicle completes the task, thereby reducing the overall task completion time.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a vehicle federated learning method based on clustering and computation offloading, the method comprising:

[0008] Construct a system network model of vehicle federated learning that includes multiple vehicles and a base station with a server;

[0009] The vehicles are clustered based on the improved K-means algorithm, and the vehicle with the highest score in each cluster is selected as the cluster head vehicle through comprehensive scoring;

[0010] According to the local computing and communication conditions of each cluster after clustering, a computing offloading model and a latency model are established;

[0011] The optimization problem is to minimize the time to complete the task;

[0012] Optimize offloading decisions to minimize task completion time.

[0013] In a possible implementation, the vehicles are clustered based on the improved K-means algorithm, and the vehicle with the highest score in each cluster is selected as the cluster head vehicle through comprehensive scoring, which specifically includes:

[0014] S21: Collect from vehicle K cars are randomly selected as the initial mass points {μ1, μ2, ..., μ K}, forming the initial K intermediate clusters

[0015] S22: Calculate the relationship between vehicle n and each of the K intermediate clusters The distance γ n,k ;

[0016] S23: According to the distance γ n,k The value of , adds vehicle n to the distance γ n,k in the smallest middle cluster;

[0017] S24: Repeat S22-S23 until all vehicles are added to the cluster, and finally obtain K clusters.

[0018] S25: In each cluster The fraction S of each vehicle that becomes the cluster head is calculated separately n , select the score S n The largest vehicle is the cluster head vehicle.

[0019] In a possible implementation, the distance γ n,k The calculation formula is as follows:

[0020]

[0021] in,

[0022]

[0023]

[0024] Represents vehicle n and the middle cluster The average relative distance of all vehicles in |N′ k | indicates the middle cluster The number of vehicles in the n,m Represents vehicle n and the middle cluster The relative distance between vehicles m; Represents vehicle n and the middle cluster The average relative speed of all vehicles in; v n represents the speed of vehicle n; v m Represents the middle cluster The speed of vehicle m; Indicates that if vehicle n is clustered into the middle cluster Middle cluster The degree of Non-IID; c represents different categories, represents the set of all categories; p m (c) represents the proportion of category c in the data of vehicle m participating in FL training; p(c) represents the proportion of category c when the data is independent and identically distributed; β1, β2 and β3 represent weighting coefficients.

[0025] In a possible implementation, the score S n The calculation formula is as follows:

[0026]

[0027] in,

[0028]

[0029]

[0030] f n represents the CPU frequency of local calculation of vehicle n; b n It represents the number of CPU cycles required for vehicle n to calculate a sample data; Represents the transmission power of the model parameters uploaded by vehicle n; |N k | indicates a cluster The number of vehicles in; d n,m Representation Cluster The relative distance between vehicle n and vehicle m; Δv n,m Representation Cluster The relative speed between vehicle n and vehicle m; LLT n,m Represents vehicle n and cluster The link lifetime of other vehicles m in the n represents the speed of vehicle n; v m represents the speed of vehicle m; TR represents the transmission range; ω1, ω2, ω3, ω4, ω5 are weight factors, and ω1, ω2, ω3, ω4, ω5∈(0, 1).

[0031] In a possible implementation, the computation offloading model is specifically:

[0032] In cluster In the calculation, the vehicle n that needs to be unloaded selects a vehicle m in the cluster for unloading. The amount of data O of vehicle n involved in model training n for:

[0033]

[0034] Among them, D n Indicates the size of the local data of vehicle n; a n,m represents the decision of vehicle n to offload computation to vehicle m. If a n,m =1, indicating that vehicle n chooses to unload calculations to vehicle m. If a n,m =0, indicating that vehicle n does not choose to offload computation to vehicle m; α n represents the ratio of vehicle n to vehicle m in terms of computational unloading; a m,n Indicates that vehicle m chooses to offload computation to vehicle n. If a m,n =1, indicating that vehicle m chooses to unload the calculation to vehicle n. If a m,n =0, indicating that vehicle m does not choose to offload computation to vehicle n; α m represents the ratio of vehicle m to vehicle n in terms of computational unloading, Dm Indicates the size of the local data of vehicle m.

[0035] In a possible implementation, the delay model is specifically:

[0036] The total time T spent on the entire HFL training process total for:

[0037]

[0038] Where J represents the total number of global iterations; Represents the set of different clusters after clustering; Indicates that in the jth global iteration, cluster The time it takes for the vehicle to participate in I local iterations; represents the delay of cluster head vehicle uploading local aggregation model in the jth global iteration;

[0039] The calculation formula is:

[0040]

[0041] in, Representation Cluster The computation offloading delay of represents the local model training delay of vehicle n in each local iteration; Indicates that in the first global iteration and the first local iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k The delay; I represents the total number of local iterations; Indicates that in the ith local iteration of the first global iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k Delay Indicates that in the jth global iteration and the ith local iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k Delay

[0042] The calculation formula is:

[0043]

[0044] Where L represents the number of rounds of local training; O n represents the amount of data for vehicle n involved in model training; b n represents the number of CPU cycles required for vehicle n to calculate a sample data; f n Indicates the CPU frequency of local calculation of vehicle n;

[0045] The calculation formula is:

[0046]

[0047] Among them, Z represents the size of the model parameters, Indicates that in the jth global iteration and the ith local iteration, the data uploaded from vehicle n to cluster head vehicle h k The transmission model;

[0048] The calculation formula is:

[0049]

[0050] Among them, h k represents the cluster head vehicle; represents the set of cluster head vehicles; Z represents the size of the model parameters; It represents the transmission model of the cluster head vehicle uploading the local aggregation model in the jth global iteration.

[0051] In one possible implementation, the cluster Computation offloading latency The calculation formula is:

[0052]

[0053] in, Representation Cluster Data unloading delay of vehicle n unloading local data to vehicle m,

[0054]

[0055] Among them, a n,m represents the computation offloading decision of vehicle n to vehicle m; α n represents the ratio of vehicle n to vehicle m in terms of computational unloading; D n represents the size of the local data of vehicle n; r n,m represents the transmission model,

[0056]

[0057] Where B represents the total bandwidth; N represents the number of vehicles participating in federated learning; represents the transmission power of the model parameters uploaded by vehicle n; g n,m represents the channel gain between vehicle n and vehicle m; N0 represents the noise power density.

[0058] In a possible implementation, the explicit optimization problem is to minimize the time to complete the task, specifically:

[0059] P:

[0060] St

[0061] C1:

[0062] C2:

[0063] C3:

[0064] C4:

[0065] C5:

[0066] in, represents the computational unloading decision matrix for each vehicle; α = {α1, ..., α n , ..., α N} represents the calculated unloading amount vector of each vehicle; a m,n represents the computation offloading decision of vehicle n to vehicle m; a m,n represents the computation offloading decision of vehicle m to vehicle n; α n It indicates the ratio of vehicle n to vehicle m for computational unloading. Constraints C1 and C2 indicate that the vehicle can only select one vehicle in the cluster for computational unloading. Constraint C3 indicates that the vehicle can only be one of the unloading vehicles and the service vehicle to avoid mutual unloading and forming a loop. Constraint C4 indicates that the vehicle can only select vehicles in the cluster for computational unloading and cannot select vehicles outside the cluster. Constraint C5 indicates the unloading ratio constraint.

[0067] In a possible implementation, the optimizing the offloading decision specifically comprises: optimizing the offloading decision by using a deep deterministic policy gradient algorithm.

[0068] In a second aspect, the present invention provides a vehicle federated learning system based on clustering and computation offloading, the system comprising:

[0069] A data acquisition module is configured to acquire vehicle status data;

[0070] The clustering and cluster head vehicle selection module is configured to cluster the vehicles based on the improved K-means algorithm, and select the vehicle with the highest score in each cluster as the cluster head vehicle through comprehensive scoring;

[0071] A model building module is configured to build a computing offloading model and a latency model according to the local computing and communication conditions of each cluster after clustering;

[0072] The problem determination module is configured to minimize the task completion time while satisfying the constraints;

[0073] The optimization module is configured to use the DDPG algorithm to solve the optimization problem and minimize the task completion time.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] (1) The present invention utilizes an improved K-means method to divide vehicle users into multiple clusters and select cluster head vehicles to perform hierarchical federated learning (HFL) to prevent a large number of vehicles from uploading model parameters at the same time, causing communication congestion. In order to speed up task completion, vehicles do not need to frequently transmit model parameters to distant base stations, reducing communication overhead.

[0076] (2) The present invention uses an intra-cluster computing offloading strategy to offload part of the task of a vehicle that completes the task slowly to other vehicles in the cluster that complete the task quickly, thereby increasing the speed at which the slowest vehicle completes the task and thereby reducing the overall task completion time.

[0077] Advantages of additional aspects of the present invention will be given in part in the description that follows, and in part will be obvious from the description that follows, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0079] Figure 1 A schematic flow chart of a vehicle federated learning method based on clustering and computation offloading provided in an embodiment of the present invention;

[0080] Figure 2 A wireless network model diagram of a vehicle federated learning method based on clustering and computation offloading provided in an embodiment of the present invention;

[0081] Figure 3 A schematic diagram of a process of clustering and selecting cluster head vehicles using an improved K-means method in a vehicle federated learning method based on clustering and computation offloading provided by an embodiment of the present invention;

[0082] Figure 4 This is the average reward convergence graph of computational offloading obtained by using the DDPG algorithm in an embodiment of the present invention;

[0083] Figure 5 A comparison diagram of task accuracy between a vehicle clustering and intra-cluster computing offloading method and a clustering-only without offloading method in an embodiment of the present invention;

[0084] Figure 6 It is a comparison chart of the task completion time of the vehicle clustering and intra-cluster computing unloading method, the clustering without unloading method and the non-clustering and non-unloading method in the embodiment of the present invention. DETAILED DESCRIPTION

[0085] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0086] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0087] See also Figure 1 , is a flow chart of a vehicle federated learning method based on clustering and computation offloading provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0088] Step S1: Construct a system network model of vehicle federated learning including multiple vehicles and a base station with a server, and describe in detail the infrastructure and equipment of the wireless network under vehicle federated learning.

[0089] See also Figure 2 , is a wireless network model diagram of a vehicle federated learning method based on clustering and computation offloading provided by an embodiment of the present invention. Figure 2 As shown in the figure, consider a wireless network system including multiple vehicle users and a base station (with a server). On a two-way two-lane road, there are N vehicles, and the vehicle collection is It indicates that these vehicles are within the coverage of the base station during the entire hierarchical federated learning (HFL) process. The vehicles can be divided into K clusters to collaboratively participate in HFL training, and the formed clusters can last for a period of time sufficient to support the entire HFL process.

[0090] The dataset of vehicle n participating in HFL training is represented as in represents the training data, yes The corresponding label, O n is the amount of data for vehicle n training. Each data sample i of vehicle n consists of training data X n,i and its corresponding label Y n,i These on-board data are generated by vehicles during daily operation and can be applied to various machine learning tasks, such as traffic sign classification, lane flow estimation, trajectory route planning, and high-definition map generation.

[0091] Step S2: Cluster the vehicles according to their relative positions, relative speeds and the Non-IID characteristics of the data, and determine the cluster head vehicle of each cluster by comprehensively considering the computing power, communication capability, relative position, relative speed and sustainability of the communication link of the vehicles.

[0092] See also Figure 3 , is a flow chart of clustering and selecting cluster head vehicles using an improved K-means method in a vehicle federated learning method based on clustering and computation offloading provided by an embodiment of the present invention. Figure 3 As shown,

[0093] Step S21: From the vehicle collection K cars are randomly selected as the initial mass points {μ1, μ2, ..., μ K}, forming the initial K intermediate clusters

[0094] Step S22: Calculate the vehicle n and each of the K intermediate clusters respectively The distance γ n,k , the calculation formula is as follows:

[0095]

[0096] in,

[0097]

[0098] Represents vehicle n and the middle cluster The average relative distance of all vehicles in |N′ k | indicates the middle cluster The number of vehicles in the n,m Represents vehicle n and the middle cluster The relative distance between vehicles m; Represents vehicle n and the middle cluster The average relative speed of all vehicles in; v n represents the speed of vehicle n; v m Represents the middle cluster The speed of vehicle m; Indicates that if vehicle n is clustered into the middle cluster Middle cluster The degree of Non-IID; c represents different categories, represents the set of all categories; p m (c) represents the proportion of category c in the data of vehicle m participating in FL training; p(c) represents the proportion of category c when the data is independent and identically distributed (IID); the calculation formula of p(c) is: C represents the number of categories; β1, β2, and β3 represent weighting coefficients.

[0099] It should be noted that in this embodiment In essence, the difference between non-IID and IID data distributions is used to represent data offset, which in turn affects the accuracy of the model. In this embodiment, the task of FL training is to perform category recognition. For example, the cifar-10 dataset (containing 60,000 color images of 10 categories, 6,000 images for each category, and the categories are airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks) is used for training, and the trained model can be used to recognize these categories.

[0100] Step S23: According to the vehicle n and each intermediate cluster in the K clusters The distance γ n,k , add vehicle n to the distance γ n,k The smallest middle cluster.

[0101] Step S24: Repeat steps S22-S23 until all vehicles are added to the cluster, and finally obtain K clusters.

[0102] Step S25: In each cluster The fraction S of each vehicle that becomes the cluster head is calculated separately n , select the score S n The largest vehicle is the cluster head vehicle.

[0103] Score S n The calculation formula is as follows:

[0104]

[0105] in,

[0106]

[0107] The ratio of the CPU frequency of vehicle n to the number of CPU cycles required to calculate a sample data of vehicle n indicates the cluster The computing power of vehicle n in f n represents the CPU frequency of local calculation of vehicle n; b n It represents the number of CPU cycles required for vehicle n to calculate a sample data; Representation Cluster The communication capability of vehicle n; Indicates the transmission power of the model parameters uploaded by vehicle n; is the vehicle n and the cluster The inverse of the average relative distance of other vehicles m in |N k | indicates a cluster The number of vehicles in; d n,m Representation Cluster The relative distance between vehicle n and vehicle m; is the vehicle n and the cluster The inverse of the average relative speed of other vehicles m; Δv n,m Representation Cluster The relative speed between vehicle n and vehicle m; Represents vehicle n and cluster The average link lifetime of other vehicles m in the LLT n,m Represents vehicle n and cluster The link lifetime of other vehicles m in the n represents the speed of vehicle n; v m represents the speed of vehicle m; TR represents the transmission range; ω1, ω2, ω3, ω4, ω5 are weight factors, and ω1, ω2, ω3, ω4, ω5∈(0,1).

[0108] Step S3: Model and analyze the wireless communication network of hierarchical federated learning (HFL) in the clustered vehicle network, and build corresponding computing offloading models and delay models for vehicle local calculations and uplinks of model parameter transmission from cluster member vehicles to cluster head vehicles and from cluster head vehicles to base stations.

[0109] Due to limited computing and communication capabilities, the latency of some vehicles participating in overall federated learning may be too long, and the performance of federated learning will be affected accordingly. Offloading the computing tasks of vehicles with slow model training to other vehicles with fast model training in the cluster can reduce the task completion time.

[0110] In cluster In the calculation, the vehicle n that needs to be unloaded selects a vehicle m in the cluster for unloading. Definition a n,m Calculate the unloading decision for the vehicle, if a n,m =1, then vehicle n (i.e., unloading vehicle) chooses to unload to vehicle m (i.e., service vehicle), otherwise a n,m = 0, it means that vehicle n does not choose to unload the calculation to vehicle m. Define α n Cluster The ratio of vehicle n (i.e., unloading vehicle) to vehicle m (i.e., service vehicle) is calculated. The amount of data O of vehicle n involved in model training n for:

[0111]

[0112] Among them, D nIndicates the size of the local data of vehicle n, a n,m represents the computation offloading decision of vehicle n to vehicle m, α n represents the ratio of vehicle n to vehicle m in terms of computational unloading, a m,n represents the computation offloading decision of vehicle m to vehicle n, α m represents the ratio of vehicle m to vehicle n in terms of computational unloading, D m Indicates the size of the local data of vehicle m.

[0113] The learning delay in the local-cluster head vehicle aggregation phase consists of three parts: computation offloading delay, local model training delay, and local model upload delay to the cluster head vehicle h k The time delay of cluster head vehicle aggregation and the time delay of model parameters after aggregation are not considered here because the time delay of cluster head vehicle aggregation and model delivery is very short. The synchronous model aggregation mechanism is adopted in both the local-cluster head vehicle aggregation stage and the cluster head vehicle-base station aggregation stage.

[0114] In cluster In, if a n,m =1, then vehicle n in the cluster needs to unload some local data to vehicle m, and the unloading amount is α n D n In this embodiment, orthogonal frequency division multiple access (OFDMA) is used for data transmission. Data unloading delay of vehicle n unloading local data to vehicle m for:

[0115]

[0116] Among them, r n,m Represents the transfer model.

[0117] Transmission Model n,m for:

[0118]

[0119] B represents the total bandwidth; N represents the number of vehicles participating in federated learning; represents the transmission power of the model parameters uploaded by vehicle n; g n,m represents the channel gain between vehicle n and vehicle m, taking into account the large-scale fading and small-scale fading of the channel between them. The path loss model for large-scale fading is 128.1+37.6log 10 d n,m , small-scale fading follows Rayleigh distribution, which indicates Rayleigh fading, d n,m represents the distance between vehicle n and vehicle m when unloading data; N0 represents the noise power density.

[0120] Cluster Computation offloading latency for:

[0121]

[0122] In each local iteration, the local model training delay of vehicle n is for:

[0123]

[0124] Where L represents the number of rounds of local training; O n represents the amount of data for vehicle n involved in model training; b n represents the number of CPU cycles required for vehicle n to calculate a sample data, b n It is determined by the learning task and the computing power of the mobile device; n Indicates the CPU frequency of local calculation of vehicle n.

[0125] In cluster The cluster member vehicle n in the cluster uses Orthogonal Frequency Division Multiple Access (OFDMA) to send the data to the cluster head vehicle h. k Upload its local training model parameters, define Z to represent the size of the model parameters, then in the jth global iteration and the ith local iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k Delay for:

[0126]

[0127] in, Indicates that in the jth global iteration and the ith local iteration, the data uploaded from vehicle n to cluster head vehicle h k The transmission model, its calculation formula is the same as the transmission model r n,m same.

[0128] In the jth global iteration, the cluster The time taken by the vehicle to participate in a total of I local iterations for:

[0129]

[0130] Where I represents the total number of local iterations, Indicates that in the first global iteration and the first local iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k The delay; I represents the total number of local iterations; Indicates that in the ith local iteration of the first global iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k delay.

[0131] The time delay of the cluster head vehicle-base station aggregation stage is mainly determined by the cluster head vehicle uploading the local aggregation model. Due to the strong computing and communication capabilities of the base station, the time delay of global model aggregation and downloading is also ignored. In the jth global iteration, the cluster head vehicle h k Delay in uploading local aggregate models for:

[0132]

[0133] Among them, h k represents the cluster head vehicle; represents the set of cluster head vehicles; Z represents the size of the model parameters; bs represents the base station; It represents the transmission model of the cluster head vehicle uploading the local aggregation model in the jth global iteration. Its calculation formula is the same as the transmission model r n,m same.

[0134] The total time T spent on the entire HFL training process total for:

[0135]

[0136] Where J represents the total number of global iterations; Represents the set of different clusters after clustering.

[0137] Step S4: Problem modeling, clarifying that the system optimization goal is to minimize the time to complete the task.

[0138] Clearly define the system optimization goal and minimize the task completion time by optimizing the offloading decision. The optimization problem can be modeled as:

[0139] P:

[0140] St

[0141] C1:

[0142] C2:

[0143] C3:

[0144] C4:

[0145] C5:

[0146] in, represents the computational unloading decision matrix for each vehicle, represents the computational unloading decision vector of each vehicle in the cluster, α = {α1, ..., α n , ..., α N} represents the calculated unloading amount vector of each vehicle.

[0147] Constraints C1 and C2 indicate that a vehicle can only select one vehicle in the cluster for calculation and unloading. Constraint C3 indicates that a vehicle can only be a vehicle between an unloading vehicle and a service vehicle to avoid two vehicles unloading each other and forming a loop. Constraint C4 indicates that a vehicle can only select a vehicle in the cluster for calculation and unloading, and cannot select a vehicle outside the cluster. Constraint C5 indicates an unloading ratio constraint.

[0148] Step S5: Use the deep deterministic policy gradient (DDPG) algorithm to optimize the offloading decision to minimize the task completion time.

[0149] Define the state space, action space and reward function, and build the DDPG model.

[0150] State space: The state set is: where s i ={D i , b i , f i , p i , v i , x i ,y i}, contains the information of all vehicle users in the entire network, namely, the data size of vehicle i, the number of CPU cycles required to process 1 bit of data, the CPU frequency of calculation, transmission power, speed and location information.

[0151] Action space: The action set is: A = {a1, a2, ..., a N}, the specific accepting vehicle action can be described as: a i ={a i,m , α i}, that is, uninstall association and uninstall ratio.

[0152] Reward function: In order to minimize the time of the entire FL training process, the reward function is defined as: r = -T total .

[0153] The specific algorithm process is:

[0154] Step 1: Initialize the critic network Q(s, a|θ Q ), actor network μ(s|θ μ ) parameter θ Q ,θ μ

[0155] Step 2: Initialize the parameters θ of the target network Q′, μ′ Q′ ←θ Q ,θ μ′ ←θ μ

[0156] Step 3: Initialize Replay Buffer R

[0157] Step 4: for episode = 1, M do

[0158] Step 5: Initialize the random process To perform action exploration, observe the initial state s1

[0159] Step 6: Select actions based on current strategy and exploration

[0160] Step 7: Execute the action And observe the reward r t and the new state s t+1

[0161] Step8: Store (s) in ReplayBufferR t , a t , r t ,s t+1 )

[0162] Step 9: Randomly select a batch of N samples (s i , a i , r i ,s i+1 )

[0163] Step 10: Set y i =r i +γQ′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ )

[0164] Step 11: Update Critic by minimizing the loss function:

[0165]

[0166] Step 12: Use the sampled policy gradient to update the Actor:

[0167]

[0168] Step 13: Update target network parameters:

[0169]

[0170] Step 14: end for

[0171] like Figure 4 As shown in FIG. 1 , it is the average reward convergence diagram of computational offloading obtained by using the DDPG algorithm in an embodiment of the present invention. Figure 5 As shown in FIG. 1 , it is a comparison chart of the task accuracy of the vehicle clustering and in-cluster computing offloading method and the clustering-only without offloading method in the embodiment of the present invention. Figure 6 As shown in FIG. 1 , it is a comparison chart of the task completion time of the vehicle clustering and in-cluster computing offloading method, the clustering without offloading method, and the non-clustering without offloading method in an embodiment of the present invention. Figures 4 to 6 It can be seen that the vehicle federated learning method based on clustering and computation offloading proposed in the embodiment of the present invention can obtain higher accuracy of federated learning and better convergence effect, alleviate the impact of the Non-IID characteristics of the data on the training accuracy, and shorten the task completion time.

[0172] Corresponding to the above embodiment, the present invention further provides a vehicle federated learning system based on clustering and computation offloading, comprising:

[0173] A data acquisition module is configured to acquire vehicle status data;

[0174] The clustering and cluster head vehicle selection module is configured to cluster the vehicles based on the improved K-means algorithm, and select the vehicle with the highest score in each cluster as the cluster head vehicle through comprehensive scoring;

[0175] The model building module is configured to establish a wireless communication network model of the HFL by acquiring vehicle status data and vehicle clustering conditions, and to build corresponding calculation offloading and delay models for vehicle local calculations and uplinks of model parameter transmission from cluster member vehicles to cluster head vehicles and from cluster head vehicles to base stations;

[0176] The problem determination module is configured to minimize the time to complete the task under the premise of satisfying the constraints that only one vehicle can be selected in the cluster for computational unloading, the vehicle can only be an unloading vehicle or a service vehicle, and the unloading ratio can only be within [0,1];

[0177] The optimization module is configured to use the DDPG algorithm to solve the optimization problem and minimize the task completion time.

[0178] Corresponding to the above embodiment, an embodiment of the present invention further provides an electronic device. The electronic device may include: a processor, a memory, and a communication unit. These components communicate through one or more buses. Those skilled in the art can understand that the electronic device structure can be a bus structure or a star structure, and can also include more or fewer components than above, or combine certain components, or arrange components differently.

[0179] The communication unit is used to establish a communication channel so that the electronic device can communicate with other devices.

[0180] The processor is the control center of the electronic device. It uses various interfaces and lines to connect the various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory, and calls the data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0181] Memory, used to store the execution instructions of the processor, the memory can be implemented by any type of volatile or non-volatile storage device or a combination of them, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0182] When the execution instructions in the memory are executed by the processor, the electronic device is enabled to execute part or all of the steps in the above method embodiments.

[0183] Corresponding to the above embodiment, the embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, wherein when the program is running, the device where the computer-readable storage medium is located may be controlled to execute some or all of the steps in the above method embodiment. In a specific implementation, the computer-readable storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0184] Corresponding to the above embodiment, an embodiment of the present invention further provides a computer program product, which includes executable instructions. When the executable instructions are executed on a computer, the computer executes part or all of the steps in the above method embodiment.

[0185] In the embodiments of the present invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. A and B may be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c may be single or multiple.

[0186] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0188] In several embodiments provided by the present invention, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0189] The above is only a specific embodiment of the present invention. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A vehicle federated learning method based on clustering and computation offloading, characterized in that: The method comprises: Construct a system network model of vehicle federated learning that includes multiple vehicles and a base station with a server; The vehicles are clustered based on the improved K-means algorithm, and the vehicle with the highest score in each cluster is selected as the cluster head vehicle through comprehensive scoring; According to the local computing and communication conditions of each cluster after clustering, a computing offloading model and a delay model are established, and the computing tasks of the vehicle with slow model training are partially offloaded to other vehicles with fast model training in the cluster through the computing offloading model; The optimization problem is to minimize the time to complete the task; Optimize offloading decisions to minimize task completion time; The method of clustering vehicles based on the improved K-means algorithm and selecting the vehicle with the highest score in each cluster as the cluster head vehicle through comprehensive scoring specifically includes: S21: Collect from vehicle K cars are randomly selected as the initial mass points {μ1, μ2, ..., μ K }, forming the initial K intermediate clusters S22: Calculate the relationship between vehicle n and each of the K intermediate clusters The distance γ n,k ; S23: According to the distance γ n,k The value of , adds vehicle n to the distance γ n,k in the smallest middle cluster; S24: Repeat S22-S23 until all vehicles are added to the cluster, and finally obtain K clusters. S25: In each cluster The fraction S of each vehicle that becomes the cluster head is calculated separately n , select the score S n The largest vehicle is the cluster head vehicle; The distance γ n,k The calculation formula is as follows: in, Represents vehicle n and the middle cluster The average relative distance of all vehicles in |N′ k | indicates the middle cluster The number of vehicles in the n,m Represents vehicle n and the middle cluster The relative distance between vehicles m; Represents vehicle n and the middle cluster The average relative speed of all vehicles in; v n represents the speed of vehicle n; v m Represents the middle cluster The speed of vehicle m; Indicates that if vehicle n is clustered into the middle cluster Middle cluster The degree of Non-IID; c represents different categories, represents the set of all categories; p m (c) represents the proportion of category c in the data of vehicle m participating in FL training; p(c) represents the proportion of category c when the data is independent and identically distributed; β1, β2 and β3 represent weighting coefficients; The score S n The calculation formula is as follows: in, f n represents the CPU frequency of local calculation of vehicle n; b n It represents the number of CPU cycles required for vehicle n to calculate a sample data; Represents the transmission power of the model parameters uploaded by vehicle n; |N k | indicates a cluster The number of vehicles in; d n,m Representation Cluster The relative distance between vehicle n and vehicle m; Δv n,m Representation Cluster The relative speed between vehicle n and vehicle m; LLT n,m Represents vehicle n and cluster The link lifetime of other vehicles m in the n represents the speed of vehicle n; v m represents the speed of vehicle m; TR represents the transmission range; ω1,ω2,ω3,ω4,ω5 are weight factors, and ω1,ω2,ω3,ω4,ω5∈(0,1).

2. The vehicle federated learning method based on clustering and computation offloading according to claim 1, characterized in that: The calculation offloading model is specifically: In cluster In the calculation, the vehicle n that needs to be unloaded selects a vehicle m in the cluster for unloading. The amount of data O of vehicle n involved in model training n for: Among them, D n Indicates the size of the local data of vehicle n; a n,m represents the decision of vehicle n to offload computation to vehicle m. If a n,m =1, indicating that vehicle n chooses to unload calculation to vehicle m. If a n,m =0, indicating that vehicle n does not choose to offload computation to vehicle m; α n represents the ratio of vehicle n to vehicle m in terms of computational unloading; a m,n Indicates that vehicle m chooses to offload computation to vehicle n. If a m,n =1, indicating that vehicle m chooses to unload the calculation to vehicle n. If a m,n =0, indicating that vehicle m does not choose to offload computation to vehicle n; α m represents the ratio of vehicle m to vehicle n in terms of computational unloading, D m Indicates the size of the local data of vehicle m.

3. The vehicle federated learning method based on clustering and computation offloading according to claim 1, characterized in that: The delay model is specifically: The total time T spent on the entire HFL training process total for: Where J represents the total number of global iterations; Represents the set of different clusters after clustering; Indicates that in the jth global iteration, cluster The time it takes for the vehicle to participate in I local iterations; represents the delay of cluster head vehicle uploading local aggregation model in the jth global iteration; The calculation formula is: in, Representation Cluster The computation offloading delay of represents the training delay of the local model of vehicle n in each local iteration; Indicates that in the first global iteration and the first local iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k The delay; I represents the total number of local iterations; Indicates that in the ith local iteration of the first global iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k Delay Indicates that in the jth global iteration and the ith local iteration, the model parameters are uploaded from vehicle n to the cluster head vehicle h k Delay The calculation formula is: Where L represents the number of rounds of local training; O n represents the amount of data for vehicle n involved in model training; b n represents the number of CPU cycles required for vehicle n to calculate a sample data; f n Indicates the CPU frequency of local calculation of vehicle n; The calculation formula is: Among them, Z represents the size of the model parameters, Indicates that in the jth global iteration and the ith local iteration, the data uploaded from vehicle n to cluster head vehicle h k The transmission model; The calculation formula is: Among them, h k represents the cluster head vehicle; represents the set of cluster head vehicles; Z represents the size of the model parameters; It represents the transmission model of the cluster head vehicle uploading the local aggregation model in the jth global iteration.

4. The vehicle federated learning method based on clustering and computation offloading according to claim 3 is characterized in that: cluster Computation offloading latency The calculation formula is: in, Representation Cluster Data unloading delay of vehicle n unloading local data to vehicle m, Among them, a n,m represents the computation offloading decision of vehicle n to vehicle m; α n represents the ratio of vehicle n to vehicle m in terms of computational unloading; D n represents the size of the local data of vehicle n; r n,m represents the transmission model, Where B represents the total bandwidth; N represents the number of vehicles participating in federated learning; represents the transmission power of the model parameters uploaded by vehicle n; g n,m represents the channel gain between vehicle n and vehicle m; N0 represents the noise power density.

5. The vehicle federated learning method based on clustering and computation offloading according to claim 3 is characterized in that: The explicit optimization problem is to minimize the time to complete the task, specifically: St C1: C2: C3: C4: C5: in, represents the computational unloading decision matrix for each vehicle; α = {α1, ..., α n , ..., α N } represents the calculated unloading amount vector of each vehicle; a n,m represents the computation offloading decision of vehicle n to vehicle m; a m,n represents the computation offloading decision of vehicle m to vehicle n; α n It indicates the ratio of vehicle n to vehicle m for computational unloading. Constraints C1 and C2 indicate that the vehicle can only select one vehicle in the cluster for computational unloading. Constraint C3 indicates that the vehicle can only be one of the unloading vehicles and the service vehicle to avoid mutual unloading and forming a loop. Constraint C4 indicates that the vehicle can only select vehicles in the cluster for computational unloading and cannot select vehicles outside the cluster. Constraint C5 indicates the unloading ratio constraint.

6. The vehicle federated learning method based on clustering and computation offloading according to claim 1, characterized in that: The optimization of the unloading decision is specifically: optimizing the unloading decision by using a deep deterministic policy gradient algorithm.

7. A vehicle federated learning system based on clustering and computation offloading, characterized in that: The system comprises: A data acquisition module is configured to acquire vehicle status data; The clustering and cluster head vehicle selection module is configured to cluster the vehicles based on the improved K-means algorithm, and select the vehicle with the highest score in each cluster as the cluster head vehicle through comprehensive scoring; The model building module is configured to establish a computing offloading model and a time delay model according to the local computing and communication conditions of each cluster after clustering, and to partially offload the computing tasks of the vehicle with slow model training to other vehicles with fast model training in the cluster through the computing offloading model; The problem determination module is configured to minimize the task completion time while satisfying the constraints; The optimization module is configured to use the DDPG algorithm to solve the optimization problem and minimize the task completion time; The method of clustering vehicles based on the improved K-means algorithm and selecting the vehicle with the highest score in each cluster as the cluster head vehicle through comprehensive scoring specifically includes: S21: Collect from vehicle K cars are randomly selected as the initial mass points {μ1, μ2, ..., μ K }, forming the initial K intermediate clusters S22: Calculate the relationship between vehicle n and each of the K intermediate clusters The distance γ n,k ; S23: According to the distance γ n,k The value of , adds vehicle n to the distance γ n,k in the smallest middle cluster; S24: Repeat S22-S23 until all vehicles are added to the cluster, and finally obtain K clusters. S25: In each cluster The fraction S of each vehicle that becomes the cluster head is calculated separately n , select the score S n The largest vehicle is the cluster head vehicle; The distance γ n,k The calculation formula is as follows: in, Represents vehicle n and the middle cluster The average relative distance of all vehicles in |N′ k | indicates the middle cluster The number of vehicles in the n,m Represents vehicle n and the middle cluster The relative distance between vehicles m; Represents vehicle n and the middle cluster The average relative speed of all vehicles in; v n represents the speed of vehicle n; v m Represents the middle cluster The speed of vehicle m; Indicates that if vehicle n is clustered into the middle cluster Middle cluster The degree of Non-IID; c represents different categories, represents the set of all categories; p m (c) represents the proportion of category c in the data of vehicle m participating in FL training; p(c) represents the proportion of category c when the data is independent and identically distributed; β1, β2 and β3 represent weighting coefficients; The score S n The calculation formula is as follows: in, f n represents the CPU frequency of local calculation of vehicle n; b n It represents the number of CPU cycles required for vehicle n to calculate a sample data; Represents the transmission power of the model parameters uploaded by vehicle n; |N k | indicates a cluster The number of vehicles in; d n,m Representation Cluster The relative distance between vehicle n and vehicle m; Δv n,m Representation Cluster The relative speed between vehicle n and vehicle m; LLT n,m Represents vehicle n and cluster The link lifetime of other vehicles m in the n represents the speed of vehicle n; v m represents the speed of vehicle m; TR represents the transmission range; ω1, ω2, ω3, ω4, ω5 are weight factors, and ω1, ω2, ω3, ω4, ω5∈(0,1).