A method for task offloading in Internet of Vehicles based on federated learning

Abstract

The present invention relates to a method for offloading tasks in an Internet of Vehicles (IoV) based on federated learning, and belongs to the field of information technology. This invention addresses the issue of how to make optimal offloading decisions in the dynamic environment of an IoV. It constructs an edge intelligence system framework, establishes a communication and computing model between the IoV and edge servers, and designs a computation offloading strategy based on federated learning. This strategy integrates the IoV with federated learning to minimize the processing latency of vehicle computational tasks. Based on this invention, the IoV can offload computational tasks to edge servers for processing. Simulation experiments demonstrate that this method reduces latency to a certain extent and improves the efficiency of vehicle task offloading.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a method for offloading vehicle network tasks based on federated learning. Background Art

[0002] With the continuous advancement of information technology and wireless communications, simply offloading tasks to cloud networks for processing is no longer sufficient to meet the demand for high-quality, low-latency communication services. Edge computing shifts computing and storage resources to the edge of the core network, meeting the low-energy and low-latency requirements of applications and providing technical support for autonomous driving. By offloading computing tasks to edge servers, vehicles can perform information processing at the edge, effectively reducing energy consumption and latency.

[0003] When the vehicle's computing power is limited, offloading the vehicle's computing tasks to an edge server is a very effective method. The vehicle can use the edge server's powerful processing power to process the task. Currently, partial offloading and 0-1 offloading are two classic offloading methods. Partial offloading means that the computing task can be split into multiple subtasks. In this case, you can choose not to offload, offload all, or offload proportionally. 0-1 offloading means that the computing task cannot be split into multiple subtasks. You can only choose not to offload or offload all. This method divides the task into two subtasks, one subtask is calculated locally, and the other subtask is offloaded to the server for processing. Both subtasks can be processed simultaneously.

[0004] When a vehicle encounters massive computational tasks and experimentally sensitive tasks, its own computing power may not be sufficient. In this case, the vehicle needs to offload the computational tasks to an edge server for processing. After processing, the results are transmitted back to the target vehicle, completing the computational offloading process. However, edge servers are generally equipped with limited computing resources and cannot simultaneously support the service needs of all connected vehicle users. Therefore, how to rationally plan computing resources and allocate computing power within the constraints of edge server resources remains a significant challenge. Summary of the Invention

[0005] The present invention provides a method for offloading tasks in an Internet of Vehicles (IoV) based on federated learning, the purpose of which is to minimize the processing delay of vehicle computing tasks and enable the IoV to obtain the best offloading decision under the minimum task processing delay.

[0006] The technical solution adopted by the present invention comprises the following steps:

[0007] (1) Establish the communication and computing model between the Internet of Vehicles and the edge server

[0008] Analyze the information transmission channel model of the Internet of Vehicles and divide the delay caused by the unloading process into transmission delay and computational latency Establish the communication model between the Internet of Vehicles and the server, and construct the total unloading delay T mn , establish optimization objectives and constraints;

[0009] (2) Introducing deep learning

[0010] The Double Deep Q Network algorithm (DDQN) can dynamically adapt to the Internet of Vehicles environment and has the advantages of low complexity and fast convergence to make task offloading decisions.

[0011] (3) Introducing the Internet of Vehicles Task Offloading Algorithm with Federated Learning

[0012] A client-server architecture is selected. Federated learning includes two types of participants: participant vehicles and servers. According to the federated learning algorithm, all participants will be equipped with their own data sets. Participants will train local deep learning models based on private data sets and send local model parameters to the central server. After receiving the information from the participants, the central server summarizes all model parameters, globally aggregates the model parameters and generates global model parameters, and then sends the global model parameters to each participant. The above process is repeated until the model converges and the unloading process is completed.

[0013] The specific method in step (1) of the present invention is as follows:

[0014] The delay caused by the offloading process mainly includes transmission delay and computation delay. The transmission delay is further divided into the time of uploading task data and the time of transmitting the result back. The delay of the edge server sending the computation result to the vehicle is ignored. The channel gain of the transmission from vehicle m∈{1,2,...M} to edge server n∈{1,2,...N} is expressed as:

[0015]

[0016] Where α0 is the channel gain reference value;

[0017] d m —The distance between the edge server and the vehicle;

[0018] According to Shannon's formula, the data uplink transmission rate r mn , which can be expressed as:

[0019]

[0020] in:

[0021] B——channel bandwidth;

[0022] σ 2 ——Gaussian white noise during data transmission;

[0023] h m (t) — channel gain;

[0024] I mn ——Interference power between the vehicle and the edge server;

[0025] p mn ——Transmission power from vehicle to edge server;

[0026] λ m (t) — a sign of whether there is an obstruction between the vehicle and the edge server;

[0027] Using R m (t) represents the proportion of tasks unloaded by vehicle m to the edge server in the tth time slot, and the transmission delay of the task Expressed as:

[0028]

[0029] Where: x m (t)——the amount of tasks at time t;

[0030] r mn ——Channel uplink transmission rate;

[0031] The vehicle's computing tasks are processed by the vehicle itself or offloaded to the edge server for calculation. When the computing tasks are calculated by the vehicle itself, the offloading decision is a mn =0, R m (t) represents the proportion of tasks unloaded by vehicle k to the edge server in the tth time slot, c n Indicates the amount of resources required for a unit task. Represents the computing power of the vehicle, then 1-R m (t) represents the workload ratio of vehicle m when performing local calculations, and the local calculation delay Expressed as:

[0032]

[0033] When tasks need to be offloaded to edge servers for computation, the offloading delay mainly depends on the offloading decision. Assuming that when p tasks are offloaded to the same server, the computing resources of each server are evenly distributed to each task, the computing capacity of the edge server is expressed as f mn , then the computational delay can be expressed as:

[0034]

[0035] When the task is offloaded to the edge server for computation, the total delay is the sum of the task's transmission delay and computation delay. The total delay is:

[0036]

[0037] The optimization goal is to minimize the total delay of all tasks for all vehicles, a mn represents the offloading decision of a vehicle to the edge server, which is a binary variable. V represents the number of local computing vehicles, V∈{1,2,...M}. The following are the optimization objectives and constraints:

[0038]

[0039] stC1:a mn ∈{0,1}

[0040] C2:

[0041] C3:λ m (t)∈{0,1}

[0042] C4:0≤R m (t)≤1.

[0043] The specific method in step (2) of the present invention is as follows:

[0044] The DDQN algorithm is used to solve the task offloading decision problem, where s t and s t+1 Represents the state at time t and the next time t+1, a t represents the action taken at time t, r t Indicates taking action a at time t t The reward obtained after t ,a t ) means taking action a at time t t The Q value of the network output is t After that, the Q network will output the corresponding Q value according to the state, and select the action a corresponding to the maximum Q value according to the strategy. t , after processing, we get the state and reward s at the next moment t+1 , based on the above process, experience samples are generated in chronological order, and each experience is stored in the experience pool;

[0045] The DDQN algorithm consists of two neural networks, the Q network and the Target Q. The algorithm finds the maximum action from the Q network and then finds the corresponding output value from the Target Q network. The action selection parameter is θ, the current state parameter is θ′, and the action a with the maximum output value of the Q network is selected at time t. t , and get the reward, the action a with the largest Q network output value selected at t+1 t+1 :

[0046]

[0047] The DDQN-based algorithm uses Q(s t ,a t ) as the predicted value of the network, r t +γQ(s t+1 ,a t ) as the actual value of the network, and finally perform error back propagation, where γ is the loss factor, and obtain the objective function:

[0048]

[0049] The specific method in step (3) of the present invention is as follows:

[0050] Federated learning is introduced to design a task offloading decision algorithm that protects user data privacy and security. Assuming that N clients participate in federated learning, the federated learning problem can be summarized as follows:

[0051]

[0052] where ω k represents the weight of the k-th client. Assume that the k-th client has n k Training data: Local objective function F k (w) is defined as follows:

[0053]

[0054] where f(w;x k,j ) is the loss function, which is derived based on the model parameters w and the corresponding training data;

[0055] The federated averaging algorithm distributes the aggregated new global model to all clients simultaneously. After receiving it, the client immediately performs local training to obtain a local model. In the tth round of federated learning training, the client will perform the following operations:

[0056]

[0057] η t is the federated learning learning rate, It is a piece of data extracted from the client training data in the tth round. Collecting these data can get a new model

[0058]

[0059] The specific steps of the federated learning algorithm to solve the task offloading problem are as follows:

[0060] Step 1: Input the initial state parameters of the client, i.e. the vehicle;

[0061] Step 2: Initialize the two neural network parameters θ and θ′ of the DDQN algorithm;

[0062] Step 3: Initialize the DDQN algorithm experience pool B;

[0063] Step 4: training round k = 1, 2, ..., K;

[0064] Step 5: Observe the state value s k ;

[0065] Step 6: Set the state value s k Input to the Q network, select a based on the Q value of the output action k ;

[0066] Step 7: Execute action a k , get the reward and the next moment state a k and s k+1 ;

[0067] Step 8: Set the parameter (s k , a k , s k+1 ,θ k ) into experience pool B;

[0068] Step 9: Collect a sufficient number of samples and randomly extract a certain number of small batches of samples from the experience pool to train each network;

[0069] Step 10: Setup

[0070] Step 11: Error back propagation updates Q and θ;

[0071] Step 12: The target network satisfies the update θ′=θ;

[0072] Step 13: Upload the Q network and target network training parameters to the central server for federation aggregation;

[0073] Step 14: The central server accepts the training parameters and uses the federated averaging algorithm Perform model aggregation;

[0074] Step 15: The central server aggregates and sends the model parameters;

[0075] Repeat the above process until the model converges, and the model training is completed.

[0076] The algorithm of the present invention is divided into two parts: server and client. The server and each local client, i.e., vehicle, have the same network structure model. Through multiple rounds of communication between the server and the client, the model is trained together.

[0077] Server side: First, initialize the parameters. During the training process, randomly select a certain number of clients. After all clients have completed the update, the local model parameters are transmitted to the server. The server will integrate the local model parameters of the previously randomly selected clients to obtain the latest global parameters. Finally, the server distributes the latest global parameters to all clients for the next round of updates.

[0078] On the vehicle side of the present invention: first, the local client performs a fixed number of updates based on local data, and updates according to the federated gradient descent to obtain new local model parameters. The updated local model parameters are then transmitted to the central server. The server integrates the calculated parameters to obtain the latest global model parameters. Finally, the client receives the latest global model parameters sent by the server and performs the next round of updates.

[0079] On the server side of the present invention, the algorithm for aggregating parameters is a federated averaging algorithm. That is, the central server initializes the model parameters and executes several rounds. Assuming there are M clients in total, the clients selected in each round simultaneously train the model locally based on the model parameters wt of the tth round issued by the server, obtain the local model parameters and upload them back to the server. The server aggregates the model parameters of each client using a weighted average method based on the number of samples from each party to obtain the model for the next round. The specific process of federated learning is as follows:

[0080] 1) Vehicle selection: The edge server first selects the vehicles participating in the training according to the vehicle selection algorithm and downloads the current global model;

[0081] 2) Download the local model: The edge server randomly selects a vehicle to participate in the training and sends its initial model to the target vehicle;

[0082] 3) Local model training: Vehicles conduct distributed learning. Participating vehicles train models based on their own local datasets and use federated learning to calculate updates to the global model.

[0083] 4) Upload the global model: The vehicle sends its quantized local gradients to the edge server;

[0084] 5) Global aggregation: After receiving the local gradients of the vehicles, the edge server aggregates the local gradients and updates the global model.

[0085] The beneficial effect of the present invention is to address the problem of how to make the best offloading decision in the dynamic environment of the Internet of Vehicles, build an edge intelligent system framework, establish a communication and computing model between the Internet of Vehicles and the edge server, and design a computing offloading strategy based on federated learning, which integrates the Internet of Vehicles with federated learning to minimize the processing delay of vehicle computing tasks. Based on this invention, the Internet of Vehicles can offload computing tasks to the edge server for processing. Simulation experiments have proved that this method reduces the delay to a certain extent and improves the efficiency of vehicle task offloading. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 This is a schematic diagram of the impact of the task volume on the unloading delay of the present invention. DETAILED DESCRIPTION

[0087] The following steps are involved:

[0088] (1) Establish the communication and computing model between the Internet of Vehicles and the edge server

[0089] The delay caused by the offloading process mainly includes transmission delay and computation delay. The transmission delay is further divided into the time of uploading task data and the time of transmitting the result back. The delay of the edge server sending the computation result to the vehicle is ignored. The channel gain of the transmission from vehicle m∈{1,2,...M} to edge server n∈{1,2,...N} is expressed as:

[0090]

[0091] Where α0 is the channel gain reference value;

[0092] d m —The distance between the edge server and the vehicle;

[0093] According to Shannon's formula, the data uplink transmission rate r mn , which can be expressed as:

[0094]

[0095] in:

[0096] B——channel bandwidth;

[0097] σ 2 ——Gaussian white noise during data transmission;

[0098] h m (t) — channel gain;

[0099] I mn ——Interference power between the vehicle and the edge server;

[0100] p mn——Transmission power from vehicle to edge server;

[0101] λ m (t) — a sign of whether there is an obstruction between the vehicle and the edge server;

[0102] Using R m (t) represents the proportion of tasks unloaded by vehicle m to the edge server in the tth time slot, and the transmission delay of the task Expressed as:

[0103]

[0104] Where: x m (t)——the amount of tasks at time t;

[0105] r mn ——Channel uplink transmission rate;

[0106] The vehicle's computing tasks are processed by the vehicle itself or offloaded to the edge server for calculation. When the computing tasks are calculated by the vehicle itself, the offloading decision is a mn =0, R m (t) represents the proportion of tasks unloaded by vehicle k to the edge server in the tth time slot, c n Indicates the amount of resources required for a unit task. Represents the computing power of the vehicle, then 1-R m (t) represents the workload ratio of vehicle m when performing local calculations, and the local calculation delay Expressed as:

[0107]

[0108] When tasks need to be offloaded to edge servers for computation, the offloading delay mainly depends on the offloading decision. Assuming that when p tasks are offloaded to the same server, the computing resources of each server are evenly distributed to each task, the computing capacity of the edge server is expressed as f mn , then the computational delay can be expressed as:

[0109]

[0110] When the task is offloaded to the edge server for computation, the total delay is the sum of the task's transmission delay and computation delay. The total delay is:

[0111]

[0112] The optimization goal is to minimize the total delay of all tasks for all vehicles, a mnrepresents the offloading decision of a vehicle to the edge server, which is a binary variable. V represents the number of local computing vehicles, V∈{1,2,...M}. The following are the optimization objectives and constraints:

[0113]

[0114] stC1:a mn ∈{0,1}

[0115] C2:

[0116] C3:λ m (t)∈{0,1}

[0117] C4:0≤R m (t)≤1

[0118] (2) Introducing deep learning algorithms

[0119] The DDQN algorithm is used to solve the task offloading decision problem. Double Deep Q Network (DDQN) is an improvement on the DeepQ Network (DQN) algorithm. t and s t+1 Represents the state at time t and the next time t+1, a t represents the action taken at time t, r t Indicates taking action a at time t t The reward obtained after t ,a t ) means taking action a at time t t The Q value of the network output is t After that, the Q network will output the corresponding Q value according to the state, and select the action a corresponding to the maximum Q value according to the strategy. t , after processing, we get the state and reward s at the next moment t+1 , based on the above process, experience samples are generated in chronological order, and each experience is stored in the experience pool;

[0120] The DDQN algorithm consists of two neural networks, the Q network and the Target Q. The algorithm finds the maximum action from the Q network and then finds the corresponding output value from the Target Q network. The action selection parameter is θ, the current state parameter is θ′, and the action a with the maximum output value of the Q network is selected at time t. t , and get the reward, the action a with the largest Q network output value selected at t+1 t+1 :

[0121]

[0122] The DDQN-based algorithm uses Q(s t ,a t ) as the predicted value of the network, r t +γQ(s t+1 ,a t ) as the actual value of the network, and finally perform error back propagation, where γ is the loss factor, and obtain the objective function:

[0123]

[0124] (3) Introducing the Internet of Vehicles Task Offloading Algorithm with Federated Learning

[0125] Federated learning is introduced to design a task offloading decision algorithm that can protect user data privacy and security. Assuming that N clients participate in federated learning, the federated learning problem can be summarized as follows:

[0126]

[0127] where ω k represents the weight of the k-th client. Assume that the k-th client has n k Training data: Local objective function F k (w) is defined as follows:

[0128]

[0129] where f(w;x k,j ) is the loss function, which is derived based on the model parameters w and the corresponding training data;

[0130] The federated averaging algorithm distributes the aggregated new global model to all clients simultaneously. After receiving it, the client immediately performs local training to obtain a local model. In the tth round of federated learning training, the client will perform the following operations:

[0131]

[0132] η t is the federated learning learning rate. It is a piece of data extracted from the client training data in the tth round. Collecting these data can get a new model

[0133]

[0134] The specific steps of the federated learning algorithm to solve the task offloading problem are as follows:

[0135] Step 1: Input the initial state parameters of the client, i.e. the vehicle;

[0136] Step 2: Initialize the two neural network parameters θ and θ′ of the DDQN algorithm;

[0137] Step 3: Initialize the DDQN algorithm experience pool B;

[0138] Step 4: training round k = 1, 2, ..., K;

[0139] Step 5: Observe the state value s k ;

[0140] Step 6: Set the state value s k Input to the Q network, select a based on the Q value of the output action k ;

[0141] Step 7: Execute action a k , get the reward and the next moment state a k and s k+1 ;

[0142] Step 8: Set the parameter (s k , a k , s k+1 ,θ k ) into experience pool B;

[0143] Step 9: Collect a sufficient number of samples and randomly extract a certain number of small batches of samples from the experience pool to train each network;

[0144] Step 10: Setup

[0145] Step 11: Error back propagation updates Q and θ;

[0146] Step 12: The target network satisfies the update θ′=θ;

[0147] Step 13: Upload the Q network and target network training parameters to the central server for federation aggregation;

[0148] Step 14: The central server accepts the training parameters and uses the federated averaging algorithm Perform model aggregation;

[0149] Step 15: The central server aggregates and sends the model parameters;

[0150] Repeat the above process until the model converges, and the model training is completed;

[0151] The algorithm is divided into two parts: server and client. The server and each local client (i.e., vehicle) have the same network structure model. Through multiple rounds of communication between the server and client, the model is jointly trained.

[0152] Server: First, initialize the parameters. During the training process, randomly select a certain number of clients. After all clients have completed their updates, transmit the local model parameters to the server. The server will integrate the local model parameters of the randomly selected clients to obtain the latest global parameters. Finally, the server distributes the latest global parameters to all clients for the next round of updates.

[0153] On the vehicle side, the local client first updates the local data a fixed number of times, using federated gradient descent to obtain new local model parameters. The updated local model parameters are then transmitted to the central server, which integrates the calculated parameters to obtain the latest global model parameters. Finally, the client receives the latest global model parameters from the server and performs the next round of updates.

[0154] The algorithm for server-side parameter aggregation is the federated averaging algorithm. That is, the central server initializes the model parameters and executes several rounds. Assuming there are M clients, the clients selected in each round simultaneously train the model locally based on the model parameters wt of the tth round issued by the server, obtain the local model parameters and upload them back to the server. The server aggregates the model parameters of each client using a weighted average method based on the number of samples from each party to obtain the model for the next round. The specific process of federated learning is as follows:

[0155] 1) Vehicle selection: The edge server first selects the vehicles participating in the training according to the vehicle selection algorithm and downloads the current global model;

[0156] 2) Download the local model: The edge server randomly selects a vehicle to participate in the training and sends its initial model to the target vehicle;

[0157] 3) Local model training: Vehicles conduct distributed learning. Participating vehicles train models based on their own local datasets and use federated learning to calculate updates to the global model.

[0158] 4) Upload the global model: The vehicle sends its quantized local gradients to the edge server;

[0159] 5) Global aggregation: After receiving the local gradients of the vehicles, the edge server aggregates the local gradients and updates the global model.

[0160] The effects of the present invention are further illustrated below through simulation experiments.

[0161] The simulation experiments were conducted using the Python 3.7 programming language and a deep learning environment based on TensorFlow. The performance of the proposed algorithm in different scenarios was verified and compared with other algorithm solutions. The experimental parameters were as follows: In the vehicle-mounted edge system, the coverage range of the edge server was 200m, the vehicle speed was 40km / h, the reference channel gain α0 = -60dB, the V2I communication bandwidth was 1MHz, and the Gaussian white noise power σ 2 =-100dBm, detailed parameters are shown in the following table:

[0162] Table 1 Experimental parameters

[0163]

[0164]

[0165] The algorithm proposed in this paper and two traditional task offloading algorithms are used for simulation respectively:

[0166] (1) The algorithm proposed in this paper combines federated learning and deep learning, which can effectively solve the transmission delay problem of the vehicle side in the continuous action state;

[0167] (2) Local Computing Only (LC-Only): The edge server does not process computing tasks, and all tasks are completed at the vehicle terminal. Local computing only is a traditional approach that depends only on the local execution capability of the vehicle. Local computing only does not require additional control strategies and serves as a baseline for evaluating the optimization capabilities of other offloading strategies.

[0168] (3) Edge Offloading Only (EO-Only): All computing tasks generated by the vehicle terminal are offloaded to the edge server.

[0169] Depend on Figure 1 It can be seen that: All tasks of the LC-Only method are processed locally. Due to the limitations of local computing power and low data processing capabilities, the latency is long, which greatly increases the local load.

[0170] The EO-Only method offloads all tasks to the edge, which greatly reduces the local load. However, due to the increase in task volume, the task transmission delay increases, which inevitably increases the task offloading delay.

[0171] The algorithm proposed in the present invention provides a better task offloading strategy for task offloading, which optimizes the processing capabilities of the local and edge ends. It can be clearly concluded that its performance is better than the other two algorithms. Under the same task volume, the processing delay of this algorithm is the lowest, proving that the algorithm proposed in the present invention has better performance.

Claims

1. A method for offloading tasks in an Internet of Vehicles (IoV) based on federated learning, characterized in that: The following steps are involved: (1) Establish the communication and computing model between the Internet of Vehicles and the edge server; Analyze the information transmission channel model of the Internet of Vehicles and divide the delay caused by the unloading process into transmission delay and computational latency , establish the communication model between the Internet of Vehicles and the server, and construct the total unloading delay , establish optimization objectives and constraints; The specific method is as follows: The delay caused by the offloading process includes transmission delay and calculation delay. The transmission delay is divided into the time of uploading task data and the time of returning the result. The delay of sending the calculation result from the edge server to the vehicle is ignored. The vehicle is defined as , edge server The channel gain of transmission is expressed as: ； in ——Channel gain reference value; —The distance between the edge server and the vehicle; According to Shannon's formula, the data uplink transmission rate , expressed as: ； in: — channel bandwidth; ——Gaussian white noise during data transmission; ——channel gain; ——Interference power between the vehicle and the edge server; ——Transmission power from vehicle to edge server; ——Whether there is an obstruction sign between the vehicle and the edge server; use Indicates the Vehicles in time slot The proportion of tasks offloaded to edge servers and the transmission delay of tasks Expressed as: ； in: —— Time workload; ——Channel uplink transmission rate; The vehicle's computing tasks are processed by the vehicle itself or offloaded to the edge server for calculation. When the computing tasks are calculated by the vehicle itself, the offloading decision is made. , Indicates the Vehicles in time slot The proportion of tasks offloaded to edge servers, Indicates the amount of resources required for a unit task. represents the computing power of the vehicle, then Indicates vehicle The proportion of tasks performed locally and the local computing latency Expressed as: ； When the task needs to be offloaded to the edge server for computation, the offloading delay depends on the offloading decision. When tasks are offloaded to the same server, the computing resources of each server are evenly distributed to each task. The computing capacity of the edge server is expressed as , then the calculation delay is expressed as: ； When the task is offloaded to the edge server for computation, the total delay is the sum of the task's transmission delay and computation delay. The total delay is: ； The optimization goal is to minimize the total delay of all tasks for all vehicles. represents the unloading decision of the vehicle to the edge server, which is a binary variable. Indicates the number of local calculated vehicles, , the optimization objectives and constraints are as follows: ； ； ； ； ； (2) Introducing deep learning; The Double Deep Q Network algorithm (DDQN) can dynamically adapt to the Internet of Vehicles environment and has the advantages of low complexity and fast convergence to make task offloading decisions. (3) Introducing the Internet of Vehicles task offloading algorithm based on federated learning; A client-server architecture is chosen. Federated learning involves two types of participants: participant vehicles and servers. According to the federated learning algorithm, all participants are equipped with their own datasets. Participants train local deep learning models based on their private datasets and send the local model parameters to the central server. After receiving the participants' information, the central server aggregates all model parameters, generates global model parameters, and then distributes the global model parameters to each participant. This process is repeated until the model converges and the offloading process is complete. The specific method is as follows: Federated learning is introduced to design a task offloading decision algorithm that protects user data privacy and security. Assuming that there are 𝑁 clients participating in federated learning, the federated learning problem can be summarized as follows: ； ， ； in Indicates the The weight of the client, assuming Clients have Training data: , local objective function The definition of is as follows: ； in is the loss function, which is based on the model parameters And the corresponding training data; The federated averaging algorithm distributes the aggregated new global model to all clients at the same time. After receiving it, the client immediately performs local training to obtain the local model. During a round of federated learning training, the client will perform the following operations: ； is the federated learning learning rate, It is A piece of data extracted from the client training data is collected to get a new model : ； The specific steps of the federated learning algorithm to solve the task offloading problem are as follows: Step 1: Input the initial state parameters of the client, that is, the vehicle; Step 2: Initialization Algorithm two neural network parameters and ; Step 3: Initialization Algorithm experience pool B; Step 4: training round k=1, 2, ..., K; Step 5: Observe the state value ; Step 6: Set the status value Input to Network, based on output action Value Selection ; Step 7: Execute the action , get the reward and the next moment state and ; Step 8: Set the parameters ( , ) into experience pool B; Step 9: Collect a sufficient number of samples and randomly extract a certain number of small batches of samples from the experience pool to train each network; Step 10: Setup ; Step 11: Error back propagation update and ; Step 12: Target network meets update requirements ; Step 13: Upload the Q network and target network training parameters to the central server for federation aggregation; Step 14: The central server accepts the training parameters and uses the federated averaging algorithm Perform model aggregation; Step 15: The central server sends the model parameters after aggregation; Repeat the above process until the model converges, and the model training is completed.

2. The method for offloading tasks in an Internet of Vehicles based on federated learning according to claim 1, characterized in that: The specific method in step (2) is as follows: use The algorithm solves the task offloading decision problem, where and Indicates time and the next moment The state of Indicates time Actions taken, Indicates time Take action After receiving the reward, Indicates time Take action After the network output Value, when the input state back, The network will output the corresponding value, select the maximum value based on the strategy Action corresponding to the value , after processing, we get the state and reward of the next moment , based on the above process, experience samples are generated in chronological order, and each experience is stored in the experience pool; Algorithm includes Network and Two neural networks, the algorithm starts from Find the maximum action in the network, and then The network finds the corresponding output value, and the action selection parameter is , the current state parameters are ,exist Time selection The action with the largest network output value , and receive rewards, in Selected The action with the largest network output value : ； based on The algorithm is based on As the predicted value of the network, As the actual value of the network, the error is finally back-propagated, where As the loss factor, the objective function is obtained: 。 3. The method for offloading tasks in an Internet of Vehicles (IoV) based on federated learning according to claim 1, characterized in that: The algorithm is divided into two parts: server and client. The server and each local client, i.e., the vehicle, have the same network structure model. Through multiple rounds of communication between the server and the client, the model is trained together. Server side: First, initialize the parameters. During the training process, randomly select a certain number of clients. After all clients have completed the update, the local model parameters are transmitted to the server. The server will integrate the local model parameters of the previously randomly selected clients to obtain the latest global parameters. Finally, the server distributes the latest global parameters to all clients for the next round of updates.

4. The method for offloading tasks in an Internet of Vehicles based on federated learning according to claim 3, characterized in that: The client: First, the local client updates the local data a fixed number of times, and updates according to the federated gradient descent to obtain new local model parameters. The updated local model parameters are then transmitted to the central server. The server integrates the calculated parameters to obtain the latest global model parameters. Finally, the client receives the latest global model parameters sent by the server and performs the next round of updates.

5. The method for offloading tasks in an Internet of Vehicles based on federated learning according to claim 3, characterized in that: The algorithm for aggregating parameters on the server side is the federated average algorithm, that is, the central server initializes the model parameters and executes several rounds. Assuming that there are Clients, each round of selected clients are based on the first Wheel model parameters The model is trained locally, the local model parameters are obtained and uploaded back to the server. The server aggregates the model parameters of each client using a weighted average method based on the number of samples from each party to obtain the next round of models. The specific process of federated learning is as follows: 1) Vehicle selection: The edge server first selects the vehicles participating in the training according to the vehicle selection algorithm and downloads the current global model; 2) Downloading the local model: The edge server randomly selects a vehicle to participate in the training and sends its initial model to the target vehicle; 3) Local model training: Vehicles conduct distributed learning. Participating vehicles train models based on their own local datasets and use federated learning to calculate updates to the global model. 4) Upload the global model: The vehicle sends its quantized local gradients to the edge server; 5) Global aggregation: After receiving the local gradients of the vehicles, the edge server aggregates the local gradients and updates the global model.

Images