Internet of vehicles task unloading method based on federated learning

Abstract

The invention relates to an Internet of Vehicles task unloading method based on federated learning, and belongs to the technical field of information. Aiming at the problem of how to make an optimal unloading decision in a dynamic environment of the Internet of Vehicles, a framework based on an edge intelligent system is constructed, a communication and calculation model between the Internet of Vehicles and an edge server is established, a calculation unloading strategy based on federated learning is designed, the Internet of Vehicles and the federated learning are fused, and the optimal unloading decision is made. The processing time delay of the vehicle calculation task is minimized, the calculation task can be unloaded to an edge server to be processed based on the Internet of Vehicles, simulation experiments prove that the method reduces the time delay to a certain extent, and the vehicle task unloading efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a method for unloading vehicle network tasks based on federated learning. Background Art

[0002] With the continuous advancement of information technology and wireless communications, carrying tasks to cloud network processing can no longer meet people's demand for high-quality, low-latency communication services. Edge computing transfers computing and storage resources to the edge of the core network to meet the requirements of low energy consumption and low latency for applications, and provides technical empowerment for unmanned driving. After the vehicle offloads computing tasks to the edge server, it can realize edge information processing, thereby effectively reducing energy consumption and latency.

[0003] When the computing power of the vehicle itself is limited, it is very effective to offload the computing tasks of the vehicle to the edge server so that the vehicle can use the powerful processing power of the edge server to process the task. At present, partial offloading and 0-1 offloading are two classic offloading methods. The partial offloading method means that the computing task can be split into multiple subtasks. At this time, you can choose not to offload, unload all, or unload in proportion. 0-1 offloading means that the computing task cannot be split into multiple subtasks, and you can only choose not to offload or unload all. This method is to divide the task into two subtasks, one subtask is calculated locally, and the other subtask is offloaded to the server for processing. The two subtasks can be processed at the same time.

[0004] When the vehicle encounters massive computing tasks and experimental sensitivity tasks, the computing power of the vehicle itself cannot meet the requirements. At this time, the vehicle needs to offload the computing tasks to the edge server for processing, and then transmit the computing results back to the target vehicle after processing to complete the computing offloading process. However, since the computing resources equipped by the edge server are generally limited, it cannot support the service needs of all Internet of Vehicles users at the same time. Therefore, how to reasonably plan computing resources and allocate computing power under the constraints of edge server resource limitations is still a huge challenge. Summary of the invention

[0005] The present invention provides a method for unloading tasks in an Internet of Vehicles based on federated learning, the purpose of which is to minimize the processing delay of vehicle computing tasks so that the Internet of Vehicles can obtain the best unloading decision under the minimum task processing delay.

[0006] The technical solution adopted by the present invention comprises the following steps:

[0007] (1) Establish the communication and computing model between the Internet of Vehicles and the edge server

[0008] Analyze the information transmission channel model of the Internet of Vehicles and divide the delay caused by the unloading process into transmission delay and computational latency Establish the communication model between the Internet of Vehicles and the server, and construct the total unloading delay T mn , establish optimization objectives and constraints;

[0009] (2) Introducing deep learning

[0010] The Double Deep Q Network algorithm DDQN can dynamically adapt to the Internet of Vehicles environment and has the advantages of low complexity and fast convergence speed to make task offloading decisions;

[0011] (3) Introducing the Internet of Vehicles Task Offloading Algorithm Using Federated Learning

[0012] The client-server architecture is selected. There are two types of participants in federated learning: participant vehicles and servers. According to the federated learning algorithm, all participants will be equipped with their own data sets. Participants will train local deep learning models based on private data sets and send local model parameters to the central server. After receiving the information from the participants, the central server summarizes all model parameters, globally aggregates model parameters and generates global model parameters, and then sends the global model parameters to each participant. The above process is repeated until the model converges and the unloading process is completed.

[0013] The specific method in step (1) of the present invention is as follows:

[0014] The delay caused by the offloading process mainly includes transmission delay and calculation delay. The transmission delay is divided into the time of uploading task data and the time of returning the result. Ignoring the delay of the edge server sending the calculation result to the vehicle, the channel gain of the transmission of vehicle m∈{1,2,...M} and edge server n∈{1,2,...N} is defined as:

[0015]

[0016] Where α0——channel gain reference value;

[0017] d m ——The distance between the edge server and the vehicle;

[0018] According to Shannon's formula, the data uplink transmission rate r mn , which can be expressed as:

[0019]

[0020] in:

[0021] B——channel bandwidth;

[0022] σ 2 ——Gaussian white noise during data transmission;

[0023] h m (t) – channel gain;

[0024] I mn ——Interference power between the vehicle and the edge server;

[0025] p mn ——Transmission power from vehicle to edge server;

[0026] λ m (t) — whether there is an obstruction sign between the vehicle and the edge server;

[0027] Using R m (t) represents the proportion of tasks unloaded by vehicle m to the edge server in the tth time slot, and the transmission delay of the task Expressed as:

[0028]

[0029] Where: x m (t)——the amount of tasks at time t;

[0030] r mn ——Channel uplink transmission rate;

[0031] The vehicle's computing tasks are processed by the vehicle itself or offloaded to the edge server for computing. When the computing tasks are calculated by the vehicle itself, the offloading decision a mn =0, R m (t) represents the proportion of tasks unloaded by vehicle k to the edge server in the tth time slot, c n Indicates the amount of resources required for a unit task. Represents the computing power of the vehicle, then 1-R m (t) represents the task ratio of vehicle m when performing local calculations, and the local calculation delay Expressed as:

[0032]

[0033] When tasks need to be offloaded to edge servers for computation, the offloading latency mainly depends on the offloading decision. Assuming that when p tasks are offloaded to the same server, the computing resources of each server are evenly allocated to each task, the computing capacity of the edge server is expressed as f mn , then the computational delay can be expressed as:

[0034]

[0035] When the task is offloaded to the edge server for calculation, the total delay is the sum of the transmission delay and calculation delay of the task. The total delay is:

[0036]

[0037] The optimization goal is to minimize the total delay of all tasks for all vehicles, a mn represents the unloading decision of the vehicle to the edge server, which is a binary variable, V represents the number of local computing vehicles, V∈{1,2,...M}, and the following are the optimization objectives and constraints:

[0038]

[0039] stC1:a mn ∈{0,1}

[0040] C2:

[0041] C3:λ m (t)∈{0,1}

[0042] C4:0≤R m (t)≤1.

[0043] The specific method in step (2) of the present invention is as follows:

[0044] The DDQN algorithm is used to solve the task offloading decision problem, where s t and t+1 represents the state at time t and the next time t+1, a t represents the action taken at time t, r t Indicates taking action a at time t t The reward obtained after t ,a t ) means taking action a at time t t The Q value of the network output is t After that, the Q network will output the corresponding Q value according to the state, and select the action a corresponding to the maximum Q value according to the strategy. t , after processing, we get the state and reward s at the next moment t+1 , based on the above process, experience samples are generated in chronological order, and each experience is stored in the experience pool;

[0045] The DDQN algorithm consists of two neural networks, the Q network and the Target Q. The algorithm finds the maximum value action from the Q network, and then finds the corresponding output value from the TargetQ network. The action selection parameter is θ, the current state parameter is θ′, and the action a with the maximum output value of the Q network is selected at time t. t , and get the reward, the action a with the largest Q network output value selected at t+1 t+1 :

[0046]

[0047] The DDQN-based algorithm uses Q(s t ,a t ) as the predicted value of the network, r t +γQ(s t+1 ,a t ) is used as the actual value of the network, and finally the error is back-propagated, where γ is the loss factor, and the objective function is obtained:

[0048]

[0049] The specific method in step (3) of the present invention is as follows:

[0050] Federated learning is introduced to design a task offloading decision algorithm that protects the privacy and security of user data. Assuming that N clients participate in federated learning, the federated learning problem is summarized as follows:

[0051]

[0052] where ω k represents the weight of the kth client. Assuming that the kth client has n k Training data: Local objective function F k (w) is defined as follows:

[0053]

[0054] where f(w;x k,j ) is the loss function, which is derived based on the model parameters w and the corresponding training data;

[0055] The federated average algorithm distributes the aggregated new global model to all clients at the same time. After receiving it, the client immediately performs local training to obtain the local model. In the tth round of federated learning training, the client will perform the following operations:

[0056]

[0057] η t is the federated learning rate, It is a piece of data extracted from the client training data in the tth round. Collecting these data can get a new model

[0058]

[0059] The specific steps of the federated learning algorithm to solve the task offloading problem are as follows:

[0060] Step 1: Input the initial state parameters of the client, i.e. the vehicle;

[0061] Step 2: Initialize the two neural network parameters θ and θ′ of the DDQN algorithm;

[0062] Step 3: Initialize the DDQN algorithm experience pool B;

[0063] Step 4: training round k = 1, 2, ..., K;

[0064] Step 5: Observe the state value s k ;

[0065] Step 6: Set the state value s k Input to the Q network, select a based on the Q value of the output action k ;

[0066] Step 7: Execute action a k , get the reward and the next state a k and k+1 ;

[0067] Step 8: Set the parameter (s k , a k ,s k+1 ,θ k ) into experience pool B;

[0068] Step 9: Collect a sufficient number of samples and randomly select a certain number of small batches of samples from the experience pool to train each network;

[0069] Step 10: Setup

[0070] Step 11: Error back propagation updates Q and θ;

[0071] Step 12: The target network satisfies the update θ′=θ;

[0072] Step 13: Upload the training parameters of the Q network and the target network to the central server for federation aggregation;

[0073] Step 14: The central server accepts the training parameters and uses the federated average algorithm Perform model aggregation;

[0074] Step 15: The central server aggregates and sends down the model parameters;

[0075] Repeat the above process until the model converges, and the model training is completed.

[0076] The algorithm of the present invention is divided into two parts, a server and a client. The server and each local client, i.e., the vehicle, have the same network structure model, and the model is trained together through multiple rounds of communication between the server and the client.

[0077] Server side: First, initialize the parameters. During the training process, randomly select a certain number of clients. After all clients have finished updating, transmit the local model parameters to the server. The server will integrate the local model parameters of the randomly selected clients to obtain the latest global parameters. Finally, the server distributes the latest global parameters to all clients for the next round of updates.

[0078] The vehicle end of the present invention: first, the local client performs a fixed number of updates based on the local data, and obtains new local model parameters based on the federated gradient descent update, and then transmits the updated local model parameters to the central server, and the server integrates the calculated parameters to obtain the latest global model parameters. Finally, the client receives the latest global model parameters sent by the server and performs the next round of updates.

[0079] The algorithm for aggregating parameters on the server side of the present invention is a federated average algorithm, that is, the central server initializes the model parameters and executes several rounds. Assuming that there are M clients in total, the clients selected in each round simultaneously train the model locally according to the model parameters wt of the tth round issued by the server, obtain the local model parameters and upload them back to the server, and the server aggregates the model parameters of each client in a weighted average manner according to the number of samples from each party to obtain the model of the next round. The specific process of federated learning is as follows:

[0080] 1) Vehicle selection: The edge server first selects the vehicle participating in the training according to the vehicle selection algorithm to download the current global model;

[0081] 2) Download the local model: randomly select the vehicle to participate in the training, and the edge server sends its initial model to the target vehicle;

[0082] 3) Local model training: Vehicles conduct distributed learning. Participating vehicles train models based on their own local data sets and use federated learning to calculate updates to the global model.

[0083] 4) Upload the global model: The vehicle sends its quantized local gradients to the edge server;

[0084] 5) Global aggregation: After receiving the local gradients of the vehicles, the edge server aggregates the local gradients and updates the global model.

[0085] The beneficial effect of the present invention is to address the problem of how to make the best offloading decision in the dynamic environment of the Internet of Vehicles, construct an edge intelligent system framework, establish a communication and computing model between the Internet of Vehicles and the edge server, and design a computing offloading strategy based on federated learning, which integrates the Internet of Vehicles with federated learning to minimize the processing delay of vehicle computing tasks. Based on this invention, the Internet of Vehicles can offload computing tasks to the edge server for processing. Simulation experiments have proved that this method reduces the delay to a certain extent and improves the efficiency of vehicle task offloading. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 This is a schematic diagram of the impact of the task volume on the unloading delay of the present invention. DETAILED DESCRIPTION

[0087] The following steps are involved:

[0088] (1) Establish the communication and computing model between the Internet of Vehicles and the edge server

[0089] The delay caused by the offloading process mainly includes transmission delay and calculation delay. The transmission delay is divided into the time of uploading task data and the time of returning the result. Ignoring the delay of the edge server sending the calculation result to the vehicle, the channel gain of the transmission of vehicle m∈{1,2,...M} and edge server n∈{1,2,...N} is defined as:

[0090]

[0091] Where α0——channel gain reference value;

[0092] d m ——The distance between the edge server and the vehicle;

[0093] According to Shannon's formula, the data uplink transmission rate r mn , which can be expressed as:

[0094]

[0095] in:

[0096] B——channel bandwidth;

[0097] σ 2 ——Gaussian white noise during data transmission;

[0098] h m (t) – channel gain;

[0099] I mn ——Interference power between the vehicle and the edge server;

[0100] p mn——Transmission power from vehicle to edge server;

[0101] λ m (t) — whether there is an obstruction sign between the vehicle and the edge server;

[0102] Using R m (t) represents the proportion of tasks unloaded by vehicle m to the edge server in the tth time slot, and the transmission delay of the task Expressed as:

[0103]

[0104] Where: x m (t)——the amount of tasks at time t;

[0105] r mn ——Channel uplink transmission rate;

[0106] The vehicle's computing tasks are processed by the vehicle itself or offloaded to the edge server for computing. When the computing tasks are calculated by the vehicle itself, the offloading decision a mn =0, R m (t) represents the proportion of tasks unloaded by vehicle k to the edge server in the tth time slot, c n Indicates the amount of resources required for a unit task. Represents the computing power of the vehicle, then 1-R m (t) represents the task ratio of vehicle m when performing local calculations, and the local calculation delay Expressed as:

[0107]

[0108] When tasks need to be offloaded to edge servers for computation, the offloading latency mainly depends on the offloading decision. Assuming that when p tasks are offloaded to the same server, the computing resources of each server are evenly allocated to each task, the computing capacity of the edge server is expressed as f mn , then the computational delay can be expressed as:

[0109]

[0110] When the task is offloaded to the edge server for calculation, the total delay is the sum of the transmission delay and calculation delay of the task. The total delay is:

[0111]

[0112] The optimization goal is to minimize the total delay of all tasks for all vehicles, a mnrepresents the unloading decision of the vehicle to the edge server, which is a binary variable, V represents the number of local computing vehicles, V∈{1,2,...M}, and the following are the optimization objectives and constraints:

[0113]

[0114] stC1:a mn ∈{0,1}

[0115] C2:

[0116] C3:λ m (t)∈{0,1}

[0117] C4:0≤R m (t)≤1

[0118] (2) Introducing deep learning algorithms

[0119] The DDQN algorithm is used to solve the task offloading decision problem. Double Deep Q Network (DDQN) is an improvement on the DeepQ Network (DQN) algorithm. t and t+1 represents the state at time t and the next time t+1, a t represents the action taken at time t, r t Indicates taking action a at time t t The reward obtained after t ,a t ) means taking action a at time t t The Q value of the network output is t After that, the Q network will output the corresponding Q value according to the state, and select the action a corresponding to the maximum Q value according to the strategy. t , after processing, we get the state and reward s at the next moment t+1 , based on the above process, experience samples are generated in chronological order, and each experience is stored in the experience pool;

[0120] The DDQN algorithm consists of two neural networks, the Q network and the Target Q. The algorithm finds the maximum value action from the Q network, and then finds the corresponding output value from the Target Q network. The action selection parameter is θ, the current state parameter is θ′, and the action a with the maximum output value of the Q network is selected at time t. t , and get the reward, the action a with the largest Q network output value selected at t+1 t+1 :

[0121]

[0122] The DDQN-based algorithm uses Q(s t ,a t ) as the predicted value of the network, r t +γQ(s t+1 ,a t ) is used as the actual value of the network, and finally the error is back-propagated, where γ is the loss factor, and the objective function is obtained:

[0123]

[0124] (3) Introducing the Internet of Vehicles Task Offloading Algorithm Using Federated Learning

[0125] Federated learning is introduced to design a task offloading decision algorithm that can protect the privacy and security of user data. Assuming that N clients participate in federated learning, the federated learning problem is summarized as follows:

[0126]

[0127] where ω k represents the weight of the kth client. Assuming that the kth client has n k Training data: Local objective function F k (w) is defined as follows:

[0128]

[0129] where f(w;x k,j ) is the loss function, which is derived based on the model parameters w and the corresponding training data;

[0130] The federated average algorithm distributes the aggregated new global model to all clients at the same time. After receiving it, the client immediately performs local training to obtain the local model. In the tth round of federated learning training, the client will perform the following operations:

[0131]

[0132] η t is the federated learning learning rate. It is a piece of data extracted from the client training data in the tth round. Collecting these data can get a new model

[0133]

[0134] The specific steps of the federated learning algorithm to solve the task offloading problem are as follows:

[0135] Step 1: Input the initial state parameters of the client, i.e. the vehicle;

[0136] Step 2: Initialize the two neural network parameters θ and θ′ of the DDQN algorithm;

[0137] Step 3: Initialize the DDQN algorithm experience pool B;

[0138] Step 4: training round k = 1, 2, ..., K;

[0139] Step 5: Observe the state value s k ;

[0140] Step 6: Set the state value s k Input to the Q network, select a based on the Q value of the output action k ;

[0141] Step 7: Execute action a k , get the reward and the next state a k and k+1 ;

[0142] Step 8: Set the parameter (s k , a k ,s k+1 ,θ k ) into experience pool B;

[0143] Step 9: Collect a sufficient number of samples and randomly select a certain number of small batches of samples from the experience pool to train each network;

[0144] Step 10: Setup

[0145] Step 11: Error back propagation updates Q and θ;

[0146] Step 12: The target network satisfies the update θ′=θ;

[0147] Step 13: Upload the training parameters of the Q network and the target network to the central server for federation aggregation;

[0148] Step 14: The central server accepts the training parameters and uses the federated average algorithm Perform model aggregation;

[0149] Step 15: The central server aggregates and sends down the model parameters;

[0150] Repeat the above process until the model converges, that is, the model training is completed;

[0151] The algorithm is divided into two parts: server and client. The server and each local client, i.e., the vehicle, have the same network structure model. Through multiple rounds of communication between the server and the client, the model is trained together.

[0152] Server side: First, initialize the parameters. During the training process, randomly select a certain number of clients. After all clients have finished updating, transmit the local model parameters to the server. The server will integrate the local model parameters of the randomly selected clients to obtain the latest global parameters. Finally, the server distributes the latest global parameters to all clients for the next round of updates.

[0153] Vehicle side: First, the local client updates the local data a fixed number of times, and updates it according to the federated gradient descent to obtain new local model parameters. Then, the updated local model parameters are transmitted to the central server. The server integrates the calculated parameters to obtain the latest global model parameters. Finally, the client receives the latest global model parameters sent by the server and performs the next round of updates.

[0154] The algorithm for server-side aggregation parameters is the federated average algorithm, that is, the central server initializes the model parameters and executes several rounds. Assuming that there are M clients in total, the clients selected in each round simultaneously train the model locally according to the model parameters wt of the tth round issued by the server, obtain the local model parameters and upload them back to the server. The server aggregates the model parameters of each client in a weighted average manner according to the number of samples from each party to obtain the model for the next round. The specific process of federated learning is as follows:

[0155] 1) Vehicle selection: The edge server first selects the vehicle participating in the training according to the vehicle selection algorithm to download the current global model;

[0156] 2) Download the local model: randomly select the vehicle to participate in the training, and the edge server sends its initial model to the target vehicle;

[0157] 3) Local model training: Vehicles conduct distributed learning. Participating vehicles train models based on their own local data sets and use federated learning to calculate updates to the global model.

[0158] 4) Upload the global model: The vehicle sends its quantized local gradients to the edge server;

[0159] 5) Global aggregation: After receiving the local gradients of the vehicles, the edge server aggregates the local gradients and updates the global model.

[0160] The effect of the present invention is further illustrated by simulation experiments below.

[0161] The simulation experiment uses Python 3.7 programming language and deep learning environment based on TensorFlow to complete the experimental simulation. The performance in different scenarios is verified through experiments and compared with other algorithm solutions. Introduction of experimental parameter configuration: In the vehicle-mounted edge system, it is assumed that the coverage range of the edge server is 200m, the vehicle speed is 40km / h, the reference channel gain α0=-60dB, the V2I communication bandwidth is 1MHZ, and the Gaussian white noise power σ 2 =-100dBm, the detailed parameters are shown in the following table:

[0162] Table 1 Experimental parameters

[0163]

[0164]

[0165] The algorithm proposed in this invention and two traditional task offloading algorithms are used for simulation respectively:

[0166] (1) The algorithm proposed in this invention combines federated learning and deep learning, which can effectively solve the transmission delay problem of the vehicle side in the continuous action state;

[0167] (2) Local Computing Only (LC-Only): The edge server does not process computing tasks, all of which are completed at the vehicle terminal. Local computing only is a traditional approach that only depends on the local execution capabilities of the vehicle. Local computing only does not require additional control strategies and serves as a baseline for evaluating the optimization capabilities of other offloading strategies.

[0168] (3) Edge Offloading Only (EO-Only): All computing tasks generated by the vehicle terminal are offloaded to the edge server.

[0169] Depend on Figure 1 It can be seen that all tasks of the LC-Only method are processed locally. Due to the limitation of local computing power and low data processing capability, the latency is long, which greatly increases the local load.

[0170] The EO-Only method offloads all tasks to the edge, which greatly reduces the local load. However, due to the increase in the amount of tasks, the task transmission delay increases, which inevitably increases the task offloading delay.

[0171] The algorithm proposed in the present invention provides a better task offloading strategy for task offloading, which optimizes the processing capabilities of the local and edge ends. It can be obviously concluded that its performance is better than the other two algorithms. Under the same task volume, the processing delay of the algorithm is the lowest, which proves that the algorithm proposed in the present invention has better performance.

Claims

1. A method for unloading tasks in an Internet of Vehicles based on federated learning, characterized in that: The following steps are involved: (1) Establish the communication and computing model between the Internet of Vehicles and the edge server Analyze the information transmission channel model of the Internet of Vehicles and divide the delay caused by the unloading process into transmission delay and computational latency Establish the communication model between the Internet of Vehicles and the server, and construct the total unloading delay T mn , establish optimization objectives and constraints; (2) Introducing deep learning The Double Deep Q Network algorithm DDQN can dynamically adapt to the Internet of Vehicles environment and has the advantages of low complexity and fast convergence speed to make task offloading decisions; (3) Introducing the Internet of Vehicles Task Offloading Algorithm Using Federated Learning The client-server architecture is selected. There are two types of participants in federated learning: participant vehicles and servers. According to the federated learning algorithm, all participants will be equipped with their own data sets. Participants will train local deep learning models based on private data sets and send local model parameters to the central server. After receiving the information from the participants, the central server summarizes all model parameters, globally aggregates model parameters and generates global model parameters, and then sends the global model parameters to each participant. The above process is repeated until the model converges and the unloading process is completed.

2. The method for unloading Internet of Vehicles tasks based on federated learning according to claim 1, characterized in that: The specific method in step (1) is as follows: The delay caused by the offloading process mainly includes transmission delay and calculation delay. The transmission delay is divided into the time of uploading task data and the time of returning the result. Ignoring the delay of the edge server sending the calculation result to the vehicle, the channel gain of the transmission of vehicle m∈{1,2,...M} and edge server n∈{1,2,...N} is defined as: Where α0——channel gain reference value; d m ——The distance between the edge server and the vehicle; According to Shannon's formula, the data uplink transmission rate r mn , which can be expressed as: in: B——channel bandwidth; σ 2 ——Gaussian white noise during data transmission; h m (t) – channel gain; I mn ——Interference power between the vehicle and the edge server; p mn ——Transmission power from vehicle to edge server; λ m (t) — whether there is an obstruction sign between the vehicle and the edge server; Using R m (t) represents the proportion of tasks unloaded by vehicle m to the edge server in the tth time slot, and the transmission delay of the task It is expressed as: Where: x m (t)——the amount of tasks at time t; r mn ——Channel uplink transmission rate; The vehicle's computing tasks are processed by the vehicle itself or offloaded to the edge server for computing. When the computing tasks are calculated by the vehicle itself, the offloading decision a mn =0, R m (t) represents the proportion of tasks unloaded by vehicle k to the edge server in the tth time slot, c n Indicates the amount of resources required for a unit task. represents the computing power of the vehicle, then 1-R m (t) represents the task ratio of vehicle m when performing local calculations, and the local calculation delay It is expressed as: When tasks need to be offloaded to edge servers for computation, the offloading latency mainly depends on the offloading decision. Assuming that when p tasks are offloaded to the same server, the computing resources of each server are evenly allocated to each task, the computing capacity of the edge server is expressed as f mn , then the computational delay can be expressed as: When the task is offloaded to the edge server for computation, the total delay is the sum of the task's transmission delay and computation delay. The total delay is: The optimization goal is to minimize the total delay of all tasks for all vehicles, a mn represents the unloading decision of the vehicle to the edge server, which is a binary variable, V represents the number of local computing vehicles, V∈{1, 2, ...M}, and the following are the optimization objectives and constraints: s.t.C1:a mn ∈{0,1} C3:λ m (t)∈{0,1} C4:0≤R m (t)≤1。 3. The method for unloading Internet of Vehicles tasks based on federated learning according to claim 1, characterized in that: The specific method in step (2) is as follows: The DDQN algorithm is used to solve the task offloading decision problem, where s t and t+1 represents the state at time t and the next time t+1, a t represents the action taken at time t, r t Indicates taking action a at time t t The reward obtained after t , a t ) means taking action a at time t t The Q value of the network output is t After that, the Q network will output the corresponding Q value according to the state, and select the action a corresponding to the maximum Q value according to the strategy. t , after processing, we get the state and reward s at the next moment t+1 , based on the above process, experience samples are generated in chronological order, and each experience is stored in the experience pool; The DDQN algorithm consists of two neural networks, the Q network and the Target Q. The algorithm finds the maximum value action from the Q network, and then finds the corresponding output value from the Target Q network. The action selection parameter is θ, the current state parameter is θ°, and the action a with the maximum output value of the Q network is selected at time t. t , and get the reward, the action a with the largest Q network output value selected at t+1 t+1 : The DDQN-based algorithm uses Q(s t , a t ) as the predicted value of the network, r t +γQ(s t+1 , a t ) is used as the actual value of the network, and finally the error is back-propagated, where γ is the loss factor, and the objective function is obtained:

4. The method for unloading Internet of Vehicles tasks based on federated learning according to claim 1, characterized in that: The specific method in step (3) is as follows: Federated learning is introduced to design a task offloading decision algorithm that protects the privacy and security of user data. Assuming that N clients participate in federated learning, the federated learning problem is summarized as follows: where ω k represents the weight of the kth client. Assuming that the kth client has n k Training data: Local objective function F k (w) is defined as follows: where f(w;x k，j ) is the loss function, which is derived based on the model parameters w and the corresponding training data; The federated average algorithm distributes the aggregated new global model to all clients at the same time. After receiving it, the client immediately performs local training to obtain the local model. In the tth round of federated learning training, the client will perform the following operations: η t is the federated learning rate, It is a piece of data extracted from the client training data in the tth round. Collecting these data can get a new model The specific steps of the federated learning algorithm to solve the task offloading problem are as follows: Step 1: Input the initial state parameters of the client, i.e. the vehicle; Step 2: Initialize the two neural network parameters θ and θ′ of the DDQN algorithm; Step 3: Initialize the DDQN algorithm experience pool B; Step 4: training round k = 1, 2, ..., K; Step 5: Observe the state value s k ; Step 6: Set the state value s k Input to the Q network, select a based on the Q value of the output action k ; Step 7: Execute action a k , get the reward and the next state a k and k+1 ; Step 8: Set the parameter (s k , a k ,s k+1 ,θ k ) into experience pool B; Step 9: Collect a sufficient number of samples and randomly select a certain number of small batches of samples from the experience pool to train each network; Step 10: Setup Step 11: Error back propagation updates Q and θ; Step 12: The target network satisfies the update θ°=θ; Step 13: Upload the training parameters of the Q network and the target network to the central server for federation aggregation; Step 14: The central server accepts the training parameters and uses the federated average algorithm Perform model aggregation; Step 15: The central server aggregates and sends down the model parameters; Repeat the above process until the model converges, and the model training is completed.

5. The method for offloading Internet of Vehicles tasks based on federated learning according to claim 4 is characterized in that: The algorithm is divided into two parts, server and client. The server and each local client, i.e., vehicle, have the same network structure model, and the model is trained together through multiple rounds of communication between the server and the client. Server side: First, initialize the parameters. During the training process, randomly select a certain number of clients. After all clients have finished updating, transmit the local model parameters to the server. The server will integrate the local model parameters of the randomly selected clients to obtain the latest global parameters. Finally, the server distributes the latest global parameters to all clients for the next round of updates.

6. The method for unloading Internet of Vehicles tasks based on federated learning according to claim 5, characterized in that: The vehicle side: first, the local client updates the local data a fixed number of times, updates according to the federated gradient descent, and obtains new local model parameters, and then transmits the updated local model parameters to the central server. The server integrates the calculated parameters to obtain the latest global model parameters. Finally, the client receives the latest global model parameters sent by the server and performs the next round of updates.

7. The method for unloading Internet of Vehicles tasks based on federated learning according to claim 5, characterized in that: On the server side, the algorithm for aggregating parameters is a federated average algorithm, that is, the central server initializes the model parameters and executes several rounds. Assuming that there are M clients in total, the clients selected in each round simultaneously train the model locally according to the model parameters wt of the tth round issued by the server, obtain the local model parameters and upload them back to the server. The server aggregates the model parameters of each client in a weighted average manner according to the number of samples from each party to obtain the model of the next round. The specific process of federated learning is as follows: 1) Vehicle selection: The edge server first selects the vehicle participating in the training according to the vehicle selection algorithm to download the current global model; 2) Download the local model: randomly select the vehicle to participate in the training, and the edge server sends its initial model to the target vehicle; 3) Local model training: Vehicles conduct distributed learning. Participating vehicles train models based on their own local data sets and use federated learning to calculate updates to the global model. 4) Upload the global model: The vehicle sends its quantized local gradients to the edge server; 5) Global aggregation: After receiving the local gradients of the vehicles, the edge server aggregates the local gradients and updates the global model.

Images