A credit-prioritized task offloading method for vehicle networks based on federated learning

Abstract

The present invention discloses a method for unloading tasks in a vehicle network with credit priority based on federated learning, and the steps include: 1) EC and CV send their own location information and available resource information to CC respectively; wherein EC is an edge cloud server, CV is a vehicle in a vehicle network, and CC is a cloud center; 2) when CV needs to unload a task, it sends the unloading task information to DC; wherein DC is a decision center in the vehicle network; 3) for a current batch of unloading tasks to be processed, DC obtains the credit value of each CV that sends the unloading task from CC; then sets the processing priority of each unloading task according to the credit value, and preferentially formulates task unloading decisions for unloading tasks with high processing priority and sends them to the corresponding EC and CV; wherein CC calculates the credit value of the corresponding CV according to the behavior information of CV. The present invention uses the credit value in task unloading decision-making, thereby improving decision-making performance.

Description

Technical Field

[0001] The present invention belongs to the field of Internet of Vehicles and privacy security, and specifically relates to a method for unloading credit-prioritized tasks in an in-vehicle network based on federated learning. Background Art

[0002] With the development of the Internet of Vehicles, more vehicles can be connected to more computing resources through the network, and these vehicles can achieve short-distance, low-latency communication through the Internet of Vehicles. In the Internet of Vehicles environment, individual connected vehicles are limited by computing resources and cannot independently complete the corresponding computing tasks. For example, tasks such as autonomous driving, augmented reality, and real-time data analysis. At the same time, since the cloud computing model is limited by network distance, it cannot provide low-latency service response for applications. In order to improve the service quality of a single connected vehicle application, a vehicle edge computing system (VEC) can be formed through vehicles, roadside units, and base stations, so as to fully improve the idle computing resources of the Internet of Vehicles and improve the service quality of vehicle applications. However, in the distributed Internet of Vehicles environment, each vehicle is a separate individual, and due to different reasons, vehicles have bad behaviors; these bad behaviors will affect the overall efficiency of the vehicle edge computing system. Therefore, in the presence of bad behaviors, how to complete the task offloading of vehicle edge computing is an urgent problem to be solved in the Internet of Vehicles. Summary of the invention

[0003] In view of the technical problems existing in the prior art, the present invention proposes a task offloading method based on federated learning and in-vehicle network credit priority, which calculates the reputation value of vehicle behavior by making full use of the information of the completion of task offloading execution, and then uses the reputation value in task offloading decision, thereby improving the decision-making performance.

[0004] The technical solution of the present invention is as follows:

[0005] A method for unloading credit-prioritized tasks in a vehicle network based on federated learning, the steps of which include:

[0006] 1) EC and CV send their location information and available resource information to CC respectively; EC is the edge cloud server, CV is the vehicle in the vehicle network, and CC is the cloud center;

[0007] 2) When CV needs to offload tasks, it sends the offload tasks to DC, where DC is the decision center in the vehicle network;

[0008] 3) For the current batch of unloading tasks to be processed, DC obtains the credit value of each CV that sends the unloading task from CC; then sets the processing priority of each unloading task according to the credit value, and makes task unloading decisions for unloading tasks with high processing priority and sends them to the corresponding EC and CV; wherein CC calculates the credit value of the corresponding CV according to the behavior information of CV.

[0009] Optionally, the method for CC to calculate the reputation value of the corresponding CV based on the behavior information of CV is as follows: first, six features are calculated based on the behavior information of each CV in the training data set: offset feature Transmission offset feature Data offset characteristics Workload offset feature Duration offset feature and vehicle flow data features; then train the MLP model according to the features corresponding to each CV to obtain the reputation value model; then input the six features of the target CV into the reputation value model to obtain the reputation value of the target CV.

[0010] Optionally, DC uses branch and bound method to process offload tasks with the same priority: To solve; among them, represents the reduction rate of task delay, represents the computing time of edge computing, c i is the credit value of the CV corresponding to the i-th offloading task, n is the total number of offloading tasks with the same priority, is the average task amount; the solution method is: for each offloading task, in the mth round of resource allocation, a constant resource mδ is allocated to the offloading task from an EC to obtain a corresponding solution space s; if the boundary value of the current solution s is less than the minimum value of the existing solution, then the m+1th round of resource allocation is performed, and a constant resource (m+1)δ is allocated to the offloading task from an EC, and the solution space s is updated until the boundary value of s is less than the minimum value of the existing solution; the resource allocation stage should meet the constraints: each offloading task can only be offloaded to at most one EC and the resources used to allocate by the EC are not greater than the resources it owns.

[0011] Optionally, the EC periodically obtains the latest reputation model from the CC, and then trains the reputation model using data obtained by completing the offloading task locally and uploads it to the CC; then the CC aggregates the received reputation model using a federated learning algorithm to update the reputation model.

[0012] A server, characterized in that it comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step in the above method.

[0013] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the above method when executed by a processor.

[0014] The present invention proposes a reputation value calculation method based on federated learning. The method is based on a vehicle-mounted edge computing system, and through distributed training of a multi-layer neural network, the training of the neural network model is completed without uploading the user's original data, and a reputation value model is obtained to protect the user's privacy and security. The reputation value model calculates the user's reputation value based on the vehicle behavior.

[0015] The present invention designs a task offloading decision algorithm based on reputation value, which takes the reputation value into consideration and prioritizes the tasks with high reputation value to obtain better resources. At the same time, the decision result satisfies the service quality through the optimization of the heuristic algorithm. Simulation experiments verify that our system can well complete the task offloading decision under the vehicle network.

[0016] The entire algorithm architecture is as follows Figure 1 As shown in the figure, the VEC architecture diagram based on software-defined networks is described. It mainly includes three levels: cloud center (CC), edge cloud (EC), and connected vehicles (CV). In VEC, the system has a CC to centrally handle computing tasks that the EC cannot complete independently. CC is also the central service in the federated learning architecture. Roadside units (RSUs) can act as ECs and deploy edge computing services. In addition, vehicles on the road can also become ECs. As user roles, vehicles offload their tasks to the VEC system, and they obtain the calculation results. The base station acts as a decision center (DC) and is responsible for making decisions on task offloading within its coverage area.

[0017] The present invention designs a credit-prioritized task offloading method for vehicle networks based on federated learning. By making full use of the information on task offloading completion, the vehicle behavior reputation value is calculated, and then the reputation value is used in task offloading decision-making, thereby improving decision-making performance. The method has the following advantages and effects:

[0018] 1. Privacy protection capability. In the training process of the reputation value model, the invention is based on the method of federated learning. In the process of predictive model training, the client holding the data, that is, the EC, trains the model locally and uploads the trained model to the central cloud side. The central cloud side obtains a global predictive model through the federated learning model aggregation.

[0019] 2. Reliable unloading performance. The invention determines the size of the vehicle application's ability to obtain resources by calculating the reputation value of the vehicle's behavior. This method prevents some vehicles with weak bad behaviors from being directly denied service. The reputation value is updateable, and the continuously updated reputation value enables the vehicle's reputation value to better reflect whether the vehicle's behavior is bad.

[0020] 3. The present invention can efficiently realize the task offloading of vehicle-mounted applications. Through the heuristic task offloading algorithm, the decision center can make decision tasks with delay optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a model diagram of the software-defined vehicle edge computing platform (VEC).

[0022] Figure 2 Schematic diagram of the execution flow of the vehicle-mounted task in the method of the present invention.

[0023] Figure 3 Schematic diagram of the federated learning training process designed in the method of the present invention.

[0024] Figure 4 Flow chart of the implementation process of the method of the present invention

[0025] Figure 5 The accuracy of the test under different numbers of communication rounds.

[0026] Figure 6 The accuracy change diagram of training under different communication rounds.

[0027] Figure 7 Accuracy tested for different numbers of clients.

[0028] Figure 8 Accuracy variation for training with different numbers of clients.

[0029] Fig. 9 The communication time on the Internet during training with different numbers of nodes.

[0030] Fig.10 A comparison chart of offloading algorithms under poor location information.

[0031] Fig.11 A comparison chart of offloading algorithms under poor task information. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below through specific embodiments and drawings.

[0033] In the proposed model, each task N from the vehicle in VEC can be divided into multiple serial subtasks. We use ω, I, and δ to represent the workload, data volume, and CPU resources, respectively. The training process is as follows Figure 2 As shown in , task N consists of a series of subtasks on the vehicle side, which has several parameters, such as workload ω, data volume I, and task workload offloading rate ρ, where ρ represents the percentage of the workload of task N executed by EC. λ1(N) and λ2(N) represent the data compression ratio after CV and EC calculation processing, respectively. In addition, Represents task N in CV i The resources, δ(N,M) are allocated by EC, o represents the offloading decision, 1 represents offloading, and 0 represents not offloading.

[0034] The implementation process of the present invention is divided into five parts. Figure 4 The implementation of each part is described in detail as follows.

[0035] 1. Data reporting

[0036] EC and CV report information to CC, they report information related to location and information related to available resources.

[0037] 2. Decision Making

[0038] DC obtains the reputation information of CV from CC, and then DC will make specific task offloading decisions and return this decision to CV and EC.

[0039] To solve the following delay optimization problem P1, we adopt an approximate optimization method. First, we transform the overall optimization into a task delay problem in several optimization sets by credit priority. Second, we use a heuristic algorithm, namely the branch and bound method, to solve the optimization problem. For tasks N with the same n credit priorities, i , we address the following problems:

[0040]

[0041] in represents the reduction rate of task delay, that is, the ratio of the delay of edge computing to that of fully local computing, where represents the computation time of edge computing. We designed a credit-priority-based task offloading decision algorithm based on the branch and bound method. The algorithm aims to find task offloading and resource allocation decisions when CV has a task offloading request to EC. The system efficiency here refers to the overall weighted delay reduction of tasks as described in the previous section. In order to make credit-based task offloading decisions, the task offloading algorithm needs to consider a new parameter c i ,This parameter is the reputation value of the CV corresponding to the i-th unloading task. In DC, when a batch of tasks need to be decided, the information of these tasks and the reputation value of the vehicle need to be obtained first. Then, by classifying the credit value from large to small, the tasks are divided into three priorities, which respectively represent no bad behavior, weak bad behavior and bad behavior last time. Then, DC calls the task unloading algorithm in sequence. After the highest priority task unloading decision is made, the remaining resources are scheduled for the second time, and the final scheduling is for low-priority tasks. The general algorithm flow is described as follows.

[0042] The algorithm solves two problems, namely EC selection and resource allocation during task offloading. The branch boundary uses a depth-first traversal method to find the optimal solution. In the resource allocation stage, the method uses a progressive allocation method. During the search process, the solution space is represented as s, and the i-th task is represented as N i In VEC, there is task N i CV has resources In order to N i In the first round of resource allocation, the branch and bound algorithm starts from M j N i Allocate constant resources δ(N i(c) ), and in the next round N i Assign 2δ(N i(c) ), M j is the jth edge server EC. The resource allocation phase should satisfy the constraints: each task can only be offloaded to at most one edge server and the resources allocated to the edge server are not greater than the resources it has. i ECM allocation completed j After the resources are found, if the boundary value of the current solution is less than the minimum value of the existing solution searched by the branch and bound method, the branch and constraint algorithm will perform a depth-first traversal, that is, the algorithm is likely to find the optimal solution from the current solution s. The branch and bound method is to traverse the depth of the search tree, and obtain multiple solution spaces during the search process. The solution that has been found in the solution space (complete the decision on offloading all tasks) is the existing solution.

[0043] 3. Task Execution

[0044] DC distributes the task offloading decision to the specific EC and vehicle CV. Then the VEC system performs task offloading and specific task execution according to the decision.

[0045] 4. Update reputation value

[0046] The invention designs a joint learning mechanism to train the prediction model to calculate the credit value. Each DC uses a multi-layer perceptron (MLP) algorithm in a distributed learning environment. We construct the following features based on the parameters in Table 1.

[0047] (1) Calculate the offset. It is the relative error between the predicted execution delay and the actual execution delay of each task, that is, x1:

[0048] (2) Transmission offset. This is the relative error between the predicted data transmission time and the actual transmission time of each task, i.e., x2:

[0049] (3) Data offset. It is the relative error between the amount of data reported by CV and the amount of data actually transmitted, i.e. x3:

[0050] (4) Workload offset. It is the relative error between the reported workload and the total workload actually used, i.e. x4:

[0051] (5) Duration offset. It represents the relative error between the predicted unloading duration and the actual duration, i.e., x5:

[0052] (6) Vehicle flow data feature. It represents the vehicle flow feature at location p, and its value is proportional to the average vehicle flow speed v. We mark this feature as x6.

[0053] Table 1: Task offloading completion parameter table

[0054]

[0055]

[0056] We designed an MLP algorithm to calculate the credit value, which is a model with 2 hidden layers, the first hidden layer has 8 neural units, the second hidden layer has 32 neural units, and uses ReLu activation. We used Google's Federated Averaging (FedAvg) learning algorithm to federate the training of the MLP model. Each EC has a training dataset, which is obtained by the locally offloaded tasks. The centralized node CC is responsible for the aggregation of the model. During the model training process, the EC obtains the latest model from the centralized node CC, and then uploads the trained model parameters to the central node CC. The centralized node CC then updates the latest model using the FedAvg algorithm. In the credit value calculation stage, if the vehicle's task is offloaded, its credit value needs to be updated. After the task offloading is completed, the EC obtains the vehicle's task execution information and part of the task information from the CV (such as workload and data volume). Then, the EC calculates the above 6 features. Finally, these feature values ​​are calculated by the designed MLP model, and the credit value is stored in the central node.

[0057] To formulate the optimization problem in federated learning, we denote the local MLP model in EC b as w b During the training of the local model, we define the optimization objective, that is, the loss (mean square error) of the local model as:

[0058]

[0059] in For the MLP model, we usually take Fi (w) = l(x i ,y i ,w)=(x i -y i ) 2 In VEC, assume that there are B EC customers performing local model training, and each EC has a dataset D b , size is n b According to FedAvg joint learning, we can rewrite the above formula as:

[0060]

[0061] We use stochastic gradient descent (SGD) as the optimization algorithm for federated learning. As shown in the following formula, each EC client b calculates the batch gradient with a fixed learning rate η That is, in the current round model w e The central server aggregates the gradients and applies the updates by averaging the gradients of its local data.

[0062]

[0063] Experimental data and conclusions

[0064] We implemented the proposed task offloading algorithm and conducted experiments on an Intel(R) Xeon(R) E5-2620 CPU v4@2.10GHz with 32GB memory and Linux operating system. Our experiments consist of two parts. The first part evaluates the federated learning of credit priority calculation. The second part evaluates the task offloading algorithm with credit value.

[0065] In the first part of the federated learning of credit value calculation, we constructed a training dataset of 3000 samples and a test dataset of 1000 samples. In the proposed algorithm, we use the PySyft federated learning framework to simulate distributed federated learning, explicitly using the federated learning algorithm of FedAvg. Then, in order to analyze the impact of the federated learning algorithm and the effect parameters in the federated learning, as shown in Table II, we tested 5 groups of learning experiments, including 4 groups of federated learning experiments and the traditional centralized MLP algorithm. During the training of the federated model, the number of clients B is set to 50, and the epoch of local MLP training is 1 in each round of communication before the averaging step. As shown in Table II, when the same calculation round parameters are set, that is, the federated learning sets the communication round to 100 and the centralized MLP learning sets the epochs to 100, the accuracy of the federated learning is not as good as the traditional centralized MLP learning method, which is also in line with expectations. At the same time, we set the credit value to three priorities, namely, high credit value priority, medium credit value priority, and low credit value priority. We use {3,2,1} to represent the priority. As can be seen from Table 2, for the classification accuracy of credit value priority, the joint learning algorithm can achieve an accuracy of nearly 90%, which shows that it is credible. At the same time, we compared the changes in the parameter C of the joint learning algorithm, which determines the customer's participation rate in each round of communication and affects the accuracy. Figure 5 and Figure 6 As shown, more clients participating in the update of the federated learning model means that it can quickly reach good accuracy, but will lead to fluctuations in the model training process.

[0066] Table 2: Performance of learning algorithms under different parameters

[0067]

[0068] Figure 7 and Figure 8 The test accuracy and training loss are compared for different numbers of clients, where 3000, 6000, 9000, and 12000 total training data are used to keep the amount of data for each client the same. Figure 8 It is shown that more clients lead to slower optimization during training. This experimental result shows that when the number of ECs in VEC increases, the training performance decreases. In order to improve the training performance, the communication rounds of federated learning training should be increased. Fig. 9 shows the communication time on the Internet during training. We can see that when the number of ECs participating in training increases, the total communication time also increases. This result shows that federated learning requires more communication overhead when more clients participate in training.

[0069] Task offloading verification

[0070] We designed a one-way multi-lane scenario as the road condition. In the simulation process, we only consider the performance of one unloading decision. We set the initial position, speed and task information as our test case. Table III shows the main parameters and initial information of the test case. In the table, W, P, α, θ is related to the network transmission model, as shown in formula (1), Represents the time slot. For task N i , vector representation (p,v,δ,ω,I,ρ,λ,c) for RSU R i , vector represents (p,δ). In order to evaluate the performance of our task offloading algorithm, we tested three algorithms. The first one is the proposed task offloading algorithm based on credit value calculation. The second one is the branch and constraint algorithm without credit value. The third one is the random selection and maximum allocation algorithm (RM). The RM algorithm randomly selects an EC connected by one hop, and then allocates all resources of the EC to the task to obtain the decision result. In this experiment, we set two kinds of bad behaviors, namely, bad behavior of location information and bad behavior of task information. For location error behavior, the experiment interferes with the location of the vehicle according to the credit value. For task information bad behavior, the experiment interferes with the workload of the task according to the credit value. Fig.10 and Fig.11 Corresponding to the performance of these two groups of experiments.

[0071] Table 3: Simulation parameters

[0072]

[0073] The first comparison metric is defined in P1, which means the weighted average latency reduction rate. Another metric is the weighted task multiplier, which can be expressed as task throughput. The task multiplier is the ratio between the number of tasks completed by VEC and the number of tasks without task offloading. The weighted task multiplier takes into account the priority of the credit value, and the multiplier ratio of each task is multiplied by the priority (3,2,1) before being averaged, i.e. in and Refers to the amount of work completed by VEC and the global workload in the same time period. Fig.10 and Fig.11 As shown in Figure 2, we compare the weighted average delay reduction index and the weighted task multiplier. It can be seen that by introducing the credit value, the higher the credit value, the better the resources can be obtained, the credit-based delay reduction rate can be reduced, and the number of concurrent tasks will also increase. In addition, by comparing the branch and constrained task offloading in RM, we see that the branch and constraint method can effectively optimize the task offloading optimization problem, which is also in line with the proposed algorithm setting.

[0074] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for unloading credit-prioritized tasks in a vehicle network based on federated learning, the steps comprising: 1) EC and CV send their location information and available resource information to CC respectively; EC is the edge cloud server, CV is the vehicle in the vehicle network, and CC is the cloud center; 2) When CV needs to offload a task, it sends the offloading task information to DC, where DC is the decision-making center in the vehicle network; 3) For the current batch of unloading tasks to be processed, DC obtains the credit value of each CV that sends the unloading task from CC; then sets the processing priority of each unloading task according to the credit value, and makes task unloading decisions for unloading tasks with high processing priority and sends them to the corresponding EC and CV; wherein CC calculates the credit value of the corresponding CV according to the behavior information of CV.

2. The method according to claim 1, characterized in that The method by which CC calculates the reputation value of the corresponding CV based on the behavior information of CV is as follows: first, six features are calculated based on the behavior information of each CV in the training data set: offset feature Transmission offset feature Data offset characteristics Workload offset feature Duration offset feature and vehicle flow data characteristics; Then, the MLP model is trained according to the features corresponding to each CV to obtain the reputation value model; then the six features of the target CV are input into the reputation value model to obtain the reputation value of the target CV.

3. The method according to claim 1 or 2, characterized in that: DC uses branch and bound method to process offload tasks with the same priority: To solve; among them, represents the reduction rate of task delay, represents the computing time of edge computing, c i is the credit value of the CV corresponding to the i-th offloading task, n is the total number of offloading tasks with the same priority, is the average task amount; the solution method is: for each offloading task, in the mth round of resource allocation, a constant resource mδ is allocated to the offloading task from an EC to obtain a corresponding solution space s; if the boundary value of the current solution s is less than the minimum value of the existing solution, then the m+1th round of resource allocation is performed, and a constant resource (m+1)δ is allocated to the offloading task from an EC, and the solution space s is updated until the boundary value of s is less than the minimum value of the existing solution; the resource allocation stage should meet the constraints: each offloading task can only be offloaded to at most one EC and the resources used to allocate by the EC are not greater than the resources it owns.

4. The method according to claim 1 or 2, characterized in that: EC periodically obtains the latest reputation value model from CC, and then trains the reputation value model using the data obtained by completing the offloading task locally and uploads it to CC; then CC uses the federated learning algorithm to aggregate the received reputation value model and update the reputation value model.

5. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing each step in the method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images