Method, system and electronic device for unloading tasks in connected vehicle network

The meta-initialization parameters obtained through the meta-learning algorithm training are initialized to the A3C algorithm model, which solves the adaptability problem of dynamic changes in the Internet of Vehicles environment, and realizes the rapid acquisition of optimal task offload decisions, meeting the delay requirements of delay-sensitive tasks.

CN116112525BActive Publication Date: 2024-11-08ZHONGSHU MASSIVE (ZHONGSHAN) TECH INVESTMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310086023.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-02-02
Publication Date
2024-11-08
Estimated Expiration
2043-02-02

AI Technical Summary

Technical Problem

The prior art is difficult to quickly adapt to dynamic changes in the Internet of Vehicles environment, the traditional DRL algorithm has poor adaptability, it is difficult to meet the delay requirements of delay-sensitive tasks, and the task offload decision efficiency is low.

Method used

Through the meta-learning algorithm, the samples in different environments of unloading the vehicle network task are trained, and the meta-initialization parameters are obtained, and the network parameters of the A3C algorithm model are initialized, so that it can quickly adapt to the dynamically changing vehicle network environment, thereby obtaining the optimal task unloading decision .

Benefits of technology

It realizes the rapid acquisition of optimal task offload decisions in a dynamic environment, meets the delay requirements of sensitive tasks, and improves the efficiency and adaptability of task offload decisions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116112525B_ABST
    Figure CN116112525B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, system and electronic device for unloading tasks in an Internet of Vehicles, and relates to the technical field of vehicle edge computing and machine learning. The method comprises: establishing a network topology structure for unloading tasks in an Internet of Vehicles; obtaining characteristic information of vehicle tasks and characteristic information of servers through the network topology structure; establishing an optimization function with delay, energy consumption and service quality as optimization targets based on the network topology structure for unloading tasks in an Internet of Vehicles; establishing a Markov decision process model based on the optimization function, and configuring the state space, action space and reward function of the Markov decision process model; training model parameters of a meta-learning algorithm model according to sample sets of unloading tasks in an Internet of Vehicles under different environments to obtain meta-initialization parameters of the meta-learning algorithm model; initializing network parameters of an A3C algorithm model through the meta-initialization parameters, and solving the Markov decision process model using the A3C algorithm model after initializing the network parameters to obtain a task unloading decision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vehicle edge computing and machine learning technology, and more specifically, to a method, system and electronic device for offloading vehicle network tasks. Background Art

[0002] With the development of artificial intelligence (AI), 5G mobile communications and Internet of Vehicles (IoV), a large number of smart car applications have emerged, such as autonomous driving and collision warning. However, executing these computationally intensive and delay-sensitive tasks is a huge challenge for resource-constrained vehicle devices. Cloud computing technology can be used to offload tasks to resource-rich remote cloud servers for processing, thereby solving the problem of insufficient computing power of vehicle devices. However, the high latency caused by long-distance task transmission cannot meet the requirements of vehicles for delay-sensitive tasks. Therefore, Mobile Edge Computing (MEC) is introduced into the IoV. The MEC server is deployed in the roadside unit (RSU) on the roadside to provide short-range task offloading services for vehicles. Compared with cloud computing task offloading, edge computing offloading can alleviate the problem of insufficient computing resources of vehicle equipment and reduce the delay of task offloading. However, cloud computing and edge computing have their own advantages. Cloud servers can provide more computing resources than MEC servers. Especially for computing-intensive tasks, MEC servers cannot meet the computing resource requirements. At the same time, in the Internet of Vehicles, the task offloading process is usually affected by different factors, such as wireless communication channels, network connection quality, mobility of vehicle-mounted devices, etc. Different task types have different requirements for latency. Therefore, how to make the optimal offloading decision based on the end-edge cloud is the most critical issue in the task offloading of the Internet of Vehicles. It needs to dynamically decide whether the task should be processed locally, on the edge server, or on the cloud server. On the one hand, in the Internet of Vehicles, a large number of tasks have strict requirements on latency, and traditional deep learning algorithms such as convolutional neural networks have a slow learning speed and cannot meet the latency requirements of latency-sensitive tasks; on the other hand, although a large number of tasks in the Internet of Vehicles are similar, they often require retraining of the model, resulting in inefficient decision-making.

[0003] The task offloading algorithm in the prior art models the task offloading decision problem as a Markov decision process model, and then solves it through a machine learning algorithm. Although this type of algorithm can solve the computational explosion problem faced by traditional algorithms, due to the complexity of the Internet of Vehicles environment, every time the Internet of Vehicles environment changes or a new type of task arrives, the offloading decision must be retrained. Therefore, the traditional DRL algorithm has poor adaptability and slow convergence speed, making it difficult to adapt to the dynamically changing environment of the Internet of Vehicles and to meet the requirements of delay-sensitive tasks. Summary of the invention

[0004] In order to solve the adaptability problem of the prior art to the dynamically changing Internet of Vehicles environment, the purpose of the present invention is to provide a method, system and electronic device for Internet of Vehicles task offloading, which trains samples under different Internet of Vehicles task offloading environments through a meta-learning algorithm to obtain meta-initialization parameters, and initializes the network parameters of the A3C algorithm model based on the meta-initialization parameters, so that the A3C algorithm model can quickly adapt to the dynamically changing Internet of Vehicles environment, thereby quickly obtaining the optimal task offloading decision from the dynamic environment, thereby meeting the latency requirements of sensitive tasks.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions:

[0006] In a first aspect of the present application, a method for offloading a task in an Internet of Vehicles is provided, the method comprising:

[0007] Establish a network topology for vehicle networking task offloading;

[0008] Acquire characteristic information of the vehicle's tasks and characteristic information of the server through the network topology structure;

[0009] Based on the network topology structure of the Internet of Vehicles task offloading, an optimization function with latency, energy consumption and service quality as optimization objectives is established;

[0010] A Markov decision process model is established based on the optimization function, and the state space, action space and reward function of the Markov decision process model are configured, where the state space corresponds to the characteristic information of the task and the characteristic information of the server, and the action space contains the execution subject of the task offloading;

[0011] The model parameters of the meta-learning algorithm model are trained according to the sample sets of Internet of Vehicles task offloading in different environments to obtain the meta-initialization parameters of the meta-learning algorithm model;

[0012] The network parameters of the A3C algorithm model are initialized by the meta-initialization parameters, and the A3C algorithm model after the network parameters are initialized is used to solve the Markov decision process model to obtain the task offloading decision.

[0013] In one implementation, the characteristic information of the task includes storage resources, computing resources, and maximum tolerable delay required for task offloading;

[0014] The characteristic information of the server includes the storage resources and computing resources of the server, wherein the server includes edge servers and cloud servers.

[0015] In one embodiment, the optimization function is calculated as:

[0016] in, represents the propagation delay, Indicates the queuing delay, Indicates processing delay;

[0017] in, b r,i represents the unloading decision of the i-th subtask on the r-th vehicle, and correspondingly, b r,i = b0 means offloading the task to the local server. If b r,i = b1 means offloading the task to the corresponding edge server for execution, b r,i = b2 means offloading the task to the adjacent edge server for execution. r,i = b3 means offloading the task to the cloud server for execution; e local 、e edge and e cloud Represents the energy consumed by local devices, edge servers, and cloud servers to run each CPU cycle;

[0018] in,

[0019] In one embodiment, the execution entity of task offloading includes offloading the task to a local vehicle for processing, or offloading to a corresponding edge server for processing, or offloading to an edge server adjacent to the edge server for processing, or offloading to a cloud server for processing.

[0020] In one embodiment, the reward function is calculated as R i =-ξ1T i -ξ2E i +ξ3Q i , where ξ1, ξ2, ξ3 represent the weights of the delay model, energy consumption model, and service quality model, respectively, and they all satisfy: Among them, T i represents the delay model, E i represents the energy consumption model, Q i Represents the quality of service model.

[0021] In a second aspect of the present application, a vehicle networking task offloading system is provided, the system comprising:

[0022] The first module is used to establish the network topology structure for offloading Internet of Vehicles tasks;

[0023] A feature information acquisition module, used to acquire feature information of the vehicle's task and feature information of the server through a network topology structure;

[0024] The second module is used to establish an optimization function with latency, energy consumption and service quality as optimization objectives based on the network topology structure of vehicle networking task offloading;

[0025] The third module is used to establish a Markov decision process model based on the optimization function, and configure the state space, action space and reward function of the Markov decision process model, wherein the state space corresponds to the characteristic information of the task and the characteristic information of the server, and the action space contains the execution subject of the task offloading;

[0026] A training module is used to train the model parameters of the meta-learning algorithm model according to the sample sets of the Internet of Vehicles task offloading in different environments to obtain the meta-initialization parameters of the meta-learning algorithm model;

[0027] The decision solving module is used to initialize the network parameters of the A3C algorithm model through the meta-initialization parameters, and use the A3C algorithm model after the network parameters are initialized to solve the Markov decision process model to obtain the task offloading decision.

[0028] In one implementation, the feature information acquisition module is further used to:

[0029] The characteristic information of the task includes the storage resources, computing resources and maximum tolerable delay required for task offloading;

[0030] The characteristic information of the server includes the storage resources and computing resources of the server, wherein the server includes edge servers and cloud servers.

[0031] In one embodiment, the second module is further configured to:

[0032] The calculation formula of the optimization function is:

[0033] in, represents the propagation delay, Indicates the queuing delay, Indicates processing delay;

[0034] in, b r,i represents the unloading decision of the i-th subtask on the r-th vehicle, and correspondingly, b r,i = b0 means offloading the task to the local server. If b r,i = b1 means offloading the task to the corresponding edge server for execution, b r,i = b2 means offloading the task to the adjacent edge server for execution. r,i = b3 means offloading the task to the cloud server for execution; e local 、e edge and ecloud Represents the energy consumed by local devices, edge servers, and cloud servers to run each CPU cycle;

[0035] in,

[0036] In one embodiment, the third module is further used for:

[0037] The execution subject of task offloading includes offloading the task to a local vehicle for processing, or offloading to a corresponding edge server for processing, or offloading to an edge server adjacent to the edge server for processing, or offloading to a cloud server for processing.

[0038] According to a third aspect of the present application, an electronic device is provided, comprising a processor and a memory storing a computer program, wherein when the processor executes the computer program, the steps of a method for unloading a task in an Internet of Vehicles as described in any one of the first aspects of the present application are implemented.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention provides a method for offloading tasks in an Internet of Vehicles (IoV). Samples in different environments of IoV task offloading are trained by a meta-learning algorithm to obtain meta-initialization parameters. The network parameters of an A3C algorithm model are initialized based on the meta-initialization parameters, so that the A3C algorithm model can quickly adapt to the dynamically changing IoV environment, thereby quickly obtaining the optimal task offloading decision from the dynamic environment, thereby meeting the delay requirements of sensitive tasks.

[0041] In addition, the present application also provides a vehicle networking task offloading system and electronic device, which have the same technical effects as the above-mentioned vehicle networking task offloading method, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0043] Figure 1 A flowchart showing a method for offloading a task in an Internet of Vehicles provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram showing a vehicle device and a server according to an embodiment of the present application;

[0045] Figure 3 A schematic diagram showing the structure of a Markov decision process model provided in an embodiment of the present application;

[0046] Figure 4A schematic diagram showing the internal model structure of the Markov decision process model provided in an embodiment of the present application;

[0047] Figure 5 A comparison diagram of simulation results provided by the embodiments of the present application is shown;

[0048] Figure 6 A functional block diagram of a vehicle networking task offloading system provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0050] Please refer to Figure 1 , Figure 1 A flowchart of a method for offloading a task in an Internet of Vehicles provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0051] S110, establishing a network topology structure for offloading Internet of Vehicles tasks.

[0052] In this embodiment, as common knowledge of those skilled in the art, Figure 2 As shown in the figure, the vehicle is connected to the edge server (MEC) through the roadside unit (RSU), and the vehicle and the edge server (MEC) are connected to the cloud server through the base station. The tasks on the vehicle can be processed locally or offloaded to the edge server and cloud server for processing. The network topology of the Internet of Vehicles task offloading can be determined. The number of vehicles is 40, the number of tasks on each vehicle is 10, the number of edge servers is 10, and the number of cloud servers is 1. Since different tasks require different computing resources and storage resources, when offloading tasks, it is necessary to divide the tasks into multiple independent subtasks, and then make offloading decisions for these subtasks separately. Assume that the task definition of the vehicle is as follows: in represents the set of tasks on the rth vehicle, each of which contains n independent subtasks, v i represents the i-th subtask. The characteristic information of the vehicle is represented as: {D r ,C r}, where D r represents the storage resources on the rth car, C r In this embodiment, the storage resource of each vehicle is 200MB, and the computing resource is 1500~2500MHz.

[0053] S120, acquiring characteristic information of the vehicle's task and characteristic information of the server through the network topology structure.

[0054] In this embodiment, the network topology structure regards the server as a node, so the characteristic information of the vehicle connected to the server and the characteristic information of the server itself can be obtained. Since it involves the task offloading of the vehicle, in one embodiment, the characteristic information of the task includes the storage resources, computing resources and maximum tolerable delay required for task offloading; the characteristic information of the server includes the storage resources and computing resources of the server, where the server includes edge servers and cloud servers. That is, the characteristic information of the task is expressed as: where d i represents the storage resources required by the i-th subtask, c i represents the computing resources required for the i-th subtask, Represents the maximum tolerable delay of the i-th subtask.

[0055] The characteristic information of the server is expressed as: j ,C j},0≤j≤k, where D j represents the storage resources of the jth edge server, C j It should be noted that, in this embodiment, {D0, C0} represents the characteristic information of the cloud server.

[0056] S130, establishing an optimization function with latency, energy consumption and service quality as optimization targets based on the network topology structure of the Internet of Vehicles task offloading.

[0057] In this embodiment, due to the node information of the network topology structure of the Internet of Vehicles task offloading, the influence of delay, energy consumption and service quality on task offloading is considered. Therefore, this embodiment establishes an optimization function with delay, energy consumption and service quality as optimization targets, wherein the delay includes processing delay, queuing delay and propagation delay. It should be noted that different offloading decisions correspond to different processing delays, queuing delays and propagation delays.

[0058] S140, establishing a Markov decision process model based on the optimization function, and configuring the state space, action space and reward function of the Markov decision process model, wherein the state space corresponds to the characteristic information of the task and the characteristic information of the server, and the action space includes the execution subject of the task offloading.

[0059] It should be understood that the execution subject of task offloading includes the vehicle itself, that is, local processing, or processing on the edge server. Considering the storage resources and computing resources of the edge server itself, based on the characteristic information of the server obtained by the network topology structure, when the storage resources and computing resources of a certain edge server itself are insufficient, it can be considered whether the storage resources and computing resources of the edge server adjacent to the edge server are sufficient, so as to serve as the execution subject of the task offloading decision. Modeling the task offloading decision problem as a Markov decision process model based on the optimization function, and designing the state space, action space, and reward function are conventional technical means of those skilled in the art, so no redundant description is made in this embodiment.

[0060] S150, training model parameters of the meta-learning algorithm model according to the sample sets of Internet of Vehicles task offloading in different environments to obtain meta-initialization parameters of the meta-learning algorithm model.

[0061] In this embodiment, Figure 3 As shown, the decision model proposed in the above embodiment can be divided into an internal model and an external model. Specifically, the internal model focuses on task offloading decisions. The internal model is responsible for receiving the characteristic information of the vehicle equipment and the server, the characteristic information of the task, and providing the final offloading decision for the task; the external model focuses on environmental changes and is responsible for training the meta-initialization parameters for the internal model. When the environment changes, such as the performance changes of the MEC server and the vehicle or the transmission bandwidth changes between the vehicle equipment and the MEC server, the network parameters of the neural network in the internal model can be adjusted through the trained meta-initialization parameters, so that the model can adapt to the new environment more quickly.

[0062] The meta-learning algorithm model can adopt the MAML algorithm model, whose principle is to store the decision and execution results in different environments into the experience set. Randomly select training samples in one environment for learning, and then randomly select another environment after learning. The purpose is to ensure that the parameters of the model will not be close to the optimal parameters in a specific environment. The parameters trained in this way are used as the meta-initialization parameters of the internal model.

[0063] S160, initializing the network parameters of the A3C algorithm model through the meta-initialization parameters, and solving the Markov decision process model using the A3C algorithm model after the network parameters are initialized to obtain the task offloading decision.

[0064] In this embodiment, the network parameters of the Markov decision process model are initialized based on the meta-initialization parameters trained in the embodiment in the above step S140, so that the network parameters of the Markov decision process model are tuned, and the initialized Markov decision process model is solved by using the A3C algorithm of deep learning, such as Figure 4As shown, in this embodiment, the A3C algorithm can use a multi-core CPU to establish multiple threads for asynchronous training. In A3C, the gradient calculation uses the advantage function, and the specific expression is as follows: Among them A π (s,a) is the advantage function, which can be expressed as: A π (s,a)=Q π (s,a)-V π (s,a). A3C is placed in the cloud server, and the MEC server and the vehicle-mounted equipment upload the feature information to the intelligent agent in the cloud server. The Actor in the cloud server acts according to the feature information, and the Critic outputs the value according to the reward fed back by the Actor and the observation of the environment state, outputs the estimated value of the value function, and guides the Actor network to update the parameters. N+1 threads are created based on the A3C algorithm, and an Actor network and a Critic network are designed for each thread. Each thread runs independently, and the Actor in each thread observes the current state, outputs and executes actions according to its own network parameters. The Critic network then calculates the TD-error based on the generated reward and the state at the next moment and the current state and action, and passes it to the Actor network. The Actor network uses gradient descent to update its parameters based on the TD-error. In particular, the global network does not participate in the actual detection, but copies and shares the parameters detected by the first N threads. In this embodiment, the structures of the Actor and Critic neural networks are set to: 11×256×256×256×4, 23×256×256×256×1, respectively, with three hidden layers and 265 neurons. In testing the rapid adaptability under different MEC environments, the convergence of the Markov decision process model initialized with meta-initialization parameters and the Markov decision process model with random initialization parameters under different test environments is compared. Considering the convergence of the model when training with meta-initialization parameters and random initialization parameters in a new environment, the comparison results are shown in Figure 2. Figure 5 As shown in the figure, the horizontal axis is the total number of training rounds, and the vertical axis is the reward value generated by the reward function. The simulation results show that the Markov decision process model using meta-initialization parameters converges significantly faster than the model trained with random initialization parameters, achieving the expected effect.

[0065] In summary, this embodiment provides a method for unloading tasks in an Internet of Vehicles (IoV), which trains different environmental parameters through a meta-learning algorithm to obtain meta-initialization parameters, initializes the network parameters of the A3C algorithm model based on the meta-initialization parameters, and uses the initialized A3C algorithm model to solve the Markov decision process model, so that the Markov decision process model can adapt to the dynamically changing IoV environment, thereby quickly obtaining the optimal task offloading decision from the dynamic environment, thereby meeting the latency requirements of sensitive tasks.

[0066] In one embodiment, the characteristic information of the task includes storage resources, computing resources and maximum tolerable delay required for task offloading;

[0067] The characteristic information of the server includes the storage resources and computing resources of the server, wherein the server includes edge servers and cloud servers.

[0068] The above embodiments have described the characteristic information of the task and the characteristic information of the server in detail, so they will not be repeated here.

[0069] In one embodiment, the calculation formula of the optimization function is:

[0070] in, represents the propagation delay, Indicates the queuing delay, Indicates processing delay;

[0071] in, b r,i represents the unloading decision of the i-th subtask on the r-th vehicle, and correspondingly, b r,i = b0 means offloading the task to the local server. If b r,i = b1 means offloading the task to the corresponding edge server for execution, b r,i = b2 means offloading the task to the adjacent edge server for execution. r,i = b3 means offloading the task to the cloud server for execution; e local 、e edge and e cloud Represents the energy consumed by local devices, edge servers, and cloud servers to run each CPU cycle;

[0072] in,

[0073] In this embodiment, a delay model is established according to different unloading decisions. The delay model includes three parts: processing delay, queuing delay, and propagation delay.

[0074] The processing delay is expressed as:

[0075] in represents the processing delay of the i-th subtask on the r-th vehicle, r,i The unloading decision of the i-th subtask on the r-th vehicle is as follows: r,i = b0 means local execution. If b r,i = b1 means offloading the task to the corresponding edge server for execution, br,i = b2 means offloading the task to the adjacent edge server for execution. r,i = b3 means offloading the task to the cloud server for execution.

[0076] The queuing delay is expressed as:

[0077] Where L j Represents the set of tasks to be processed on the jth server.

[0078] The propagation delay is expressed as:

[0079] Where R V2E represents the transmission speed of tasks uploaded from vehicle equipment to edge servers; R E2E Represents the transmission speed between edge servers; R V2C Indicates the transmission speed between vehicle equipment and cloud server.

[0080] Therefore, the delay model can be established as: Where N represents the number of subtasks on the rth car.

[0081] The energy consumption model is established according to different offloading decisions. The energy consumption of the same task when it is processed locally or offloaded to the MEC server or cloud server is different. Therefore, the energy consumption of the i-th subtask on the r-th vehicle under different offloading decisions is defined as:

[0082]

[0083] where e local 、e edge 、e cloud Respectively represent the energy consumed for processing each bit of data locally or offloaded to the MEC server and cloud server. In this embodiment, they are defined as: local =0.3J / Hz, e edge =0.15J / Hz, e cloud =0.1J / Hz.

[0084] Therefore, the total energy consumption generated by the rth vehicle can be defined as:

[0085] A service quality model is established based on latency. Since some tasks in the Internet of Vehicles are sensitive to latency, a service quality model is introduced to evaluate the task offloading decision, which is specifically expressed as follows:

[0086] Therefore, the task offloading service quality of the rth vehicle can be expressed as:

[0087]

[0088] An optimization function is established based on latency, energy consumption, and service quality. The ultimate goal of the optimization is to reduce latency and energy consumption while improving service quality, so it is expressed as:

[0089] In one embodiment, the execution entity of task offloading includes offloading the task to a local vehicle for processing, or offloading to a corresponding edge server for processing, or offloading to an edge server adjacent to the edge server for processing, or offloading to a cloud server for processing.

[0090] In this embodiment, the execution subject of the task offloading decision has been described in detail in the above embodiments, so no redundant explanation is given here.

[0091] In one embodiment, the reward function is calculated as R i =-ξ1T i -ξ2E i +ξ3Q i , where ξ1, ξ2, ξ3 represent the weights of the delay model, energy consumption model, and service quality model, respectively, and they all satisfy: Among them, T i represents the delay model, E i represents the energy consumption model, Q i Represents the quality of service model.

[0092] In this embodiment, the task offloading decision problem is modeled as a Markov decision process model, and the state space, action space, and reward function are designed;

[0093] The state space is represented as: It should be understood that only the feature information of the MEC server currently connected to the vehicle and the adjacent servers is considered.

[0094] The action space is represented as: a i ={0,1,2,3}, where a i Equal to 0, 1, 2, and 3 respectively indicate that the i-th subtask is processed locally or offloaded to the corresponding MEC server, the adjacent MEC server, and the cloud server for processing.

[0095] Since the optimization goal is to reduce system latency and energy consumption and improve service quality, and the goal of deep reinforcement learning is to maximize the reward value, the reward function is set to: R i =-ξ1T i -ξ2E i +ξ3Q i , where ξ1, ξ2, and ξ3 represent the weights of delay, energy consumption, and service quality, respectively, and the three simultaneously satisfy:

[0096] The vehicle networking task offloading system provided by the present invention is described below. The vehicle networking task offloading system described below and the vehicle networking task offloading method described above can refer to each other.

[0097] Figure 6 is a structural diagram of a vehicle networking task offloading system provided by an embodiment of the present application, such as Figure 6 As shown, the system includes:

[0098] The first module 610 is used to establish a network topology structure for offloading Internet of Vehicles tasks;

[0099] The characteristic information acquisition module 620 is used to acquire characteristic information of the vehicle's task and characteristic information of the server through the network topology structure;

[0100] The second module 630 is used to establish an optimization function with latency, energy consumption and service quality as optimization targets based on the network topology structure of the Internet of Vehicles task offloading;

[0101] The third module 640 is used to establish a Markov decision process model based on the optimization function, and configure the state space, action space and reward function of the Markov decision process model, wherein the state space corresponds to the characteristic information of the task and the characteristic information of the server, and the action space includes the execution subject of the task offloading;

[0102] A training module 650 is used to train the model parameters of the meta-learning algorithm model according to the sample sets of the Internet of Vehicles task offloading in different environments to obtain the meta-initialization parameters of the meta-learning algorithm model;

[0103] The decision solving module 660 is used to initialize the network parameters of the A3C algorithm model through the meta-initialization parameters, and solve the Markov decision process model using the A3C algorithm model after the network parameters are initialized to obtain the task offloading decision.

[0104] It can be seen that the Internet of Vehicles task offloading system provided by the above embodiment trains different environmental parameters through a meta-learning algorithm to obtain meta-initialization parameters, and initializes the network parameters of the Markov decision process model based on the meta-initialization parameters, so that the Markov decision process model can adapt to the dynamically changing Internet of Vehicles environment, thereby quickly obtaining the optimal task offloading decision from the dynamic environment, thereby meeting the latency requirements of sensitive tasks.

[0105] In one implementation, the feature information acquisition module is further used to:

[0106] The characteristic information of the task includes the storage resources, computing resources and maximum tolerable delay required for task offloading;

[0107] The characteristic information of the server includes the storage resources and computing resources of the server, wherein the server includes edge servers and cloud servers.

[0108] In one embodiment, the second module is further configured to:

[0109] The calculation formula of the optimization function is:

[0110] in, represents the propagation delay, Indicates the queuing delay, Indicates processing delay;

[0111] in, b r,i represents the unloading decision of the i-th subtask on the r-th vehicle, and correspondingly, b r,i = b0 means offloading the task to the local server. If b r,i = b1 means offloading the task to the corresponding edge server for execution, b r,i = b2 means offloading the task to the adjacent edge server for execution. r,i = b3 means offloading the task to the cloud server for execution; e local 、e edge and e cloud Represents the energy consumed by local devices, edge servers, and cloud servers to run each CPU cycle;

[0112] in,

[0113] In one embodiment, the third module is further used for:

[0114] The execution subject of task offloading includes offloading the task to a local vehicle for processing, or offloading to a corresponding edge server for processing, or offloading to an edge server adjacent to the edge server for processing, or offloading to a cloud server for processing.

[0115] According to a third aspect of the present application, an electronic device is provided, comprising a processor and a memory storing a computer program, wherein when the processor executes the computer program, the steps of a method for unloading a task in an Internet of Vehicles as described in any one of the first aspects of the present application are implemented.

[0116] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for unloading tasks in an Internet of Vehicles, characterized in that: include: Establish a network topology for vehicle networking task offloading; Acquire characteristic information of the vehicle's tasks and characteristic information of the server through the network topology structure; Based on the network topology structure of the Internet of Vehicles task offloading, an optimization function with latency, energy consumption and service quality as optimization objectives is established; A Markov decision process model is established based on the optimization function, and the state space, action space and reward function of the Markov decision process model are configured, where the state space corresponds to the characteristic information of the task and the characteristic information of the server, and the action space contains the execution subject of the task offloading; The model parameters of the meta-learning algorithm model are trained according to the sample sets of Internet of Vehicles task offloading in different environments to obtain the meta-initialization parameters of the meta-learning algorithm model; The network parameters of the A3C algorithm model are initialized by the meta-initialization parameters, and the A3C algorithm model after the network parameters are initialized is used to solve the Markov decision process model to obtain the task offloading decision.

2. A method for offloading Internet of Vehicles tasks according to claim 1, characterized in that: The characteristic information of the task includes the storage resources, computing resources and maximum tolerable delay required for task offloading; The characteristic information of the server includes the storage resources and computing resources of the server, wherein the server includes edge servers and cloud servers.

3. The method for unloading Internet of Vehicles tasks according to claim 1, characterized in that: The calculation formula of the optimization function is: in, represents the propagation delay, Indicates the queuing delay, Indicates processing delay; in, b r,i represents the unloading decision of the i-th subtask on the r-th vehicle, and correspondingly, b r,i = b0 means offloading the task to the local server. If b r,i = b1 means offloading the task to the corresponding edge server for execution, b r,i = b2 means offloading the task to the adjacent edge server for execution. r,i = b3 means offloading the task to the cloud server for execution; e local 、e edge and e cloud Represents the energy consumed by local devices, edge servers, and cloud servers to run each CPU cycle; in, 4. The method for offloading Internet of Vehicles tasks according to claim 1, characterized in that: The execution subject of task offloading includes offloading the task to a local vehicle for processing, or offloading to a corresponding edge server for processing, or offloading to an edge server adjacent to the edge server for processing, or offloading to a cloud server for processing.

5. The method for unloading Internet of Vehicles tasks according to claim 1, characterized in that: The reward function is calculated as R i =-ξ1T i -ξ2E i +ξ3Q i , where ξ1, ξ2, ξ3 represent the weights of the delay model, energy consumption model, and service quality model, respectively, and they all satisfy: Among them, T i represents the delay model, E i represents the energy consumption model, Q i Represents the quality of service model.

6. A vehicle networking task offloading system, characterized in that: The system includes: The first module is used to establish the network topology structure for offloading Internet of Vehicles tasks; A feature information acquisition module, used to acquire feature information of the vehicle's task and feature information of the server through a network topology structure; The second module is used to establish an optimization function with latency, energy consumption and service quality as optimization objectives based on the network topology structure of vehicle networking task offloading; The third module is used to establish a Markov decision process model based on the optimization function, and configure the state space, action space and reward function of the Markov decision process model, wherein the state space corresponds to the characteristic information of the task and the characteristic information of the server, and the action space contains the execution subject of the task offloading; A training module is used to train the model parameters of the meta-learning algorithm model according to the sample sets of the Internet of Vehicles task offloading in different environments to obtain the meta-initialization parameters of the meta-learning algorithm model; The decision solving module is used to initialize the network parameters of the A3C algorithm model through the meta-initialization parameters, and use the A3C algorithm model after the network parameters are initialized to solve the Markov decision process model to obtain the task offloading decision.

7. The vehicle networking task offloading system according to claim 6, characterized in that: The feature information acquisition module is also used for: The characteristic information of the task includes the storage resources, computing resources and maximum tolerable delay required for task offloading; The characteristic information of the server includes the storage resources and computing resources of the server, wherein the server includes edge servers and cloud servers.

8. The vehicle networking task offloading system according to claim 6, characterized in that: The second module is also used for: The calculation formula of the optimization function is: in, represents the propagation delay, Indicates the queuing delay, Indicates processing delay; in, b r,i represents the unloading decision of the i-th subtask on the r-th vehicle, and correspondingly, b r,i = b0 means offloading the task to the local server. If b r,i = b1 means offloading the task to the corresponding edge server for execution, b r,i = b2 means offloading the task to the adjacent edge server for execution. r,i = b3 means offloading the task to the cloud server for execution; e local 、e edge and e cloud Represents the energy consumed by local devices, edge servers, and cloud servers to run each CPU cycle; in, 9. The vehicle networking task offloading system according to claim 6, characterized in that: The third module is also used for: The execution subject of task offloading includes offloading the task to a local vehicle for processing, or offloading to a corresponding edge server for processing, or offloading to an edge server adjacent to the edge server for processing, or offloading to a cloud server for processing.

10. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the steps of a method for unloading a task in an Internet of Vehicles as described in any one of claims 1 to 5 are implemented.