A method and system for task offloading and resource allocation in Internet of Vehicles based on self-attention mechanism

Abstract

The present invention provides a method and system for Internet of Vehicles task unloading and resource allocation based on a self-attention mechanism, the method comprising the following steps: constructing an Internet of Vehicles task unloading model based on the Internet of Vehicles highway environment VEC architecture; obtaining a joint optimization problem of task unloading and resource allocation based on the Internet of Vehicles task unloading model; converting the joint optimization problem into a Markov decision process; using an A-MADDPG algorithm based on a self-attention mechanism to solve the Markov decision process and obtain a joint optimization scheme; completing Internet of Vehicles task unloading and resource allocation based on the joint optimization scheme. The task unloading joint optimization scheme designed by the present invention, through intelligent collaboration, not only selects a suitable unloading object but also selects a suitable communication channel for unloading transmission, and the idle resources of each edge server are dynamically allocated according to task requirements, thereby ensuring that all vehicles in the entire environment can be at an efficient operating level.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Vehicles, and specifically relates to an Internet of Vehicles task offloading and resource allocation method and system based on a self-attention mechanism. Background Art

[0002] Due to the limited computing power and storage capacity of vehicles, it is difficult to complete complex computing tasks within the specified time. This is unacceptable for a series of tasks involving user safety. The goal of task offloading is to solve the problem of insufficient vehicle computing resources and achieve the fastest completion of all tasks in the Internet of Vehicles at the lowest cost.

[0003] To achieve effective task offloading, vehicle edge computing (VEC) is proposed as an innovative computing architecture. It aims to make full use of the computing resources inside and around the vehicle to provide real-time, distributed computing capabilities, and use the communication between vehicles and between vehicles and roadside infrastructure to achieve task offloading and distributed processing, thereby improving computing efficiency and reducing energy consumption. Due to the high mobility of vehicles, the vehicle networking environment and communication conditions are constantly changing. In this dynamic environment, it is a rather difficult problem to select a suitable offloading object for each task of each vehicle to ensure the completion delay of all tasks as low as possible. From a holistic perspective, there is both a competitive relationship between vehicles: each vehicle hopes to obtain more computing resources and communication resources to ensure the effective completion of local tasks; and a cooperative relationship: each vehicle should minimize the impact on the task calculation of other vehicles while obtaining more resources, and assist other vehicles as much as possible while completing its own tasks. Therefore, how to coordinate the offloading decisions of each vehicle so that computing resources and communication resources can be allocated as reasonably as possible according to the status of each vehicle is a problem that needs to be solved. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a method and system for Internet of Vehicles task unloading and resource allocation based on a self-attention mechanism. Each vehicle can intelligently select the optimal unloading strategy for the task according to the current environmental status to achieve effective processing of tasks and efficient use of resources.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for task offloading and resource allocation in an Internet of Vehicles based on a self-attention mechanism comprises the following steps:

[0007] Based on the VEC architecture of the connected vehicle highway environment, a connected vehicle task offloading model is constructed;

[0008] Based on the Internet of Vehicles task offloading model, a joint optimization problem of task offloading and resource allocation is obtained;

[0009] Converting the joint optimization problem into a Markov decision process;

[0010] A-MADDPG algorithm based on self-attention mechanism is used to solve the Markov decision process and obtain a joint optimization solution;

[0011] Based on the joint optimization scheme, Internet of Vehicles task offloading and resource allocation are completed.

[0012] Preferably, the method for constructing the Internet of Vehicles task offloading model is:

[0013] Based on the signal transmission time of different task offloading links, a communication model of task offloading is constructed; wherein the task offloading link includes a V2I link and a V2V link;

[0014] Based on the computing time on different task offloading objects, a computing model for task offloading is constructed; wherein the task offloading objects include local smart vehicles, MEC edge servers, and vehicle edge servers;

[0015] Based on the communication model and the calculation model, the Internet of Vehicles task offloading model is obtained.

[0016] Preferably, based on the Internet of Vehicles task offloading model, the total execution delay of local intelligent vehicle tasks is minimized as the goal, and the method for obtaining the joint optimization problem of task offloading and resource allocation is:

[0017] Analyze the Internet of Vehicles task offloading model to obtain the task offloading delay of a preset vehicle at a preset time;

[0018] Using multiple constraints to constrain the task delay, a joint optimization problem of task offloading and resource allocation is obtained;

[0019] The multiple constraints include c1-c7:

[0020] Constraints c1 and c2 indicate that each task can only choose one of the three task offloading objects: the local smart vehicle, the MEC edge server, and the neighboring vehicle edge server to perform the computing task;

[0021] Constraint c3 means that only one subchannel can be selected for transmission each time offloading;

[0022] Constraint c4 represents all idle resources of the MEC edge server that can provide edge services at time t;

[0023] Constraint c5 represents all resources that each vehicle can provide computing services at time t;

[0024] Constraint c6 means that vehicle i can successfully transmit the task at time t whether it is offloaded to the MEC edge server or to the neighboring vehicle edge server;

[0025] Constraint c7 means that each task is executed within the delay limit.

[0026] Preferably, the method of converting the joint optimization problem into a Markov decision process is:

[0027] Each local intelligent vehicle is regarded as an intelligent agent, which searches for a joint optimization scheme of task offloading and resource allocation according to the observed state, performs the offloading action and obtains a reward;

[0028] Among them, the status includes the vehicle's own information, the current task information, the information of neighboring vehicles within the current vehicle's communication range, and the task roadside unit RSU which is one of the entities of the Internet of Vehicles task offloading model; the action includes the selection of the task execution target, the selection of the offloading channel, and the selection of the proportion of computing resources.

[0029] Preferably, a type of Transformer module SAME layer consisting of a fully connected layer, a multi-head self-attention mechanism layer and a fully connected layer is introduced into the multi-agent deep deterministic policy gradient MADDPG algorithm to construct a framework of the A-MADDPG algorithm based on the self-attention mechanism; the A-MADDPG algorithm framework follows the idea of ​​distributed execution based on the Actor-Critic network structure and centralized training.

[0030] Preferably, the A-MADDPG algorithm based on the self-attention mechanism is used to solve the Markov decision process, and the method for obtaining the joint optimization solution is:

[0031] Initialize the Internet of Vehicles environment;

[0032] Based on the initialized Internet of Vehicles environment, the vehicle's Actor-Critic network and network parameters are initialized using the attention mechanism, convolutional neural network, and the correlation between task offloading actions;

[0033] Based on the random computing tasks generated by local intelligent vehicles within the preset moving speed range, action decisions are generated through the Actor network;

[0034] The local intelligent vehicle performs the task offloading action based on the action decision, interacts with the Internet of Vehicles environment, and obtains corresponding rewards; the effect of the executed task offloading action is evaluated through the Critic network;

[0035] The parameters of the Actor network and Critic network are updated alternately. When the training process converges, the optimal Actor network parameters of the local intelligent vehicle are obtained.

[0036] Based on the optimal Actor network parameters, each vehicle selects the optimal task offloading object, task offloading link and computing resource allocation to obtain the joint optimization solution.

[0037] The present invention also provides a vehicle networking task unloading and resource allocation system based on a self-attention mechanism, and the method described is applied, including:

[0038] The task offloading model building module is used to build a vehicle networking task offloading model based on the vehicle networking highway environment VEC architecture;

[0039] A joint optimization problem acquisition module, used to obtain a joint optimization problem of task offloading and resource allocation based on the Internet of Vehicles task offloading model;

[0040] A problem conversion module, used for converting the joint optimization problem into a Markov decision process;

[0041] A solution module, used for solving the Markov decision process by using an A-MADDPG algorithm based on a self-attention mechanism to obtain a joint optimization solution;

[0042] The task offloading module is used to complete the Internet of Vehicles task offloading and resource allocation based on the joint optimization solution.

[0043] Preferably, the task offloading model building module includes:

[0044] A communication model building module, used to build a communication model for task offloading based on signal transmission time of different task offloading links; wherein the task offloading link includes a V2I link and a V2V link;

[0045] A computing model building module, used to build a computing model for task offloading based on the computing time on different task offloading objects; wherein the task offloading objects include local smart vehicles, MEC edge servers and vehicle edge servers;

[0046] The task offloading model building module is used to obtain the Internet of Vehicles task offloading model based on the communication model and the computing model.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows: based on the VEC architecture of the Internet of Vehicles highway environment, a task offloading model of the Internet of Vehicles is constructed; based on the task offloading model of the Internet of Vehicles, a joint optimization problem of task offloading and resource allocation is obtained; the joint optimization problem is converted into a Markov decision process; each vehicle can intelligently select the optimal joint optimization scheme for the task according to the current environmental state to achieve effective processing of the task and efficient utilization of resources; the joint optimization scheme includes not only the determination of the offloading object, but also the allocation of channel resources and the allocation of computing power resources; the task vehicle intelligently selects channel resources to reduce signal interference between vehicles and improve data transmission. transmission capacity, making task offloading faster and more reliable; the task vehicle no longer adopts the idea of ​​average distribution according to the task requirements and the target computing power resources, but intelligently selects appropriate computing power resources to reduce computing delay and resource waste, thereby improving the computing efficiency of the entire system; the present invention takes into account the correlation between states, and also introduces an attention mechanism to accelerate training and extract features, and thereby optimizes the classic multi-agent deep deterministic policy gradient algorithm, and adopts the A-MADDPG algorithm based on the self-attention mechanism to solve the Markov decision process to obtain a joint optimization scheme; based on the joint optimization scheme, the Internet of Vehicles task offloading and resource allocation are completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A flowchart of a method for Internet of Vehicles task offloading and resource allocation based on a self-attention mechanism according to an embodiment of the present invention;

[0050] Figure 2 It is a structural schematic diagram of a vehicle networking task offloading model based on VEC according to an embodiment of the present invention;

[0051] Figure 3 This is a SAME layer diagram of the Transformer module based on the self-attention mechanism in an embodiment of the present invention;

[0052] Figure 4 Figure 2 is an Actor and Critic network diagram of the A-MADDPG algorithm according to an embodiment of the present invention; wherein (a) is an Actor network structure diagram; and (b) is a Critic network structure diagram. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Embodiment 1

[0056] like Figure 1 As shown, a method for task offloading and resource allocation of Internet of Vehicles based on a self-attention mechanism includes the following steps:

[0057] S1: Building a vehicle networking task offloading model based on the vehicle networking highway environment VEC architecture;

[0058] The present invention examines the general architecture of VEC in highway scenarios. The tasks of local smart vehicles may be executed locally, or may be offloaded to the edge server MEC for execution, or may be offloaded to the neighboring vehicle edge server for execution.

[0059] The entities of the Internet of Vehicles task offloading model include:

[0060] (1) Local smart vehicles: Smart vehicles have computing power and communication equipment. They can not only perform local tasks, but also act as edge servers to collaborate with neighboring vehicles to complete tasks. They can also perceive the environment, collect data, participate in training, and make task offloading decisions.

[0061] (2) Roadside Unit (RSU): RSU is a device with base station functions deployed on both sides of the road, providing communication support to assist in task offloading and result feedback of intelligent vehicles.

[0062] (3) Edge servers: including MEC and vehicle edge servers. MEC refers to the server deployed next to the roadside unit RSU, which mainly provides computing support; vehicle edge server refers to the intelligent vehicle that provides edge computing services and collaborates with other vehicles to complete their tasks.

[0063] The Internet of Vehicles task offloading model consists of the following three stages:

[0064] (1) Task generation: Local intelligent vehicles continuously generate computing tasks while driving on the road (including: safety tasks, such as traffic anomalies, road hazards, and autonomous driving; entertainment tasks, such as movies, short videos, VR experiences, and games).

[0065] (2) Task analysis and offloading decision: The local intelligent vehicle makes intelligent decisions on the current pending tasks based on the perceived environmental status: whether to offload, through which link channel to offload to which edge server, how much computing resources the edge server needs to provide, etc.

[0066] (3) Task calculation and completion: Based on the decision results, the target provides the corresponding computing resources to complete the task calculation; if the calculation is completed by the edge server, the calculation result needs to be transmitted back to the task generating vehicle.

[0067] In this embodiment, a VEC-based vehicle networking task offloading model on a highway is investigated, such as Figure 2 As shown. RSU base stations are deployed on the roadside, and MEC edge servers are deployed next to RSUs. It is assumed that the coverage of RSUs does not overlap. Local smart vehicles can perform local tasks or serve as edge servers to assist other vehicles in completing tasks. Vehicle tasks can be performed locally, or offloaded to a certain RSU through the cellular V2I uplink, or offloaded to neighboring vehicles through the V2V link. V2I vehicles communicate with the base station RSU through orthogonal frequency division multiplexing technology, and communications between different V2I vehicles do not interfere with each other. In order to improve spectrum utilization, V2V vehicles reuse V2I communication spectrum resources. However, this also brings interference. There may be signal interference from multiple V2V vehicles on the V2I link, and there may be signal interference from V2I vehicles and other V2V vehicles on the V2V link. There is interference on both the V2I link and the V2V link, resulting in reduced communication quality and low task offloading efficiency. How to allocate Internet of Vehicles resources to reduce the impact of interference and improve task offloading efficiency is one of the purposes of task offloading.

[0068] A further implementation method is to construct a vehicle networking task offloading model by:

[0069] Based on the signal transmission time of different task offloading links, a communication model of task offloading is constructed; wherein the task offloading link includes a V2I link and a V2V link; in this embodiment, the method of constructing the communication model is as follows:

[0070] Without loss of generality, assume that there are J V2I links, represented by the set Φ = {1, 2, 3, ..., J}; there are K V2V links, represented by the set Ψ = {1, 2, 3, ... ., K}; there are M orthogonal subchannels, represented by the set Θ = {1, 2, 3, ..., M}. Assume that the transmission characteristics of the V2I and V2V communication channels are mainly determined by the channel loss, which can be predicted by the Friis Model. It can be seen that the signal transmission models of the jth V2I link and the kth V2V link are

[0071]

[0072] and

[0073]

[0074] Among them, X j and X k They represent the signals sent by the jth V2I link and the kth V2V link, and their transmission powers are ; They represent the channel gains of the jth V2I link, the link from the kth V2V transmitter to the RSU, the kth V2V link, the link from the jth V2I transmitter to the kth V2V receiver, and the link from the k'th V2V transmitter to the kth V2V receiver, respectively, and they can all be characterized by the Freese model; is an indicator function with a value of 0 or 1: 1 indicates that the k-th V2V link multiplexes the m-th subchannel, and 0 indicates that the k-th V2V link does not occupy the m-th subchannel; N B and N v They represent the receiver noise respectively, and both obey the noise Then, the signal-to-interference-plus-noise ratio (SINR) of the jth V2I link and the kth V2V link are respectively

[0075] and

[0076] in, represents the interference of all V2V vehicles on the jth V2I link, represents the interference of other V2V vehicles to the kth V2V link. Using the Shannon channel capacity formula, the channel capacities of the jth V2I link and the kth V2V link can be calculated as

[0077] and

[0078] Where W represents the total spectrum bandwidth. Therefore, when vehicle i needs to offload the task at the current time t to the MEC server or other vehicle edge servers, the time required for task offloading transmission is

[0079] or

[0080] in, Represents the size of the current task. After the edge server completes the task, its result will be transmitted back to vehicle i. Compared with task offloading, the result feedback task is very small, so it is assumed that the transmission time can be ignored.

[0081] Based on the computing time on different task offloading objects, a computing model for task offloading is constructed; wherein the task offloading objects include local intelligent vehicles, MEC edge servers, and vehicle edge servers; in this embodiment, the method for constructing the computing model is as follows:

[0082] When computing tasks, computing resources are no longer allocated evenly, but the idle resources with appropriate proportions are intelligently selected to perform the calculations according to the task latency requirements, so that computing resources can be flexibly allocated, which is both efficient and avoids resource waste. Assuming that the task of intelligent vehicle i at time t is executed locally, the required computing time is

[0083]

[0084] in, Indicates the number of CPU rounds required to complete the current task. represents the current local idle resources required by vehicle i to complete the current task Similarly, if the task of vehicle i at time t is offloaded to the edge server to complete, then the time to offload to the MEC server or neighbor vehicle k for calculation is

[0085] or

[0086] in, represents the current idle resources that the MEC server needs to provide to complete the task of vehicle i at time t The proportion of represents the current idle resources that neighbor vehicle k needs to provide to complete the task of vehicle i at time t % of.

[0087] Based on the communication model and the computing model, a vehicle networking task offloading model is obtained.

[0088] S2: Based on the Internet of Vehicles task offloading model, the joint optimization problem of task offloading and resource allocation is obtained;

[0089] A further implementation method is to minimize the total execution delay of local intelligent vehicle tasks based on the Internet of Vehicles task offloading model, and obtain the method for the joint optimization problem of task offloading and resource allocation as follows:

[0090] Analyze the Internet of Vehicles task offloading model to obtain the task offloading delay of the preset vehicle at the preset time;

[0091] Using multiple constraints to constrain task delays, we can obtain the joint optimization problem of task offloading and resource allocation.

[0092] In this embodiment, based on the analysis of the above communication model and calculation model, the task delay of vehicle i at time t is given as:

[0093]

[0094] in, Indicates the selection of the offloading object: the current task is to be executed locally, offloaded to the MEC server, or offloaded to a certain vehicle k. Only one method can be selected to execute, that is, satisfy The purpose of the task offloading described in the present invention is to enable all tasks to be effectively executed within their respective time thresholds in the shortest possible time. To this end, the present invention provides the optimization problem of task offloading in the Internet of Vehicles as follows:

[0095] ,

[0096] in, represents the unloading action of all tasks of all vehicles, and the unloading action of vehicle i on the current task at time t Contains three sub-actions, namely the execution target selection action of the task , channel selection action for task offloading Select actions based on the computing power resource ratio ; The indicator function representing the mth subchannel unloading task reused by vehicle i at time t, is the preset minimum signal-to-interference-noise ratio value of the offload link, is the delay constraint of the task of vehicle i at time t;

[0097] The multiple constraints of the Internet of Vehicles task offloading optimization problem described in the present invention include c1-c7:

[0098] Constraints c1 and c2 indicate that each task can only choose one of the three task offloading objects: the local smart vehicle, the MEC edge server, and the neighboring vehicle edge server to perform the computing task;

[0099] Constraint c3 means that only one subchannel can be selected for transmission each time offloading;

[0100] Constraint c4 represents all idle resources of the MEC edge server that can provide edge services at time t;

[0101] Constraint c5 represents all resources that each vehicle can provide computing services at time t;

[0102] Constraint c6 means that vehicle i can successfully transmit the task at time t whether it is offloaded to the MEC edge server or to the neighboring vehicle edge server;

[0103] Constraint c7 means that each task is executed within the delay limit.

[0104] S3: The joint optimization problem is transformed into a Markov decision process, and the four elements of multi-agent deep reinforcement learning are obtained. Each vehicle is an agent that can make independent decisions and optimize decisions through learning; the current observation of the agent is a state, including the agent's current self-information (including vehicle position, speed, driving direction and idle resources, etc.), pending task information (including task size, required resources and time limit, etc.), neighbor vehicle information and neighboring RSU information; the actions include channel selection, offloading target selection and computing resource ratio selection; the reward function is determined by constraints such as task execution delay and related communication quality, available computing resources, etc.

[0105] A further implementation method is to transform the joint optimization problem into a Markov decision process:

[0106] Each local intelligent vehicle is regarded as an intelligent agent. According to the observed state, the intelligent agent finds a joint optimization solution for task offloading and resource allocation to perform the offloading action and obtain the reward.

[0107] The status includes the vehicle's own information, the current task information, the information of neighboring vehicles within the current vehicle's communication range, and the task roadside unit RSU, which is one of the entities in the Internet of Vehicles task offloading model.

[0108] Specifically, in this embodiment, the Markov decision process includes three elements: state space, action space and reward function. Each vehicle is regarded as an intelligent agent, and the vehicle will find a strategy to perform actions according to the observed state, so as to obtain rewards.

[0109] State space: Use represents the state observed by agent i at time t ,in Represents important information about the local smart vehicle, including location, speed, available resources, and available transmission interfaces. Indicates the current task information, namely the task size, number of CPU rounds required, and time limit requirements. Represents the information of neighboring vehicles within its communication range, Represents the information of nearby RSUs, including the location of the RSU, the number of tasks currently processed by its MEC server, and the available computing resources.

[0110] Action space: The agent performs actions based on the observed states , each action includes three choices: the choice of task execution target , Selection of offloading channel And the choice of computing power resource ratio .

[0111] Reward function: The task offloading solution of the present invention designs the following reward function

[0112]

[0113] in, It is a hyperparameter used to limit the value range of the reward function; is an indicator function that takes values ​​of 0 or 1; in particular, setting the parameter ,in It indicates the penalty when the computing resources provided by the server are insufficient. It indicates the penalty for unsuccessful offload transmission due to a low signal-to-interference-to-noise ratio of the offload link. When the task execution time exceeds the maximum time limit, a penalty is imposed. When the task execution time does not exceed the maximum time limit, the penalty increases with the increase of the delay.

[0114] S4: The A-MADDPG algorithm based on the self-attention mechanism is used to solve the Markov decision process and obtain a joint optimization solution; the unloading task optimization problem given in the present invention is a multi-constraint optimization problem and also a difficult problem. In particular, in the dynamic changing environment of multiple vehicles with high mobility in the Internet of Vehicles, multi-agent deep reinforcement learning can give full play to multi-agent collaboration and continuously optimize strategies and network parameters through interactive learning between agents and the environment.

[0115] A further implementation method is to introduce a type of Transformer module consisting of a fully connected layer, a multi-head self-attention mechanism layer and a fully connected layer into the multi-agent deep deterministic policy gradient MADDPG algorithm, wherein the multi-head self-attention mechanism layer and the subsequent fully connected layer constitute the encoder layer. To simplify the network architecture representation, this Transformer module is called the SAME layer, such as Figure 3 As shown; the framework of the A-MADDPG algorithm based on the self-attention mechanism is constructed; the A-MADDPG algorithm is a multi-agent deep reinforcement learning algorithm based on the Actor-Critic network structure and the idea of ​​centralized training and distributed execution, which is suitable for solving the offloading task optimization problem of the present invention.

[0116] like Figure 4As shown in the figure, (a) is the Actor network structure diagram; (b) is the Critic network structure diagram. The Actor network, based on the state input observed by the agent, outputs the corresponding strategies (probability matrix) of the three sub-actions of offloading object selection, channel selection and computing resource ratio selection through convolutional neural network (CNN) and multiple Transformer module SAME layers; the Critic network extracts features based on the global state of all agents through multiple CNN layers to reduce the dimension of high-dimensional state, and then uses multiple Transformer module SAME layers to estimate the value of the three sub-actions of offloading object selection, channel selection and computing resource ratio selection, and updates them respectively through the mean square error function. According to the updated Critic network, the value of the three sub-actions of each agent is calculated, and then the policy gradient algorithm is used to update the parameters of the Actor network for each sub-action. By alternately training the Actor network and the Critic network, the performance and collaboration ability of the agent can be gradually improved.

[0117] Since A-MADDPG outputs deterministic action values, in order to avoid falling into the local optimal solution, the present invention also adopts a combination of the ε-greedy strategy and the Gaussian noise strategy. The ε-greedy strategy is used to control the proportion of random exploration, and the Gaussian noise strategy is used to increase the randomness of the strategy, so as to better balance exploration and utilization, and explore the state space more comprehensively, thereby improving the performance and stability of the intelligent agent.

[0118] A further implementation method is to use the A-MADDPG algorithm based on the self-attention mechanism to solve the Markov decision process, and the method for obtaining the joint optimization solution is:

[0119] (A1) Input: Hyperparameters such as network learning rate, network soft update, and training setting parameters are used as algorithm inputs;

[0120] (A2) Initialization: Initialize the Internet of Vehicles environment; specifically, register the information of each intelligent agent, including their initial location information and available resources; initialize channel resources and computing resources to ensure that the system has sufficient communication and computing capabilities; initialize the information of the roadside unit (RSU) and send it to each vehicle to ensure smooth communication between the vehicle and the infrastructure.

[0121] Based on the initialized Internet of Vehicles environment, the vehicle's Actor-Critic network and network parameters are initialized using the attention mechanism, convolutional neural network, and the correlation between task offloading actions;

[0122] (A3) Perform the following operations for each episode:

[0123] (A3.1) For each time instant t, perform the following operations:

[0124] (A3.1.1) Based on the random computing tasks generated by the local intelligent vehicle within the preset moving speed range, the action decision is generated through the Actor network; in this embodiment, for each intelligent agent i, according to the current state s i (t), select actions through the Actor network, and explore and correct them by combining the ε-greedy strategy and Gaussian strategy noise method, and finally determine action a i (t)= Each action includes output unloading object selection , Uninstall channel selection And the proportion of computing resources Three sub-actions;

[0125] (A3.1.2) All agents perform actions and combine all actions to obtain a new environment; that is, the local intelligent vehicle performs task offloading actions based on action decisions, interacts with the Internet of Vehicles environment, and obtains corresponding rewards;

[0126] (A3.1.3) For each agent i, calculate the reward r i (t), enter the next state s i (t+1), and the experience (s i (t), a i (t), r i (t), s i (t+1)) put into its experience pool E i middle;

[0127] The effect of the task offloading action is evaluated through the Critic network;

[0128] (A3.2) For each iteration, do the following:

[0129] (A3.2.1) For each agent i, perform the following operations:

[0130] (A3.2.1.1) From its experience pool E i Randomly extract samples of size batch_size in the extraction;

[0131] (A3.2.1.2) According to the offloading object selection, channel selection and computing resource ratio, the mean square error function of each of the three sub-actions is selected to update the parameters of the Critic network of agent i ;

[0132] (A3.2.1.3) Use the policy gradient method to update the parameters of the Actor network of agent i for the three sub-actions ;

[0133] (A3.2.1.4) Using soft update parameters , soft update the parameters of the target Actor network of agent i ;

[0134] (A4) Output: Output the target Actor network parameters of all agents .

[0135] That is, the parameters of the Actor network and the Critic network are updated alternately. When the training process converges, the optimal Actor network parameters of the local intelligent vehicle are obtained.

[0136] Based on the optimal Actor network parameters, each vehicle selects the optimal task offloading object, task offloading link and computing resource allocation to obtain a joint optimization solution.

[0137] In summary, the present invention models the optimization problem with the total execution delay of all vehicle tasks as the target. Under the constraints of task offloading transmission, delay limit and computing resource allocation, an optimal task offloading scheme based on self-attention mechanism and multi-agent reinforcement learning is designed for all vehicle tasks to meet the minimum total execution delay.

[0138] S5: Complete Internet of Vehicles task offloading and resource allocation based on the joint optimization solution.

[0139] Embodiment 2

[0140] The present invention also provides a vehicle networking task unloading and resource allocation system based on a self-attention mechanism, which uses a vehicle networking task unloading and resource allocation method based on a self-attention mechanism, comprising:

[0141] The task offloading model building module is used to build a vehicle networking task offloading model based on the vehicle networking highway environment VEC architecture;

[0142] A joint optimization problem acquisition module is used to obtain the joint optimization problem of task offloading and resource allocation based on the Internet of Vehicles task offloading model;

[0143] Problem transformation module, used to transform the joint optimization problem into a Markov decision process;

[0144] The solution module is used to solve the Markov decision process using the A-MADDPG algorithm based on the self-attention mechanism to obtain a joint optimization solution;

[0145] The task offloading module is used to complete the Internet of Vehicles task offloading and resource allocation based on the joint optimization solution.

[0146] A further implementation method is that the task offloading model building module includes:

[0147] A communication model building module, used to build a communication model of task offloading based on signal transmission time of different task offloading links; wherein the task offloading link includes a V2I link and a V2V link;

[0148] A computing model building module is used to build a computing model for task offloading based on the computing time on different task offloading objects; wherein the task offloading objects include local smart vehicles, MEC edge servers, and vehicle edge servers;

[0149] The task offloading model building module is used to obtain the Internet of Vehicles task offloading model based on the communication model and the calculation model.

[0150] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for task offloading and resource allocation in Internet of Vehicles based on self-attention mechanism, characterized in that: The following steps are involved: Based on the VEC architecture of the connected vehicle highway environment, a connected vehicle task offloading model is constructed. The method for constructing the connected vehicle task offloading model is as follows: Based on the signal transmission time of different task offloading links, a communication model of task offloading is constructed; wherein the task offloading link includes a V2I link and a V2V link; Based on the computing time on different task offloading objects, a computing model for task offloading is constructed; wherein the task offloading objects include local smart vehicles, MEC edge servers, and vehicle edge servers; Based on the communication model and the calculation model, obtaining the Internet of Vehicles task offloading model; Based on the Internet of Vehicles task offloading model, a joint optimization problem of task offloading and resource allocation is obtained; based on the Internet of Vehicles task offloading model, the total execution delay of local intelligent vehicle tasks is minimized as the goal, and a method for obtaining the joint optimization problem of task offloading and resource allocation is as follows: Analyze the Internet of Vehicles task offloading model to obtain the task offloading delay of a preset vehicle at a preset time; Using multiple constraints to constrain task offloading delay, we can obtain the joint optimization problem of task offloading and resource allocation. The multiple constraints include c1-c7: Constraints c1 and c2 indicate that each task can only choose one of the three task offloading objects: the local smart vehicle, the MEC edge server, and the neighboring vehicle edge server to perform the computing task; Constraint c3 means that only one subchannel can be selected for transmission each time offloading; Constraint c4 represents all idle resources of the MEC edge server that can provide edge services at time t; Constraint c5 represents all resources that each vehicle can provide computing services at time t; Constraint c6 means that vehicle i can successfully transmit the task at time t whether it is offloaded to the MEC edge server or to the neighboring vehicle edge server; Constraint c7 means that each task is executed within the delay limit; Converting the joint optimization problem into a Markov decision process; A-MADDPG algorithm based on self-attention mechanism is used to solve the Markov decision process and obtain a joint optimization solution; Based on the joint optimization scheme, Internet of Vehicles task offloading and resource allocation are completed.

2. The method for task offloading and resource allocation in Internet of Vehicles based on self-attention mechanism according to claim 1, characterized in that: The method of converting the joint optimization problem into a Markov decision process is: Each local intelligent vehicle is regarded as an intelligent agent, which searches for a joint optimization scheme of task offloading and resource allocation according to the observed state, performs the offloading action and obtains a reward; Among them, the status includes the vehicle's own information, the current task information, the information of neighboring vehicles within the current vehicle's communication range, and the information of the task roadside unit RSU which is one of the entities of the Internet of Vehicles task offloading model; the action includes the selection of the task execution target, the selection of the offloading channel, and the selection of the proportion of computing resources.

3. The method for task offloading and resource allocation in Internet of Vehicles based on self-attention mechanism according to claim 1, characterized in that: A type of Transformer module SAME layer consisting of a fully connected layer, a multi-head self-attention mechanism layer and a fully connected layer is introduced into the multi-agent deep deterministic policy gradient MADDPG algorithm to construct the framework of the A-MADDPG algorithm based on the self-attention mechanism; the A-MADDPG algorithm framework follows the idea of ​​​​the Actor-Critic network structure and centralized training distributed execution.

4. The method for task offloading and resource allocation in Internet of Vehicles based on self-attention mechanism according to claim 3, characterized in that: The A-MADDPG algorithm based on the self-attention mechanism is used to solve the Markov decision process, and the method for obtaining the joint optimization solution is as follows: Initialize the Internet of Vehicles environment; Based on the initialized Internet of Vehicles environment, the vehicle's Actor-Critic network and network parameters are initialized using the attention mechanism, convolutional neural network, and the correlation between task offloading actions; Based on the random computing tasks generated by local intelligent vehicles within the preset moving speed range, action decisions are generated through the Actor network; The local intelligent vehicle performs the task offloading action based on the action decision, interacts with the Internet of Vehicles environment, and obtains corresponding rewards; The effect of the task offloading action is evaluated through the Critic network; The parameters of the Actor network and Critic network are updated alternately. When the training process converges, the optimal Actor network parameters of the local intelligent vehicle are obtained. Based on the optimal Actor network parameters, each vehicle selects the optimal task offloading object, task offloading link and computing resource allocation to obtain the joint optimization solution.

5. A vehicle networking task offloading and resource allocation system based on a self-attention mechanism, applying the method described in any one of claims 1 to 4, characterized in that: include: The task offloading model building module is used to build a vehicle networking task offloading model based on the vehicle networking highway environment VEC architecture; A joint optimization problem acquisition module, used to obtain a joint optimization problem of task offloading and resource allocation based on the Internet of Vehicles task offloading model; A problem conversion module, used for converting the joint optimization problem into a Markov decision process; A solution module, used for solving the Markov decision process by using an A-MADDPG algorithm based on a self-attention mechanism to obtain a joint optimization solution; The task offloading module is used to complete the Internet of Vehicles task offloading and resource allocation based on the joint optimization solution.

6. The vehicle networking task offloading and resource allocation system based on self-attention mechanism according to claim 5 is characterized in that: The task offloading model building module includes: A communication model building module, used to build a communication model for task offloading based on signal transmission time of different task offloading links; wherein the task offloading link includes a V2I link and a V2V link; A computing model building module, used to build a computing model for task offloading based on the computing time on different task offloading objects; wherein the task offloading objects include local smart vehicles, MEC edge servers and vehicle edge servers; The task offloading model building module is used to obtain the Internet of Vehicles task offloading model based on the communication model and the computing model.

Images