Vehicle queue task offloading and resource allocation method and device in edge computing environment, equipment, medium and product

Abstract

The application discloses a vehicle queue task offloading and resource allocation method and device in an edge computing environment, equipment, a medium and a product, relates to the field of intelligent transportation, and comprises the following steps: abstractly modeling state information of a vehicle queue, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MEC; constructing a comprehensive cost objective function; based on the comprehensive cost objective function, dividing task offloading of the vehicle queue and resource scheduling into task offloading of vehicle queue members and resource allocation of a computing end; constructing a MATD3 model; training the MATD3 model; obtaining offloading decisions of the vehicle queue based on the trained MATD3 model; constructing a DDQN model; training the DDQN model; and obtaining resource allocation decisions of the computing end based on the trained DDQN model. The application can maximize the overall efficiency of the entire vehicle queue task offloading and resource allocation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent transportation, in particular to a vehicle platoon task offloading and resource allocation method, device, equipment, medium and product in an edge computing environment. BACKGROUND

[0002] Vehicle platoon task offloading and resource allocation technology aims to optimize computing and communication efficiency in a vehicle-to-everything (V2X) environment. With the development of autonomous driving and intelligent transportation systems, vehicles generate a large number of tasks that need to be processed in real time, such as path planning, environment perception, and driving decision-making. However, the computing resources of vehicles are limited and cannot independently complete all complex calculations. Through task offloading, vehicles can transfer part of the computing tasks to edge computing nodes (MEC) or other vehicles in the platoon, thereby relieving the local computing pressure. After task offloading, resource allocation further optimizes processing latency, energy consumption, and computing resource utilization by reasonably allocating computing and communication resources. This technology can significantly improve the efficiency of vehicle-to-everything systems and support more complex autonomous driving and intelligent transportation applications. SUMMARY

[0003] The purpose of the present application is to provide a vehicle platoon task offloading and resource allocation method, device, equipment, medium and product in an edge computing environment, which can maximize the overall efficiency of the entire vehicle platoon task offloading and resource allocation.

[0004] To achieve the above purpose, the present application provides the following solutions:

[0005] In a first aspect, the present application provides a vehicle platoon task offloading and resource allocation method in an edge computing environment, comprising:

[0006] Obtaining state information of a vehicle platoon in a real-time traffic network, edge computing state information, communication between vehicle platoon members, and communication between vehicle members and MECs;

[0007] Abstractly modeling the state information of the vehicle platoon, the edge computing state information, the communication between vehicle platoon members, and the communication between vehicle members and MECs;

[0008] Based on the abstractly modeled state information of the vehicle platoon, the edge computing state information, the communication between vehicle platoon members, and the communication between vehicle members and MECs, constructing a comprehensive cost objective function;

[0009] Based on the comprehensive cost objective function, dividing the task offloading and resource scheduling of the vehicle platoon into task offloading of vehicle platoon members and resource allocation of computing ends;

[0010] Constructing a MATD3 model;

[0011] training the MATD3 model;

[0012] obtaining the unloading decision of the vehicle queue based on the trained MATD3 model;

[0013] constructing a DDQN model;

[0014] training the DDQN model;

[0015] obtaining the resource allocation decision of the computing end based on the trained DDQN model.

[0016] Optionally, the constructing the comprehensive cost objective function based on the state information of the abstracted vehicle queue, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC specifically includes the following steps:

[0017] when the task of the vehicle queue member PM is local computing, calculating the local task computing time and the local task computing energy consumption;

[0018] when the task of the PM is V2V unloading, calculating the total processing time delay and energy consumption of the vehicle n in the time slot t for selecting the V2V unloading mode for the task;

[0019] when the task of the PM is V2I unloading, calculating the total processing time delay and energy consumption of the vehicle n in the time slot t for selecting the V2I unloading mode for the task;

[0020] based on the local task computing time and the local task computing energy consumption, the total processing time delay and energy consumption of the vehicle n in the time slot t for selecting the V2V unloading mode for the task, and the total processing time delay and energy consumption of the vehicle n in the time slot t for selecting the V2I unloading mode for the task, calculating the total processing delay of each vehicle and the total computing energy consumption of each vehicle;

[0021] calculating the utilization rate of the computing resources of the vehicle queue;

[0022] based on the total processing delay of each vehicle, the total computing energy consumption of each vehicle, and the utilization rate of the computing resources of the vehicle queue, constructing the comprehensive cost objective function.

[0023] Optionally, the expression of the comprehensive cost objective function is as follows:

[0024]

[0025] s.t.

[0026]

[0027] T n (t)≤τ n

[0028]

[0029] wherein, t represents a time slot; T represents dividing a time period into t uniform time slots; C PL (t) represents a comprehensive cost function; γ1, γ2, γ3 represent weight coefficients of time cost, energy consumption cost and resource utilization rate respectively; T PL (t) represents the total processing delay of each vehicle for a task; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization rate of computing resources of the vehicle queue; n represents the nth vehicle, and N represents the total number of vehicles; a n′n (t) represents the proportion of the computing capacity of the vehicle n' allocated to processing the task of the vehicle n; β mn (t) represents; T n (t) represents the total computing time of each vehicle for a task; τ n represents the maximum allowed processing delay of the nth connected vehicle in the t time slot; the distance between the queue member n and the mth MEC, and R represents the coverage service radius of the BS where each MEC is located.

[0030] Optionally, the training of the MATD3 model and the training of the DDQN model specifically include the following steps:

[0031] initializing the MATD3 model parameters and the DDQN model parameters;

[0032] initializing the experience replay pool ReplayBuffer_MATD3 of the MATD3 model and the experience replay pool ReplayBuffer_DDQN of the DDQN model;

[0033] setting the weight coefficients γ1, γ2, γ3;

[0034] initializing the vehicle queue environment; the vehicle queue environment includes tasks, communication resources and computing resources;

[0035] determining that the maximum number of rounds for the training of the MATD3 model is EP max for each round i, resetting the environment state;

[0036] in each round i, the number of time slots is T, that is, the maximum time step of each round;

[0037] for each time slot t, the MATD3 model is responsible for generating a task offloading decision X(t) = {X1(t), X2(t), …, X n (t), …, X N (t)}, performing an action and obtaining the next state of the system;

[0038] Determine the maximum number of training rounds ep for the DDQN model max ;

[0039] When training the DDQN model, in round j, the state s of each computing end is first updated j (t);

[0040] Then, the DDQN model generates the corresponding resource allocation decision action a j (t), and perform the action;

[0041] After the action is executed, the reward value r of the DDQN model is obtained j (t) and the state at the next moment;

[0042] If the cumulative reward r of the DDQN model j (t) After reaching the maximum number of rounds ep max If the model converges before the training, stop training the DDQN model, otherwise continue to the next round;

[0043] After completing the resource allocation of all entities, calculate the computing resource utilization U of the entire queue PL (t); The subject includes: N vehicles and M MECs, that is, a total of N+M subjects;

[0044] Calculate the reward value of the MATD3 model based on the results of task offloading and resource allocation;

[0045] Accumulate the reward value at each moment and calculate the cumulative reward of the system

[0046] If the accumulated rewards After reaching the maximum number of rounds EP max If the model converges before the training, the training of the MATD3 model is stopped, otherwise it continues to the next round;

[0047] Finally, the trained MATD3 model and DDQN model are obtained.

[0048] Optionally, calculate the computing resource utilization U of the entire queue PL (t) Specifically, the following formula is used:

[0049]

[0050] Among them, n represents the nth vehicle, n' represents vehicle n', N represents the total number of vehicles, and a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks.

[0051] Optionally, the reward value of the MATD3 model is expressed as follows:

[0052]

[0053] wherein, represents the reward value of the MATD3 model; T PL (t) represents the total processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization of the computing resources of the vehicle queue.

[0054] In a second aspect, the application provides a vehicle queue task offloading and resource allocation device in an edge computing environment, comprising:

[0055] An information acquisition module is configured to acquire state information of a vehicle queue in a real-time traffic network, edge computing state information, communication between members of the vehicle queue, and communication between the members and a MEC;

[0056] An abstract modeling module is configured to abstractly model the state information of the vehicle queue, the edge computing state information, the communication between members of the vehicle queue, and the communication between the members and the MEC;

[0057] A comprehensive cost objective function construction module is configured to construct a comprehensive cost objective function based on the abstractly modeled state information of the vehicle queue, the edge computing state information, the communication between members of the vehicle queue, and the communication between the members and the MEC;

[0058] A task offloading and resource scheduling division module is configured to divide, based on the comprehensive cost objective function, task offloading of the vehicle queue and resource scheduling of a computing end into task offloading of members of the vehicle queue and resource allocation of the computing end;

[0059] A MATD3 model construction module is configured to construct a MATD3 model;

[0060] A first training module is configured to train the MATD3 model;

[0061] A vehicle queue offloading decision determination module is configured to obtain, based on the trained MATD3 model, an offloading decision of the vehicle queue;

[0062] A DDQN model construction module is configured to construct a DDQN model;

[0063] A second training module is configured to train the DDQN model;

[0064] A computing end resource allocation decision determination module is configured to obtain, based on the trained DDQN model, a resource allocation decision of the computing end.

[0065] In a third aspect, the present application provides a computer device, comprising: a memory, a processor to store a computer program on the memory and executable on the processor, and the processor executes the computer program to implement the steps of the vehicle platoon task offloading and resource allocation method in the edge computing environment according to any one of the above.

[0066] In a fourth aspect, the present application provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the steps of the vehicle platoon task offloading and resource allocation method in the edge computing environment according to any one of the above.

[0067] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the vehicle platoon task offloading and resource allocation method in the edge computing environment according to any one of the above.

[0068] According to the specific embodiments provided by the present application, the present application discloses the following technical effects:

[0069] The present application provides a vehicle platoon task offloading and resource allocation method, device, equipment, medium and product in an edge computing environment, comprising: accurately modeling the vehicle platoon, vehicle task, communication, vehicle and mobile edge computing (MEC) server and other computing resources of the vehicle platoon edge computing scene; establishing a comprehensive cost target considering the utilization rate of vehicle platoon computing resources, task processing delay, and task processing energy consumption; dividing the task offloading and resource allocation of the vehicle platoon into a queue task offloading process or a computing resource allocation process; generating a task offloading decision for each vehicle in the vehicle platoon using MATD3; generating a resource allocation scheme inside each vehicle or MEC server using DDQN; training the proposed method to obtain a trained MATD3 model and DDQN model; and using the obtained MATD3 model and DDQN model for real-time task offloading and resource allocation. As can be seen, through task offloading, the vehicle can transfer part of the computing task to the edge computing node (MEC) or other vehicles in the platoon, thereby relieving the local computing pressure, and after task offloading, the resource allocation further optimizes the processing delay, energy consumption and computing resource utilization rate through reasonable allocation of computing and communication resources, significantly improving the efficiency of the Internet of Vehicles system, supporting more complex autonomous driving and intelligent traffic applications, and the present application is applicable to the task offloading and resource allocation problem of the vehicle platoon in the edge computing environment, and under the multi-objective requirements of platoon computing resource utilization rate, task processing delay, and task processing energy consumption, the present application provides the optimal task offloading decision for each vehicle in the vehicle platoon and the resource allocation decision for each vehicle or MEC server with computing resources, and maximizes the overall efficiency of the entire vehicle platoon task offloading and resource allocation. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below only illustrate some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0071] Figure 1 A flowchart of a vehicle queue task offloading and resource allocation method in an edge computing environment according to an embodiment of the present application.

[0072] Figure 2 A structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0074] In order to make the above purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0075] Figure 1 A flowchart of a vehicle queue task offloading and resource allocation method in an edge computing environment according to an embodiment of the present application. Figure 1 The method in the present application includes:

[0076] Step 1: obtaining state information of a vehicle queue in a real-time traffic network, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MECs.

[0077] Step 2: abstractly modeling the state information of the vehicle queue, the edge computing state information, the communication between vehicle queue members, and the communication between vehicle members and MECs.

[0078] For a traffic network with mobile edge computing servers where a single vehicle queue is located, the state information of the vehicle queue in the real-time traffic network, the edge computing state information, the communication between vehicle queue members, and the communication between vehicle members and MECs are abstractly modeled, specifically including the following steps:

[0079] Step 2.1: In the traffic network, MEC servers are deployed in the base stations at the roadside, which can provide computing services for the computing resource-lacking vehicles within the coverage of the base stations.

[0080] Let the set of M MEC servers be The coverage service radius of the BS where each MEC is located is R. The vehicle platoon is composed of N vehicles, and the entire vehicle platoon is denoted as the set The vehicle platoon moves at a speed v, and every two adjacent vehicles are fixedly spaced by d v The length of each vehicle is l, and the spacing between vehicles is l+d v .

[0081] Step 2.2: Divide the time period into T uniform time slots, denoted as the set At the beginning of each time slot, the platoon members (PMs) generate a task that can be offloaded, while updating their own available computing resources ( denote the available computing resources of vehicle n at time slot t, expressed in CPU processing cycles per unit time), while the MEC servers update their own computing resources available to the platoon members ( denote the current available computing resources of the mth MEC at time slot t), and the computing resources are considered to remain unchanged within each time slot but change between different time slots.

[0082] This application collectively refers to PMs or MECs with computing resources as computing ends. The task generated by each PM in each time slot is represented in the form of a three-tuple parameter, i.e. where d n (t) denotes the task size of the nth connected vehicle at time slot t, c n (t) denotes the task complexity of the nth connected vehicle at time slot t, and τ n (t) denotes the maximum allowed processing delay of the nth connected vehicle at time slot t.

[0083] Step 2.3: Define the task offloading strategy vector X(t) = {X1(t), X2(t),..., X N (t)} of the vehicle platoon at time slot t. Each X n (t) denotes the task offloading strategy of the nth vehicle at time slot t, and X n (t) is a discrete value, taking an integer value between 1 and N+M, i.e., X n (t) ∈ {1, 2,..., N, N+1,..., N+M}. Specifically:

[0084] If X n (t) = n, it means that the nth vehicle chooses local computing;

[0085] If X n (t) = n' and n'≠n, which means that the nth vehicle offloads the task to the n'th vehicle for calculation;

[0086] If X n (t) = N + m, which means that the nth vehicle offloads the task to the mth MEC for calculation.

[0087] It should be noted that each vehicle can only choose one computing end for task offloading.

[0088] Step 2.4: Define the 3D coordinates of PM and MEC. For the nth PM, its coordinates are defined as V n (t)=(x n (t),y n (t)), where x n (t),y n (t) changes with time according to the movement of the vehicle. For each MEC, it is fixed on the roadside and its coordinates are fixed values, defined as V m =(x m ,y m ).

[0089] At time slot t, the distance between vehicle member n and vehicle member n' in the queue The distance between queue member n and the mth MEC

[0090] Step 2.5: Based on the vehicle-to-vehicle (V2V) communication propagation model in the IEEE 802.11p standard, the path loss between vehicle platoon member n and vehicle platoon member n' can be calculated as: According to the LTE-V standard vehicle-to-infrastructure (V2I) communication propagation model, the path loss between vehicle platoon member n and the mth MEC can be calculated as:

[0091] The time-varying channel gains for vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) can be expressed as: and According to Shannon's theorem, the data transmission rates between V2V and V2I are: and κ, where B V2V and B V2I are the bandwidths of V2V and V2I communications, P V2V is the communication power between vehicles, P V2Iis the communication power between the vehicle and the MEC, σ 2 represents the power noise of the transmission.

[0092] Step 3: Based on the state information of the abstracted vehicle queue, the edge computing state information, the communication between the members of the vehicle queue, and the communication between the vehicle members and the MEC, an integrated cost objective function is constructed.

[0093] Specifically, the following steps are included:

[0094] Step 3.1: When the task of the PM is locally calculated, there is no task transmission delay. When the task selects V2V offloading or V2I offloading, it includes the task original data offloading transmission process, the task calculation process, and the calculation result back transmission process. Since the back data result is small, the time delay and energy consumption of the data back transmission process are ignored.

[0095] Step 3.2: When the task of vehicle n is offloaded to local calculation, a certain proportion of computing power will be allocated to handle local tasks according to the amount of computing tasks undertaken. The local calculation time of the task is and the local calculation energy consumption is which can be represented by the following formula:

[0096]

[0097] wherein, represents the available computing resources of vehicle n in time slot t (represented by the available CPU processing period per unit time); α nn is the proportion of vehicle n allocating its computing resources to the calculation of its own task; κ is the energy coefficient depending on the integrated chip structure.

[0098] Step 3.3: When the task of vehicle n is offloaded to other member n' of the queue, the transmission time delay and energy consumption in the data offloading transmission process between vehicle n and vehicle n' can be given by the following formula:

[0099]

[0100] The processing time delay and energy consumption of the task of vehicle n in the calculation process of queue member n' can be given by the following formula:

[0101]

[0102] wherein, α n'n (t) is the proportion of vehicle n' allocating its computing power to handle the task of vehicle n; is the computing power of vehicle n' in time slot t.

[0103] Therefore, the total processing time delay and energy consumption of vehicle n in time slot t in the V2V offloading mode are respectively represented as:

[0104]

[0105] Step 3.4: When the task of vehicle n is offloaded to the mthMEC nearby, the transmission delay and energy consumption in the data offloading transmission process between vehicle n and the mthMEC can be given by the following formula:

[0106]

[0107] The processing delay and energy consumption of the task of vehicle n in the computing process on the mthMEC can be given by the following formula:

[0108]

[0109] wherein β mn (t) represents the proportion of the mthMEC allocating its computing capacity to process the task of vehicle n at time slot t, represents the current available computing resource of the mthMEC at time slot t; P MEC represents the maximum rated power of the mthMEC.

[0110] Therefore, the total processing delay and energy consumption of the task of vehicle n in the V2I offloading mode at time slot t are respectively represented as:

[0111]

[0112]

[0113] Step 3.5: At time slot t, the total computing time T n (t) of the task of each vehicle can be represented as:

[0114]

[0115] The total processing delay of the task processing of all the queue members is represented as:

[0116] Step 3.6: At time slot t, the total computing energy E n (t) of the task of each vehicle can be represented as:

[0117]

[0118] The total energy consumption of the task processing of all the queue members is represented as:

[0119] Step 3.7: In the present application, the utilization rate of the computing resource of the queue is represented by the proportion of the computing task in the vehicle queue. It is assumed that at time slot t, the total number of tasks in the queue is d total (t), wherein the number of tasks completed in the vehicle queue is d PL(t). Utilization of the queue computing resource U PL (t) can be expressed as:

[0120]

[0121] Step 3.8: At time slot t, the integrated cost function C of all the queue member task processing P L(t) is defined as the weighted sum of each cost and its corresponding weight coefficient. In order to ensure the maximization of U PL (t), the inverse of U PL (t) is used in the cost function C PL (t). The integrated cost can be expressed as the following formula:

[0122]

[0123] s.t.

[0124]

[0125] T n (t)≤τ n

[0126]

[0127] Wherein, γ1, γ2, γ3 are weight coefficients of time cost, energy cost and resource utilization, respectively.

[0128] Step 4: Based on the integrated cost objective function, the task offloading and resource scheduling of the vehicle queue are divided into the task offloading of the vehicle queue members and the resource allocation of the computing end.

[0129] Specifically, according to the integrated cost objective function of step 3, the task offloading and resource scheduling problem of the vehicle queue is divided into two nested sub-problems: one is the task offloading problem of the queue members, and the other is the resource allocation problem of the computing end.

[0130] It can be seen that when the queue offloading decision is feasible, the utilization of the queue computing resource is only affected by the offloading decision. After the task offloading decision is executed, the computing end will allocate computing resources to the tasks offloaded to itself, at this time, the resource allocation decision of each computing subject will affect the delay and energy consumption targets. Therefore, it is defined that the offloading decision of the vehicle queue at time slot t is X n (t), the resource allocation decision of the vehicle n' is {α n'n (t)}, and the resource allocation decision of the mth MEC is {β mn (t)

[0131] Step 5: Construct the MATD3 model.

[0132] Based on the MATD3 model, the unloading decision of the vehicle queue is generated, and the MATD3 model considers the interaction behavior between multiple agents. For a single agent n, its network structure includes 6 networks:

[0133] Actor network Critic1 network Critic2 network Target Actor network Target Critic1 network and Target Critic2 network wherein, represents the local state observed by agent n at the current time, represents the system state at the current time, represents the action of all agents at the current time, represents the local state observed by agent n at the next time, represents the system state at the next time, represents the action of all agents at the next time; are the network parameters of the network respectively.

[0134] In the centralized training phase, the two Critic networks take the system state and the action of all agents as input to calculate the corresponding Action-Value Function Q value; in the centralized execution phase, each agent only relies on its own local observation and the Actor network to make decisions to generate actions

[0135] In the MATD3 model, the policy objective function of each agent is:

[0136]

[0137] For agent n, the objective function expression of its Critic network is as follows:

[0138]

[0139] wherein, represents the experience replay pool.

[0140] The calculation of the target Q value y i uses the Target Actor network and the Target Critic network:

[0141]

[0142] where γ is the return discount factor.

[0143] The parameters of the two Critic networks are updated by the gradient descent method of minimizing the loss function. The Actor network is updated by the expected Q value of the Q value maximization strategy of the Critic network.

[0144] Since the unloading decision is a discrete variable and the MATD3 model generally outputs a continuous variable, a softmax function is added to the output layer of the network to output a discrete variable.

[0145] Step 6: Train the MATD3 model.

[0146] Step 7: Obtain the unloading decision of the vehicle queue based on the trained MATD3 model.

[0147] Step 8: Build a DDQN model.

[0148] Based on the DDQN model, the resource allocation decision of the computing end is generated. Double Deep Q Network (DDQN) uses two sets of different network parameters θ, θ - , θ - is used to select the action corresponding to the maximum Q value, and θ - is used to evaluate the Q value of the optimal action. The target function of DDQN is as follows:

[0149]

[0150] where θ, θ - are the parameters of the evaluation network and the target network, respectively.

[0151] DDQN approximates the state-action value function (i.e., Q function) by neural network to predict the state-action value (i.e., Q value), and during training, the agent randomly samples its network parameters from the experience pool.

[0152] Step 9: Train the DDQN model.

[0153] Step 10: Obtain the resource allocation decision of the computing end based on the trained DDQN model.

[0154] The training process of the MATD3 model and the DDQN model is as follows:

[0155] Define the action, state, and reward functions of the MATD3 algorithm. Each vehicle is an agent, and the state observed by each agent is The system state is The action of each agent is X n (t), and the action set of all agents is X(t) = {X1(t), X2(t),..., Xn (t),...,X N (t)}. The reward function of each agent is the system reward function, and the system reward function is

[0156] Define the action, state, and reward function of the DDQN algorithm. The state observed by each computing end is s j (t), j = N + M. in, represents the attribute set of the task offloaded to the j-th computing terminal, {V n (t)} represents the location set of vehicles unloaded to the jth computing terminal, V j (t) represents the position of the jth computing terminal, f j (t) represents the computing resources of the jth computing terminal. The resource allocation action a of each computing terminal j (t), when j∈[1,N], a j (t) = {α jn (t)}; when j∈[N+1,M], a j (t) = {β jn (t)}. Reward function r j (t) = -(γ1T j (t)+γ2E j (t)).

[0157] First, initialize the MATD3 network parameters and DDQN network parameters;

[0158] Next, initialize the experience replay pool ReplayBuffer_MATD3 for MATD3 and the experience replay pool ReplayBuffer_DDQN for DDQN;

[0159] Set weight coefficients γ1, γ2, γ3;

[0160] Initialize the vehicle platoon environment, including tasks, communication resources, and computing resources;

[0161] Start the MATD3 training process:

[0162] The maximum number of rounds for MATD3 training is EP max For each round i, reset the environment state;

[0163] In each round i, the number of execution time slots is T, which is the maximum time step of each round. For each time slot t, MATD3 is responsible for generating the task offloading decision X(t) = {X1(t), X2(t), ..., X n (t),...,X N (t)}, execute the action and obtain the state of the system at the next moment;

[0164] At time slot t, for each computing agent (including N vehicles and M MECs, i.e., a total of N+M agents), the DDQN training process of each agent runs multiple times within each time slot, up to ep max , i.e., the maximum number of training rounds of DDQN;

[0165] When training DDQN, in round j, first update the state s j (t) of each computing end;

[0166] Then, DDQN generates the corresponding resource allocation decision action a j (t) and executes the action;

[0167] After the action is executed, the reward value r j (t) and the next time state are obtained;

[0168] If the cumulative reward r j (t) converges before reaching the maximum number of rounds ep max , stop the training of DDQN; otherwise, proceed to the next round;

[0169] After completing the resource allocation of all agents, calculate the computing resource utilization U PL (t) of the entire queue:

[0170] Based on the results of task offloading and resource allocation, calculate the reward value of MATD3

[0171] Add up the reward values at each time to calculate the cumulative reward of the system

[0172] If the cumulative reward converges before reaching the maximum number of rounds, stop the training of MATD3; otherwise, proceed to the next round;

[0173] Finally, obtain the trained MATD3 model and DDQN model.

[0174] In actual operation, use the trained MATD3 model to generate task offloading decisions in real time; for each computing agent, use the trained DDQN model to generate resource allocation decisions in real time. Apply these decisions to the vehicle queue to optimize task processing efficiency, reduce delay and energy consumption, and achieve more efficient computing resource utilization.

[0175] To sum up, the application discloses a method for vehicle queue task offloading and resource allocation in an edge computing environment, comprising the following steps: (1) accurately modeling the vehicle queue, vehicle task, communication, vehicle, and mobile edge computing (MEC) server and other computing resources of the vehicle queue edge computing scene; (2) establishing a comprehensive cost target considering the utilization rate of the vehicle queue computing resources, task processing delay, and task processing energy consumption; (3) dividing the task offloading and resource allocation of the vehicle queue into a queue task offloading process or a computing resource allocation process; (4) generating a task offloading decision for each vehicle in the vehicle queue by using MATD3, and generating a resource allocation scheme for each vehicle or MEC server by using DDQN; (5) training the proposed method to obtain a trained MATD3 model and DDQN model; and (6) using the obtained MATD3 model and DDQN model for real-time task offloading and resource allocation. The application is suitable for the task offloading and resource allocation of a vehicle queue in an edge computing environment, and provides an optimal task offloading decision for each vehicle in the vehicle queue and an optimal resource allocation decision for each vehicle or MEC server with computing resources under the multi-target requirements of the utilization rate of the queue computing resources, task processing delay, and task processing energy consumption, so that the overall performance of the task offloading and resource allocation of the entire vehicle queue is maximized.

[0176] Based on the same inventive concept, the application also provides a device for implementing the vehicle queue task offloading and resource allocation in an edge computing environment. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more device embodiments for vehicle queue task offloading and resource allocation in an edge computing environment provided below can be referred to the limitations of the vehicle queue task offloading and resource allocation method in an edge computing environment described above, which will not be repeated here.

[0177] In one exemplary embodiment, a device for vehicle queue task offloading and resource allocation in an edge computing environment is provided, comprising:

[0178] An information acquisition module is configured to acquire state information of a vehicle queue in a real-time traffic network, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MECs.

[0179] An abstract modeling module is configured to abstractly model the state information of the vehicle queue, the edge computing state information, the communication between vehicle queue members, and the communication between vehicle members and MECs.

[0180] A comprehensive cost target function construction module is configured to construct a comprehensive cost target function based on the abstractly modeled state information of the vehicle queue, the edge computing state information, the communication between vehicle queue members, and the communication between vehicle members and MECs.

[0181] a task offloading and resource scheduling division module, configured to divide, based on the comprehensive cost objective function, the task offloading and resource scheduling of the vehicle platoon into the task offloading of the vehicle platoon members and the resource allocation of the computing end;

[0182] a MATD3 model construction module, configured to construct a MATD3 model;

[0183] a first training module, configured to train the MATD3 model;

[0184] a vehicle platoon offloading decision determination module, configured to obtain, based on the trained MATD3 model, an offloading decision of the vehicle platoon;

[0185] a DDQN model construction module, configured to construct a DDQN model;

[0186] a second training module, configured to train the DDQN model;

[0187] a computing end resource allocation decision determination module, configured to obtain, based on the trained DDQN model, a resource allocation decision of the computing end.

[0188] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram of the computer device can be as shown in Figure 2 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store vehicle platoon task offloading and resource allocation data in an edge computing environment. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a vehicle platoon task offloading and resource allocation method in an edge computing environment.

[0189] Those skilled in the art can understand that, Figure 2The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0190] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0191] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0192] In an exemplary embodiment, a computer program product is provided, including a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0193] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0194] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0195] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0196] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0197] The principles and implementation modes of the present application are described by applying specific examples herein. The above description of the embodiments is only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A method for vehicle queue task offloading and resource allocation in an edge computing environment, characterized in that: The method for offloading vehicle queue tasks and allocating resources in the edge computing environment includes: Obtain status information of vehicle queues within the real-time traffic network, edge computing status information, communication between vehicle queue members, and communication between vehicle members and MEC; Abstract modeling of the vehicle platoon status information, edge computing status information, communication between vehicle platoon members, and communication between vehicle members and MEC; A comprehensive cost objective function is constructed based on the state information of the abstractly modeled vehicle platoon, edge computing state information, communication between vehicle platoon members, and communication between vehicle members and MEC. Based on the comprehensive cost objective function, the task offloading and resource scheduling of the vehicle queue are divided into task offloading of vehicle queue members and resource allocation of the computing end; Construct the MATD3 model; Training the MATD3 model; Based on the trained MATD3 model, the unloading decision of the vehicle queue is obtained; Build the DDQN model; Training the DDQN model; Obtain resource allocation decisions on the computing side based on the trained DDQN model; The construction of a comprehensive cost objective function based on the state information of the abstractly modeled vehicle platoon, edge computing state information, communication between vehicle platoon members, and communication between vehicle members and MEC specifically includes the following steps: When the task of the vehicle platoon member PM is local computing, the local task computing time and local task computing energy consumption are calculated; When the PM's task is V2V offloading, calculate the total processing delay and energy consumption when vehicle n selects V2V offloading mode for the task at time slot t; When the PM's task is V2I offloading, calculate the total processing delay and energy consumption when vehicle n selects V2I offloading mode for the task at time slot t; Calculate the total task processing delay of each vehicle and the total computing energy consumption of each vehicle based on the local task computing time and local task computing energy consumption, the total processing delay and energy consumption of vehicle n selecting the V2V offloading mode for the task at time slot t, and the total processing delay and energy consumption of vehicle n selecting the V2I offloading mode for the task at time slot t; Calculate the utilization of computing resources of vehicle queues; Constructing a comprehensive cost objective function based on the total task processing delay of each vehicle, the total computing energy consumption of each vehicle, and the utilization rate of the vehicle fleet computing resources; The expression of the comprehensive cost objective function is as follows: st T n (t)≤τ n Where t represents the time slot; T represents the time period is divided into t uniform time slots; C PL (t) represents the comprehensive cost function; γ1, γ2, and γ3 represent the weight coefficients of time cost, energy cost, and resource utilization, respectively; T PL (t) represents the total task processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization rate of the vehicle queue computing resources; n represents the nth vehicle, N represents the total number of vehicles; a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks; β mn (t) indicates; T n (t) represents the total computation time of each vehicle’s task; τ n represents the maximum allowable processing delay of the task of the nth connected vehicle in time slot t; It represents the distance between queue member n and the mth MEC, and R represents the coverage service radius of the BS where each MEC is located.

2. The method for vehicle queue task offloading and resource allocation in an edge computing environment according to claim 1 is characterized in that: Training the MATD3 model and the DDQN model specifically include the following steps: Initialize MATD3 model parameters and DDQN model parameters; Initialize the experience replay pool ReplayBuffer_MATD3 of the MATD3 model and the experience replay pool ReplayBuffer_DDQN of the DDQN model; Set weight coefficients γ1, γ2, γ3; Initializing a vehicle platoon environment; the vehicle platoon environment includes tasks, communication resources, and computing resources; Determine the maximum number of rounds for MATD3 model training to be EP max , for each round i, reset the environment state; In each round i, the number of execution time slots is T, which is the maximum time step of each round; For each time slot t, the MATD3 model is responsible for generating the task offloading decision X(t) = {X1(t), X2(t), ..., X n (t),...,X N (t)}, execute the action and obtain the state of the system at the next moment; Determine the maximum number of training rounds ep for the DDQN model max ; When training the DDQN model, in round j, the state s of each computing end is first updated j (t); Then, the DDQN model generates the corresponding resource allocation decision action a j (t), and perform the action; After the action is executed, the reward value r of the DDQN model is obtained j (t) and the state at the next moment; If the cumulative reward r of the DDQN model j (t) After reaching the maximum number of rounds ep max If the model converges before the training, stop training the DDQN model, otherwise continue to the next round; After completing the resource allocation of all entities, calculate the computing resource utilization U of the entire queue PL (t); The subject includes: N vehicles and M MECs, that is, a total of N+M subjects; Calculate the reward value of the MATD3 model based on the results of task offloading and resource allocation; Accumulate the reward value at each moment and calculate the cumulative reward of the system If the accumulated rewards After reaching the maximum number of rounds EP max If the model converges before the training, the training of the MATD3 model is stopped, otherwise it continues to the next round; Finally, the trained MATD3 model and DDQN model are obtained.

3. The method for vehicle queue task offloading and resource allocation in an edge computing environment according to claim 2, characterized in that: Calculate the computing resource utilization U of the entire queue PL (t) Specifically, the following formula is used: Among them, n represents the nth vehicle, n' represents vehicle n', N represents the total number of vehicles, and a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks.

4. The method for vehicle queue task offloading and resource allocation in an edge computing environment according to claim 2, characterized in that: The expression of the reward value of the MATD3 model is as follows: in, Represents the reward value of the MATD3 model; T PL (t) represents the total task processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization of computing resources of the vehicle queue.

5. A vehicle queue task unloading and resource allocation device in an edge computing environment, characterized in that: The vehicle queue task unloading and resource allocation device in the edge computing environment includes: The information acquisition module is used to obtain the status information of the vehicle queue in the real-time traffic network, the edge computing status information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC; An abstract modeling module is used to abstractly model the state information of the vehicle queue, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC; A comprehensive cost objective function construction module is used to construct a comprehensive cost objective function based on the state information of the abstractly modeled vehicle queue, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MEC; a task offloading and resource scheduling division module, configured to divide the task offloading and resource scheduling of the vehicle queue into task offloading of the vehicle queue members and resource allocation of the computing end based on the comprehensive cost objective function; MATD3 model building module, used to build MATD3 model; A first training module is used to train the MATD3 model; The vehicle queue unloading decision determination module is used to obtain the unloading decision of the vehicle queue based on the trained MATD3 model; DDQN model construction module, used to build the DDQN model; A second training module, used to train the DDQN model; The resource allocation decision determination module on the computing side is used to obtain the resource allocation decision on the computing side based on the trained DDQN model; The construction of a comprehensive cost objective function based on the state information of the abstractly modeled vehicle platoon, edge computing state information, communication between vehicle platoon members, and communication between vehicle members and MEC specifically includes the following steps: When the task of the vehicle platoon member PM is local computing, the local task computing time and local task computing energy consumption are calculated; When the PM's task is V2V offloading, calculate the total processing delay and energy consumption when vehicle n selects V2V offloading mode for the task at time slot t; When the PM's task is V2I offloading, calculate the total processing delay and energy consumption when vehicle n selects V2I offloading mode for the task at time slot t; Calculate the total task processing delay of each vehicle and the total computing energy consumption of each vehicle based on the local task computing time and local task computing energy consumption, the total processing delay and energy consumption of vehicle n selecting the V2V offloading mode for the task at time slot t, and the total processing delay and energy consumption of vehicle n selecting the V2I offloading mode for the task at time slot t; Calculate the utilization of computing resources of vehicle queues; Constructing a comprehensive cost objective function based on the total task processing delay of each vehicle, the total computing energy consumption of each vehicle, and the utilization rate of the vehicle fleet computing resources; The expression of the comprehensive cost objective function is as follows: st Where t represents the time slot; T represents the time period is divided into t uniform time slots; C PL (t) represents the comprehensive cost function; γ1, γ2, and γ3 represent the weight coefficients of time cost, energy cost, and resource utilization, respectively; T PL (t) represents the total task processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization rate of the vehicle queue computing resources; n represents the nth vehicle, N represents the total number of vehicles; a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks; β mn (t) indicates; T n (t) represents the total computation time of each vehicle’s task; τ n represents the maximum allowable processing delay of the task of the nth connected vehicle in time slot t; It represents the distance between queue member n and the mth MEC, and R represents the coverage service radius of the BS where each MEC is located.

6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that the processor executes the computer program to implement the steps of the vehicle queue task unloading and resource allocation method in the edge computing environment of any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the vehicle queue task unloading and resource allocation method in the edge computing environment described in any one of claims 1 to 4 are implemented.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the vehicle queue task unloading and resource allocation method in the edge computing environment described in any one of claims 1 to 4 are implemented.

Images