Vehicle queue task unloading and resource allocation method and device in edge computing environment, equipment, medium and product

Abstract

The invention discloses a vehicle queue task unloading and resource allocation method and device in an edge computing environment, equipment, a medium and a product, and relates to the field of intelligent transportation, and the method comprises the steps: carrying out the abstract modeling of the state information of a vehicle queue, the state information of edge computing, the communication between the members of the vehicle queue, and the communication between the members of the vehicle and an MEC; constructing a comprehensive cost objective function; based on the comprehensive cost objective function, task unloading and resource scheduling of the vehicle queue are divided into task unloading of vehicle queue members and resource allocation of a computing end; an MATD3 model is constructed; the MATD3 model is trained; obtaining an unloading decision of the vehicle queue based on the trained MATD3 model; a DDQN model is constructed; the DDQN model is trained; and obtaining a resource allocation decision of the computing end based on the trained DDQN model. According to the method, the overall efficiency of task unloading and resource allocation of the whole vehicle queue can be realized to the greatest extent.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation, and in particular to a method, device, equipment, medium and product for vehicle queue task unloading and resource allocation in an edge computing environment. Background Art

[0002] Vehicle platoon task offloading and resource allocation technology aims to optimize computing and communication efficiency in vehicle-to-everything (V2X) environments. With the development of autonomous driving and intelligent transportation systems, vehicles generate a large number of tasks that require real-time processing, such as path planning, environmental perception, and driving decisions. However, the vehicle's own computing resources are limited and cannot complete all complex calculations independently. Through task offloading, vehicles can transfer part of the computing tasks to edge computing nodes (MEC) or other vehicles in the platoon, thereby alleviating local computing pressure. Resource allocation further optimizes processing latency, energy consumption, and computing resource utilization by reasonably allocating computing and communication resources after task offloading. This technology can significantly improve the efficiency of vehicle networking systems and support more complex autonomous driving and intelligent transportation applications. Summary of the invention

[0003] The purpose of this application is to provide a method, device, equipment, medium and product for vehicle queue task unloading and resource allocation in an edge computing environment, which can maximize the overall efficiency of the entire vehicle queue task unloading and resource allocation.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a method for unloading vehicle queue tasks and allocating resources in an edge computing environment, including:

[0006] Obtain status information of vehicle queues in real-time traffic networks, edge computing status information, communication between vehicle queue members, and communication between vehicle members and MEC;

[0007] Abstract modeling of the state information of the vehicle queue, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MEC;

[0008] A comprehensive cost objective function is constructed based on the state information of the abstractly modeled vehicle queue, the edge computing state information, the communication between vehicle queue members, and the communication between vehicle members and MEC;

[0009] Based on the comprehensive cost objective function, the task offloading and resource scheduling of the vehicle queue are divided into the task offloading of the vehicle queue members and the resource allocation of the computing end;

[0010] Construct the MATD3 model;

[0011] Training the MATD3 model;

[0012] Based on the trained MATD3 model, the unloading decision of the vehicle queue is obtained;

[0013] Build the DDQN model;

[0014] Training the DDQN model;

[0015] The resource allocation decision on the computing side is obtained based on the trained DDQN model.

[0016] Optionally, the construction of a comprehensive cost objective function based on the state information of the vehicle queue after abstract modeling, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC specifically includes the following steps:

[0017] When the task of the vehicle platoon member PM is local computing, the local task computing time and local task computing energy consumption are calculated;

[0018] When the PM’s task is V2V unloading, the total processing delay and energy consumption of vehicle n selecting V2V unloading mode for the task in time slot t are calculated;

[0019] When the PM's task is V2I offloading, the total processing delay and energy consumption of vehicle n selecting V2I offloading mode for the task in time slot t are calculated;

[0020] Calculate the total task processing delay of each vehicle and the total computing energy consumption of each vehicle based on the local task computing time and the local task computing energy consumption, the total processing delay and energy consumption of vehicle n selecting the V2V unloading mode for the task in time slot t, and the total processing delay and energy consumption of vehicle n selecting the V2I unloading mode for the task in time slot t;

[0021] Calculate the utilization of computing resources of the vehicle queue;

[0022] A comprehensive cost objective function is constructed based on the total task processing delay of each vehicle, the total computing energy consumption of each vehicle and the utilization rate of the computing resources of the vehicle queue.

[0023] Optionally, the expression of the comprehensive cost objective function is as follows:

[0024]

[0025] st

[0026]

[0027] T n (t)≤τ n

[0028]

[0029] Where t represents the time slot; T represents dividing the time period into t uniform time slots; C PL (t) represents the comprehensive cost function; γ1, γ2, and γ3 represent the weight coefficients of time cost, energy cost, and resource utilization, respectively; T PL (t) represents the total task processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization rate of the vehicle queue computing resources; n represents the nth vehicle, N represents the total number of vehicles; a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks; β mn (t) indicates; T n (t) represents the total task computing time of each vehicle; τ n represents the maximum allowed processing delay of the task of the nth connected vehicle in time slot t; the distance between queue member n and the mth MEC, and R represents the coverage service radius of the BS where each MEC is located.

[0030] Optionally, training the MATD3 model and training the DDQN model specifically include the following steps:

[0031] Initialize MATD3 model parameters and DDQN model parameters;

[0032] Initialize the experience replay pool ReplayBuffer_MATD3 of the MATD3 model and the experience replay pool ReplayBuffer_DDQN of the DDQN model;

[0033] Set weight coefficients γ1, γ2, γ3;

[0034] Initializing a vehicle platoon environment; the vehicle platoon environment includes tasks, communication resources, and computing resources;

[0035] Determine the maximum number of rounds for MATD3 model training to be EP max , for each round i, reset the environment state;

[0036] In each round i, the number of execution time slots is T, which is the maximum time step of each round;

[0037] For each time slot t, the MATD3 model is responsible for generating the task offloading decision X(t) = {X1(t),X2(t),...,X n (t),...,X N (t)}, execute the action and obtain the state of the system at the next moment;

[0038] Determine the maximum number of training rounds ep for the DDQN model max ;

[0039] When training the DDQN model, in round j, first update the state s of each computing end j (t);

[0040] Then, the DDQN model generates the corresponding resource allocation decision action a j (t), and perform the action;

[0041] After the action is executed, the reward value r of the DDQN model is obtained j (t) and the state at the next moment;

[0042] If the cumulative reward r of the DDQN model j (t) When the maximum number of rounds ep is reached max If the model converges before the training, stop the training of the DDQN model, otherwise continue to the next round;

[0043] After completing the resource allocation of all entities, calculate the computing resource utilization U of the entire queue PL (t); The subject includes: N vehicles and M MECs, that is, a total of N+M subjects;

[0044] Based on the results of task offloading and resource allocation, the reward value of the MATD3 model is calculated;

[0045] Accumulate the reward value at each moment and calculate the cumulative reward of the system

[0046] If the accumulated rewards After reaching the maximum number of rounds EP max If the training fails to converge, the training of the MATD3 model is stopped, otherwise the training continues to the next round;

[0047] Finally, the trained MATD3 model and DDQN model are obtained.

[0048] Optionally, calculate the computing resource utilization U of the entire queue PL (t) The specific formula is as follows:

[0049]

[0050] Where n represents the nth vehicle, n' represents vehicle n', N represents the total number of vehicles, and a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks.

[0051] Optionally, the reward value of the MATD3 model is expressed as follows:

[0052]

[0053] in, Represents the reward value of the MATD3 model; T PL (t) represents the total task processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization rate of vehicle platoon computing resources.

[0054] In a second aspect, the present application provides a vehicle queue task unloading and resource allocation device in an edge computing environment, comprising:

[0055] An information acquisition module is used to obtain status information of vehicle queues in a real-time traffic network, edge computing status information, communication between vehicle queue members, and communication between vehicle members and MEC;

[0056] An abstract modeling module, used for abstract modeling of the state information of the vehicle queue, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC;

[0057] A comprehensive cost objective function building module is used to build a comprehensive cost objective function based on the state information of the vehicle queue after abstract modeling, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC;

[0058] A task offloading and resource scheduling division module, used for dividing the task offloading and resource scheduling of the vehicle queue into task offloading of the members of the vehicle queue and resource allocation of the computing end based on the comprehensive cost objective function;

[0059] MATD3 model building module, used to build MATD3 model;

[0060] A first training module, used for training the MATD3 model;

[0061] The unloading decision determination module of the vehicle queue is used to obtain the unloading decision of the vehicle queue based on the trained MATD3 model;

[0062] DDQN model building module, used to build the DDQN model;

[0063] A second training module, used to train the DDQN model;

[0064] The resource allocation decision determination module of the computing end is used to obtain the resource allocation decision of the computing end based on the trained DDQN model.

[0065] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the vehicle queue task unloading and resource allocation method in an edge computing environment described above.

[0066] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle queue task unloading and resource allocation method in the edge computing environment described above.

[0067] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the vehicle queue task unloading and resource allocation method in any of the above-mentioned edge computing environments.

[0068] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0069] The present application provides a method, apparatus, device, medium and product for vehicle queue task offloading and resource allocation in an edge computing environment, including: accurately modeling vehicle queues, vehicle tasks, communications, vehicles and computing resources such as mobile edge computing (MEC) servers in a vehicle queue edge computing scenario; establishing a comprehensive cost target that considers vehicle queue computing resource utilization, task processing latency, and task processing energy consumption; dividing the task offloading and resource allocation of vehicle queues into a queue task offloading process or a computing resource allocation process; using MATD3 to generate a task offloading decision for each vehicle in the vehicle queue; using DDQN to generate a resource allocation plan within each vehicle or MEC server; performing network training on the proposed method to obtain a trained MATD3 model and DDQN model; the obtained MATD3 model and DDQN model are used for real-time task offloading and resource allocation. It can be seen that through task offloading, the present application allows vehicles to transfer part of the computing tasks to edge computing nodes (MEC) or other vehicles in the queue, thereby alleviating local computing pressure. After task offloading, resource allocation further optimizes processing latency, energy consumption and computing resource utilization by reasonably allocating computing and communication resources, significantly improving the efficiency of the Internet of Vehicles system and supporting more complex autonomous driving and intelligent transportation applications. In addition, the present application is applicable to the task offloading and resource allocation problems of vehicle queues in edge computing environments. Under the requirements of multiple objectives such as queue computing resource utilization, task processing latency, and task processing energy consumption, the present application provides the optimal task offloading decision for each vehicle in the vehicle queue and the resource allocation decision for each vehicle or MEC server with computing resources, thereby maximizing the overall efficiency of task offloading and resource allocation for the entire vehicle queue. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 This is a flow chart of a method for unloading vehicle queue tasks and allocating resources in an edge computing environment in one embodiment of the present application;

[0072] Figure 2 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0074] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0075] Figure 1 This is a flow chart of a method for unloading vehicle queue tasks and allocating resources in an edge computing environment in one embodiment of the present application, see Figure 1 , the method of the present invention comprises:

[0076] Step 1: Obtain the status information of the vehicle queue in the real-time traffic network, the edge computing status information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC.

[0077] Step 2: Abstract modeling of the vehicle queue state information, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MEC.

[0078] For a traffic network with a mobile edge computing server where a single vehicle queue is located, an abstract model is performed based on the real-time status information of the vehicle queue in the traffic network, the edge computing status information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC. The specific steps include:

[0079] Step 2.1: In the traffic network, the MEC server is deployed in the base station on the roadside, which can provide computing services for vehicles with insufficient computing resources within the coverage area of ​​the base station.

[0080] Suppose the set of M MEC servers is The coverage service radius of the BS where each MEC is located is R. The vehicle queue consists of N vehicles, and the entire vehicle queue is represented by the set The vehicle queue moves at a speed v, and there is a fixed interval d between every two adjacent vehicles. v The length of each vehicle is l, and the distance between vehicles is l+d. v .

[0081] Step 2.2: Divide the time period into T uniform time slots, represented as the set At the beginning of each time slot, a vehicle platoon member (PM) generates a task that can be offloaded and updates its own available computing resources. ( represents the available computing resources of vehicle n in time slot t, expressed in CPU processing cycles per unit time), and MEC servers update their own computing resources available to queue members ( represents the currently available computing resources of the m-th MEC in time slot t). The computing resources are considered to remain unchanged within each time slot, but vary in different time slots.

[0082] In this application, PMs or MECs with computing resources are collectively referred to as computing terminals. The tasks generated by PMs in each time slot are represented in the form of triple parameters, namely Among them, d n (t) represents the task size of the nth connected vehicle in time slot t, c n (t) represents the task complexity of the nth connected vehicle in time slot t, τ n (t) The maximum allowed processing delay of the task of the nth connected vehicle in time slot t.

[0083] Step 2.3: Define the task unloading strategy vector X(t) = {X1(t), X2(t), ..., X N (t)}, each X n (t) represents the task unloading strategy of the nth vehicle at time slot t, X n (t) is a discrete value, which is an integer between [1, N+M]. n (t)∈{1,2,...,N,N+1,...,N+M}. Specifically:

[0084] If X n (t) = n, indicating that the nth vehicle chooses local calculation;

[0085] If X n (t) = n' and n'≠n, which means that the nth vehicle unloads the task to the n'th vehicle for calculation;

[0086] If X n (t) = N + m, indicating that the nth vehicle offloads the task to the mth MEC for calculation.

[0087] It should be noted that each vehicle can only choose one computing end for task offloading.

[0088] Step 2.4: Define the 3D coordinates of PM and MEC. For the nth PM, its coordinates are defined as V n (t) = (x n (t),y n (t)), where x n (t),y n (t) changes with time according to the movement of the vehicle. For each MEC, it is fixed on the roadside and its coordinates are fixed values, defined as V m =(x m ,y m ).

[0089] At time slot t, the distance between vehicle member n and vehicle member n' in the queue Distance between queue member n and the mth MEC

[0090] Step 2.5: According to the vehicle to vehicle (V2V) communication propagation model in the IEEE 802.11p standard, the path loss between vehicle platoon member n and member n' can be calculated as: According to the vehicle to infrastructure (V2I) communication propagation model of the LTE-V standard, the path loss between vehicle platoon member n and the mth MEC can be calculated as:

[0091] The time-varying channel gains of vehicle to vehicle (V2V) and vehicle to infrastructure (V2I) can be expressed as: and According to Shannon's theorem, the data transmission rates between V2V and V2I are: and κ, where B V2V and B V2I are the bandwidths of V2V and V2I communications, P V2V is the communication power between vehicles, P V2Iis the communication power between vehicle and MEC, σ 2 Represents the transmitted power noise.

[0092] Step 3: Construct a comprehensive cost objective function based on the state information of the abstractly modeled vehicle queue, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MEC.

[0093] The specific steps include:

[0094] Step 3.1: When the PM's task is calculated locally, there is no task transmission delay. When the task is offloaded by V2V or V2I, it includes the task original data offloading transmission process, task calculation process and calculation result return process. Since the returned data result is small, the delay and energy consumption of the data return process are ignored.

[0095] Step 3.2: When the task of vehicle n is offloaded to the local computing, the local computing power will be allocated a certain proportion of computing power to process the local task according to the amount of computing tasks undertaken. Task local computing time and local computing energy consumption It can be expressed by the following formula:

[0096]

[0097] in, represents the available computing resources of vehicle n in time slot t (expressed in available CPU processing cycles per unit time); α nn is the proportion of vehicle n's computing resources allocated to its own task calculations; κ is the energy coefficient that depends on the integrated chip structure.

[0098] Step 3.3: When the task of vehicle n is offloaded to another member n' of the queue, the transmission delay and energy consumption during the data offloading transmission between vehicle n and vehicle n' can be given by the following formula:

[0099]

[0100] The processing delay and energy consumption of the task of vehicle n on queue member n' can be given by the following formula:

[0101]

[0102] Among them, α n'n (t) is the proportion of vehicle n's own computing power allocated to processing vehicle n's tasks; is the computing capacity of vehicle n' in time slot t.

[0103] Therefore, the total processing delay and energy consumption of vehicle n selecting the V2V offloading mode for the task at time slot t are expressed as:

[0104]

[0105] Step 3.4: When the task of vehicle n is offloaded to the nearby m-th MEC, the transmission delay and energy consumption during the data offloading transmission between vehicle n and the m-th MEC can be given by the following formula:

[0106]

[0107] The processing delay and energy consumption of the task of vehicle n in the mth MEC can be given by the following formula:

[0108]

[0109] Among them, β mn (t) represents the proportion of the m-th MEC that allocates its computing power to process vehicle n tasks at time slot t. represents the currently available computing resources of the mth MEC in time slot t; P MEC Indicates the maximum rated power of the mth MEC.

[0110] Therefore, the total processing delay and energy consumption of vehicle n selecting the V2I offloading mode for the task at time slot t are expressed as:

[0111]

[0112]

[0113] Step 3.5: At time slot t, the total computation time of each vehicle’s task is T n (t) can be expressed as:

[0114]

[0115] The total processing delay of all queue member tasks is expressed as:

[0116] Step 3.6: At time slot t, the total computational energy consumption of each vehicle’s task is E n (t) can be expressed as:

[0117]

[0118] The total energy consumption of all queue member task processing is expressed as:

[0119] Step 3.7: In this application, the ratio of computing tasks in the vehicle queue is used to represent the utilization rate of the queue computing resources. Assume that in time slot t, the total number of tasks in the queue is d total (t), where the number of tasks completed in the vehicle queue is d PL(t). Queue computing resource utilization U PL (t) can be expressed as:

[0120]

[0121] Step 3.8: At time slot t, the comprehensive cost function C of all queue members' task processing P L(t) is defined as the weighted sum of each cost and its corresponding weight coefficient. PL (t), in the cost function C PL (t)Using U PL The inverse of (t). The comprehensive cost can be expressed as follows:

[0122]

[0123] st

[0124]

[0125] T n (t)≤τ n

[0126]

[0127] Among them, γ1, γ2, and γ3 are the weight coefficients of time cost, energy cost, and resource utilization, respectively.

[0128] Step 4: Based on the comprehensive cost objective function, the task offloading and resource scheduling of the vehicle queue are divided into the task offloading of the vehicle queue members and the resource allocation of the computing end.

[0129] Specifically, according to the comprehensive cost objective function in step 3, the task offloading and resource scheduling problem of the vehicle queue is divided into two nested sub-problems: one is the task offloading problem of the queue members, and the other is the resource allocation problem on the computing end.

[0130] It can be seen that when the queue unloading decision is feasible, the utilization rate of the queue computing resources is only affected by the unloading decision. When the task unloading decision is executed, the computing end will allocate computing resources to the tasks unloaded to itself. At this time, the resource allocation decision of each computing subject will affect the delay and energy consumption targets. Therefore, the unloading decision of the vehicle queue in time slot t is defined as X n (t), the resource allocation decision of vehicle n' is {α n'n (t)}, the resource allocation decision of the m-th MEC is {β mn (t)}

[0131] Step 5: Build the MATD3 model.

[0132] Based on the MATD3 model, the unloading decision of the vehicle queue is generated. The MATD3 model takes into account the interaction between multiple agents. For a single agent n, its network structure includes 6 networks:

[0133] Actor Network Critic1 Network Critic2 Network TargetActor Network Target Critic1 Network and Target Critic2 network in, represents the local state observed by agent n at the current moment, Indicates the current system status. represents the actions of all agents at the current moment, represents the local state observed by agent n at the next moment, Indicates the system state at the next moment. Represents the actions of all agents at the next moment; are the network parameters of the network respectively.

[0134] In the centralized training phase, the two critic networks use the system state and the actions of all agents As input, the corresponding action-value function (Action-Value Function) Q value is calculated; in the centralized execution phase, each agent only relies on its own local observation Make decisions with the Actor network and generate actions

[0135] In the MATD3 model, the policy objective function of each agent is:

[0136]

[0137] For agent n, the objective function expression of its Critic network is as follows:

[0138]

[0139] in, Represents the experience replay pool.

[0140] Target Q value y i The calculation uses the TargetActor network and the Target Critic network:

[0141]

[0142] Where γ is the return discount factor.

[0143] The parameters of the two Critic networks are updated separately by minimizing the gradient descent method of the loss function. The Q value of the Critic network is used to maximize the expected Q value of the strategy and update the Actor network.

[0144] Since the offloading decision is a discrete variable, the output of the MATD3 model is usually a continuous variable. Therefore, a softmax function is added to the output layer of the network to output a discrete variable.

[0145] Step 6: Train the MATD3 model.

[0146] Step 7: Based on the trained MATD3 model, the unloading decision of the vehicle queue is obtained.

[0147] Step 8: Build the DDQN model.

[0148] Based on the DDQN model, the resource allocation decision of the computing end is generated. The Double DQN (DDQN) uses two different sets of network parameters θ, θ - , θ is used to select the action corresponding to the maximum Q value, θ - The Q value used to evaluate the optimal action. The objective function of DDQN is as follows:

[0149]

[0150] Among them, θ, θ - are the parameters of the evaluation network and the target network, respectively.

[0151] DDQN predicts the state-action value (i.e., Q-value) by approximating the state-action value function (i.e., Q-function) through a neural network. During the training process, the agent randomly samples from the experience pool to train its network parameters.

[0152] Step 9: Train the DDQN model.

[0153] Step 10: Obtain resource allocation decisions on the computing side based on the trained DDQN model.

[0154] The training process of the MATD3 model and DDQN model is as follows:

[0155] Define the action, state, and reward function of the MATD3 algorithm. Each vehicle is an agent, and the state observed by each agent is The system status is The action of each agent is X n (t), the action set of all agents is X(t) = {X1(t),X2(t),...,Xn (t),...,X N (t)}. The reward function of each agent is the system reward function, and the system reward function is

[0156] Define the action, state, and reward function of the DDQN algorithm. The state observed by each computing end is s j (t), j = N + M. in, represents the attribute set of the task offloaded to the jth computing terminal, {V n (t)} represents the location set of vehicles unloaded to the jth computing terminal, V j (t) represents the position of the jth computing end, f j (t) represents the computing resources of the jth computing terminal. The resource allocation action a of each computing terminal j (t), when j∈[1,N], a j (t) = {α jn (t)}; when j∈[N+1,M], a j (t) = {β jn (t)}. Reward function r j (t) = -(γ1T j (t)+γ2E j (t)).

[0157] First, initialize the MATD3 network parameters and DDQN network parameters;

[0158] Next, initialize the experience replay pool ReplayBuffer_MATD3 of MATD3 and the experience replay pool ReplayBuffer_DDQN of DDQN;

[0159] Set weight coefficients γ1, γ2, γ3;

[0160] Initialize the vehicle platoon environment, including tasks, communication resources, and computing resources;

[0161] Start the MATD3 training process:

[0162] The maximum number of rounds for MATD3 training is EP max . For each round i, reset the environment state;

[0163] In each round i, the number of execution time slots is T, which is the maximum time step of each round. For each time slot t, MATD3 is responsible for generating task offloading decisions X(t) = {X1(t), X2(t), ..., X n (t),...,X N (t)}, execute the action and obtain the state of the system at the next moment;

[0164] At time slot t, for each computing agent (including N vehicles and M MECs, i.e., N+M agents in total), the DDQN training process of each agent will be run multiple times in each time slot, with a maximum of ep max times, which is the maximum number of training rounds for DDQN;

[0165] When training DDQN, in round j, first update the state s of each computing end j (t);

[0166] Then, DDQN generates the corresponding resource allocation decision action a j (t), and perform the action;

[0167] After the action is executed, the reward value r is obtained j (t) and the state at the next moment;

[0168] If the cumulative reward r j (t) When the maximum number of rounds ep is reached max If the training is not converged, stop the training of DDQN; otherwise, continue to the next round;

[0169] After completing the resource allocation of all entities, calculate the computing resource utilization U of the entire queue PL (t):

[0170] Based on the results of task offloading and resource allocation, calculate the reward value of MATD3

[0171] Accumulate the reward value at each moment and calculate the cumulative reward of the system

[0172] If the accumulated rewards If convergence occurs before the maximum number of rounds is reached, the training of MATD3 is stopped; otherwise, the next round is continued;

[0173] Finally, the trained MATD3 model and DDQN model are obtained.

[0174] In actual operation, the trained MATD3 model is used to generate task offloading decisions in real time; for each computing entity, the trained DDQN model is used to generate resource allocation decisions in real time. These decisions are applied to vehicle queues to optimize task processing efficiency, reduce latency and energy consumption, and achieve more efficient computing resource utilization.

[0175] In summary, the present application discloses a method for task offloading and resource allocation of a vehicle queue in an edge computing environment, comprising the following steps: (1) Accurately modeling the vehicle queue, vehicle tasks, communications, vehicles, and computing resources such as mobile edge computing (MEC) servers in the vehicle queue edge computing scenario. (2) Establishing a comprehensive cost target that takes into account the computing resource utilization rate, task processing delay, and task processing energy consumption of the vehicle queue; (3) Dividing the task offloading and resource allocation of the vehicle queue into a queue task offloading process or a computing resource allocation process; (4) Using MATD3 to generate a task offloading decision for each vehicle in the vehicle queue; Using DDQN to generate a resource allocation plan within each vehicle or MEC server; (5) Performing network training on the proposed method to obtain a trained MATD3 model and DDQN model; (6) The obtained MATD3 model and DDQN model are used for real-time task offloading and resource allocation. The present invention is applicable to the task offloading and resource allocation problems of vehicle queues in an edge computing environment. Under the requirements of multiple objectives such as queue computing resource utilization, task processing delay, and task processing energy consumption, it provides the optimal task offloading decision for each vehicle in the vehicle queue and the resource allocation decision for each vehicle or MEC server with computing resources, so as to maximize the overall efficiency of task offloading and resource allocation of the entire vehicle queue.

[0176] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the vehicle queue task unloading and resource allocation in the edge computing environment involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the vehicle queue task unloading and resource allocation device in one or more edge computing environments provided below can be referred to the limitations of the vehicle queue task unloading and resource allocation method in the edge computing environment above, and will not be repeated here.

[0177] In an exemplary embodiment, a vehicle queue task unloading and resource allocation device in an edge computing environment is provided, comprising:

[0178] An information acquisition module is used to obtain status information of vehicle queues in a real-time traffic network, edge computing status information, communication between vehicle queue members, and communication between vehicle members and MEC;

[0179] An abstract modeling module, used for abstract modeling of the state information of the vehicle queue, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC;

[0180] A comprehensive cost objective function building module is used to build a comprehensive cost objective function based on the state information of the vehicle queue after abstract modeling, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC;

[0181] A task offloading and resource scheduling division module, used for dividing the task offloading and resource scheduling of the vehicle queue into task offloading of the members of the vehicle queue and resource allocation of the computing end based on the comprehensive cost objective function;

[0182] MATD3 model building module, used to build MATD3 model;

[0183] A first training module, used for training the MATD3 model;

[0184] The unloading decision determination module of the vehicle queue is used to obtain the unloading decision of the vehicle queue based on the trained MATD3 model;

[0185] DDQN model building module, used to build the DDQN model;

[0186] A second training module, used to train the DDQN model;

[0187] The resource allocation decision determination module of the computing end is used to obtain the resource allocation decision of the computing end based on the trained DDQN model.

[0188] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 2 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store vehicle queue task unloading and resource allocation data in an edge computing environment. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for vehicle queue task unloading and resource allocation in an edge computing environment is implemented.

[0189] Those skilled in the art will understand that Figure 2The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0190] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0191] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0192] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0193] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0194] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0195] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0196] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for unloading vehicle queue tasks and allocating resources in an edge computing environment, characterized in that: The vehicle queue task unloading and resource allocation method in the edge computing environment includes: Obtain status information of vehicle queues in real-time traffic networks, edge computing status information, communication between vehicle queue members, and communication between vehicle members and MEC; Abstract modeling of the state information of the vehicle queue, edge computing state information, communication between vehicle queue members, and communication between vehicle members and MEC; A comprehensive cost objective function is constructed based on the state information of the abstractly modeled vehicle platoon, the edge computing state information, the communication between vehicle platoon members, and the communication between vehicle members and MEC; Based on the comprehensive cost objective function, the task offloading and resource scheduling of the vehicle queue are divided into the task offloading of the vehicle queue members and the resource allocation of the computing end; Construct the MATD3 model; Training the MATD3 model; Based on the trained MATD3 model, the unloading decision of the vehicle queue is obtained; Build the DDQN model; Training the DDQN model; The resource allocation decision on the computing side is obtained based on the trained DDQN model.

2. The method for vehicle queue task unloading and resource allocation in an edge computing environment according to claim 1, characterized in that: The construction of the comprehensive cost objective function based on the state information of the vehicle queue after abstract modeling, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC specifically includes the following steps: When the task of the vehicle platoon member PM is local computing, the local task computing time and local task computing energy consumption are calculated; When the PM’s task is V2V unloading, the total processing delay and energy consumption of vehicle n selecting V2V unloading mode for the task in time slot t are calculated; When the PM's task is V2I offloading, the total processing delay and energy consumption of vehicle n selecting V2I offloading mode for the task in time slot t are calculated; Calculate the total task processing delay of each vehicle and the total computing energy consumption of each vehicle based on the local task computing time and the local task computing energy consumption, the total processing delay and energy consumption of vehicle n selecting the V2V unloading mode for the task in time slot t, and the total processing delay and energy consumption of vehicle n selecting the V2I unloading mode for the task in time slot t; Calculate the utilization of computing resources of the vehicle queue; A comprehensive cost objective function is constructed based on the total task processing delay of each vehicle, the total computing energy consumption of each vehicle and the utilization rate of the computing resources of the vehicle queue.

3. The method for unloading vehicle queue tasks and allocating resources in an edge computing environment according to claim 2, characterized in that: The expression of the comprehensive cost objective function is as follows: st T n (t)≤τ n Where t represents the time slot; T represents dividing the time period into t uniform time slots; C PL (t) represents the comprehensive cost function; γ1, γ2, and γ3 represent the weight coefficients of time cost, energy cost, and resource utilization, respectively; T PL (t) represents the total task processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization rate of the vehicle queue computing resources; n represents the nth vehicle, N represents the total number of vehicles; a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks; β mn (t) indicates; T n (t) represents the total task computing time of each vehicle; τ n It represents the maximum allowed processing delay of the task of the nth connected vehicle in time slot t; It represents the distance between queue member n and the mth MEC, and R represents the coverage service radius of the BS where each MEC is located.

4. The method for vehicle queue task unloading and resource allocation in an edge computing environment according to claim 1, characterized in that: Training the MATD3 model and training the DDQN model specifically include the following steps: Initialize MATD3 model parameters and DDQN model parameters; Initialize the experience replay pool ReplayBuffer_MATD3 of the MATD3 model and the experience replay pool ReplayBuffer_DDQN of the DDQN model; Set weight coefficients γ1, γ2, γ3; Initializing a vehicle platoon environment; the vehicle platoon environment includes tasks, communication resources, and computing resources; Determine the maximum number of rounds for MATD3 model training to be EP max , for each round i, reset the environment state; In each round i, the number of execution time slots is T, which is the maximum time step of each round; For each time slot t, the MATD3 model is responsible for generating the task offloading decision X(t) = {X1(t),X2(t),...,X n (t),...,X N (t)}, execute the action and obtain the state of the system at the next moment; Determine the maximum number of training rounds ep for the DDQN model max ; When training the DDQN model, in round j, first update the state s of each computing end j (t); Then, the DDQN model generates the corresponding resource allocation decision action a j (t), and perform the action; After the action is executed, the reward value r of the DDQN model is obtained j (t) and the state at the next moment; If the cumulative reward r of the DDQN model j (t) When the maximum number of rounds ep is reached max If the model converges before the training, stop the training of the DDQN model, otherwise continue to the next round; After completing the resource allocation of all entities, calculate the computing resource utilization U of the entire queue PL (t); The subject includes: N vehicles and M MECs, that is, a total of N+M subjects; Based on the results of task offloading and resource allocation, the reward value of the MATD3 model is calculated; Accumulate the reward value at each moment and calculate the cumulative reward of the system If the accumulated rewards After reaching the maximum number of rounds EP max If the training fails to converge, the training of the MATD3 model is stopped, otherwise the training continues to the next round; Finally, the trained MATD3 model and DDQN model are obtained.

5. The method for unloading vehicle queue tasks and allocating resources in an edge computing environment according to claim 4, characterized in that: Calculate the computing resource utilization U of the entire queue PL (t) The specific formula is as follows: Where n represents the nth vehicle, n' represents vehicle n', N represents the total number of vehicles, and a n′n (t) represents the proportion of vehicle n's computing power allocated to processing vehicle n's tasks.

6. The method for vehicle queue task unloading and resource allocation in an edge computing environment according to claim 4, characterized in that: The expression of the reward value of the MATD3 model is as follows: in, Represents the reward value of the MATD3 model; T PL (t) represents the total task processing delay of each vehicle; E PL (t) represents the total computing energy consumption of each vehicle; U PL (t) represents the utilization rate of vehicle platoon computing resources.

7. A vehicle queue task unloading and resource allocation device in an edge computing environment, characterized in that: The vehicle queue task unloading and resource allocation device in the edge computing environment includes: An information acquisition module is used to obtain status information of vehicle queues in a real-time traffic network, edge computing status information, communication between vehicle queue members, and communication between vehicle members and MEC; An abstract modeling module, used for abstract modeling of the state information of the vehicle queue, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC; A comprehensive cost objective function building module is used to build a comprehensive cost objective function based on the state information of the vehicle queue after abstract modeling, the edge computing state information, the communication between the vehicle queue members, and the communication between the vehicle members and the MEC; A task offloading and resource scheduling division module, used for dividing the task offloading and resource scheduling of the vehicle queue into task offloading of the members of the vehicle queue and resource allocation of the computing end based on the comprehensive cost objective function; MATD3 model building module, used to build MATD3 model; A first training module, used for training the MATD3 model; The unloading decision determination module of the vehicle queue is used to obtain the unloading decision of the vehicle queue based on the trained MATD3 model; DDQN model building module, used to build the DDQN model; A second training module, used to train the DDQN model; The resource allocation decision determination module of the computing end is used to obtain the resource allocation decision of the computing end based on the trained DDQN model.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the vehicle queue task unloading and resource allocation method in the edge computing environment of any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the vehicle queue task unloading and resource allocation method in the edge computing environment described in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the vehicle queue task unloading and resource allocation method in the edge computing environment described in any one of claims 1 to 6 are implemented.

Images