A method for competition and cooperation computing offloading of dense heterogeneous tasks under a vehicle edge computing network

Abstract

A kind of competition and cooperation computing offloading method for dense heterogeneous tasks under vehicle edge computing network includes: 1) constructing vehicle edge computing network system;2) obtaining information data in current system;3) initializing the parameters and iteration times of system;4) initializing the state and time slice of all vehicles;5) selecting action by corresponding strategy, generating experience tuple after vehicle executes action, and storing experience tuple into replay pool;6) updating target network of system using data in replay pool, time slice plus one, returning to step 5) until time slice reaches preset threshold;7) iteration times plus one, return to step 4) until iteration times reaches preset threshold, output vehicle computing offloading method.Compared with other task offloading methods, the present application shows lower task execution delay and higher task completion rate, and provides an innovative solution for task offloading decision problem in vehicle edge computing field.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted edge computing, specifically to a method for offloading competitive and collaborative computing for dense heterogeneous tasks in a vehicle edge computing network. Background Technology

[0002] The rapid development of the Internet of Things (IoT) and artificial intelligence (AI) has greatly driven the emergence of computationally intensive and latency-sensitive vehicle applications (such as augmented reality navigation and autonomous driving). However, due to the inherent limitations of vehicle computing power, ensuring the quality of service (QoS) for vehicle applications requiring low or ultra-low latency remains challenging. Simply increasing vehicle computing resources has proven insufficient to meet the ever-changing demands of vehicle computing tasks. Over-deploying computing resources can also lead to cost challenges and low resource utilization. Vehicle-to-everything (V2X) edge computing networks, as a novel computing paradigm, improve QoS by deploying servers with computing power and resources at the network edge. In V2X edge computing networks, roadside units are deployed to provide additional computing power to resource-constrained vehicles, thereby improving resource utilization.

[0003] In current vehicle edge computing networks, optimizing the offloading decisions for vehicle computing tasks and reducing task execution latency has become a research hotspot. Existing technologies mainly consider the collaborative relationships between multiple vehicles or the offloading decisions between vehicles and roadside units. However, these methods have several problems: First, they do not consider the heterogeneity of intensive computing tasks. In congested areas or during rush hour, roadside units with fixed computing resources cannot meet the offloading needs of a large number of vehicles, and increasing the computing power of roadside units during idle periods leads to idle computing resources, resulting in reduced resource utilization. Second, they do not effectively combine the competition between vehicles for roadside unit computing resources with the potential collaboration between vehicles for idle resources, resulting in high latency and low system resource utilization. Summary of the Invention

[0004] The purpose of this invention is to provide a method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, comprising the following steps:

[0005] 1) Construct a vehicle edge computing network system, including roadside units and multiple intelligent vehicles.

[0006] 2) Obtain information data of all intelligent vehicles, roadside units and tasks in the current vehicle edge computing network system at the same time.

[0007] 3) Initialize the parameters of the vehicle edge computing network system and initialize the iteration count episode = 1.

[0008] 4) Initialize the state of all intelligent vehicles and set time slice t = 1.

[0009] 5) Select the task vehicle at time slice t, select the corresponding action according to the current time slice strategy of the current vehicle, generate an experience tuple after the intelligent vehicle executes the corresponding action, and store the experience tuple in the replay pool.

[0010] The experience tuple includes the state of the intelligent vehicle in time slice t, the actions of the intelligent vehicle in time slice t, the reward of the intelligent vehicle in time slice t, and the state of the intelligent vehicle in time slice t+1.

[0011] 6) Update the target network of the vehicle edge computing network system using the data in the playback pool, and determine whether time t has reached the preset threshold. If yes, proceed to step 7); otherwise, set t = t + 1 and return to step 5.

[0012] 7) Determine if the iteration count (episode) has reached the preset threshold. If yes, output the vehicle unloading calculation method. If no, set episode = episode + 1 and return to step 4.

[0013] Furthermore, in the vehicle edge computing network system, the task execution strategy includes execution locally, execution offloaded to roadside units, and execution offloaded to the task vehicle.

[0014] The mission vehicle is selected through mission vehicle selection logic.

[0015] Furthermore, the latency of executing the task locally is as follows:

[0016]

[0017] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Let be the execution delay of the local computation task performed by the i-th intelligent vehicle in time slice t. f represents the computational workload for generating computational tasks for the i-th intelligent vehicle in time slice t. i This represents the computing resources of the i-th intelligent vehicle. Represents a collection of intelligent vehicles. Indicates the iteration period.

[0018] If the execution delay of the i-th intelligent vehicle performing the computation task locally in time slice t is... The maximum tolerable execution latency of the computational task of the i-th intelligent vehicle in time slice t is not more than the maximum tolerable execution latency of the computational task in time slice t. The cost of local execution is Otherwise, it is Ω, where Ω is a constant.

[0019] The delay for the task to be offloaded to the roadside unit for execution is as follows:

[0020]

[0021] In the formula, These represent task transmission delay, task waiting delay, and task execution delay, respectively. This indicates the execution delay when computational tasks are offloaded to roadside units. This represents the signal transmission rate from the i-th intelligent vehicle to the roadside unit. f represents the size of the computational task generated by the i-th intelligent vehicle in time slice t. RSU This represents the computing resources of the roadside unit.

[0022] Among them, task waiting delay As shown below:

[0023]

[0024] In the formula, This indicates the set of vehicles selected to offload computational tasks to the RSU within time slice t.

[0025] The delay in offloading the task to the task vehicle for execution is as follows:

[0026]

[0027] In the formula, This indicates the execution delay when the computation task is unloaded and executed on the task vehicle. This indicates that the computation task is transferred from the i-th intelligent vehicle to the task vehicle k. t The delay. This indicates the delay in the execution of the computation task. Indicates the mission vehicle k t Computing resources. This indicates the distance from the i-th intelligent vehicle to the task vehicle k. t The signal transmission rate.

[0028] Furthermore, the total cost C of the vehicle edge computing network system tot As shown below:

[0029]

[0030] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. This represents the iteration period. θ is a control variable. This indicates that the computation task is executed locally. This indicates that intelligent vehicles will offload computational tasks to roadside units for execution. This indicates that intelligent vehicles will offload computational tasks to task vehicles for execution. These represent the total cost of executing the computation task locally, the total cost of offloading the computation task to a roadside unit for execution, and the total cost of offloading the computation task to a task vehicle for execution, respectively.

[0031] Furthermore, the information data of all intelligent vehicles, roadside units, and tasks in the current vehicle edge computing network system at the same moment includes the size of heterogeneous tasks. CPU cycles required for the task Maximum tolerable execution delay for the task and vehicle location information

[0032] Furthermore, the vehicle's location information As shown below:

[0033]

[0034]

[0035] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period. Let x and y represent the x and y coordinates of the i-th intelligent vehicle at time slice t-1, respectively. Let x and y represent the x and y coordinates of the i-th intelligent vehicle at time slice t, respectively. R represents the movable distance of the i-th intelligent vehicle based on its position at time slice t-1. Indicates the angle of movement.

[0036] Furthermore, the parameters of the vehicle edge computing network system include the learning rate of the commentator network and the executor network, as well as the experience replay pool.

[0037] Furthermore, the steps for selecting the task vehicle at time slice t, selecting the corresponding action according to the current vehicle's current time slice strategy, generating an experience tuple after the intelligent vehicle executes the corresponding action, and storing the experience tuple in the replay pool are as follows:

[0038] 5.1) Select the task vehicle for the current time slice t using the task vehicle selection logic, and set the state of the i-th intelligent vehicle in time slice t to be...

[0039] 5.2) For the i-th intelligent vehicle, select an action based on the vehicle unloading method calculated in the current time slice t.

[0040] 5.3) The i-th intelligent vehicle performs an action And receive the corresponding reward. and the state of time slice t+1

[0041] 5.4) Empirical tuples Store it in the experience replay pool.

[0042] 5.5) Return to step 5.2) until all the experience tuples generated by all vehicles are stored in the experience replay pool.

[0043] Furthermore, the task vehicle selection logic is as follows:

[0044]

[0045] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period. k t Indicates the vehicle on the mission.

[0046] Among them, the weight value As shown below:

[0047]

[0048] In the formula, j is the intelligent vehicle serial number. The weights of distance and computational resources, d i,j f represents the distance from intelligent vehicle i to intelligent vehicle j. i This represents the computing resources of intelligent vehicle i.

[0049] Furthermore, the step of updating the target network of the vehicle edge computing network system using data from the playback pool is as follows:

[0050] 6.1) Randomly sample from the experience replay pool and use the sampled data to update the parameters of the commentator network and the enforcer network.

[0051] The parameters of both the executor network and the commentator network are updated using a time difference algorithm. The loss L(β) of the executor network is... i As shown below:

[0052]

[0053] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. This indicates the iteration cycle. B represents the experience replay pool. Let represent the reward for the i-th intelligent vehicle in time slice t. This represents the state of the i-th intelligent vehicle at time slice t+1. This represents the vehicle unloading calculation method for the i-th intelligent vehicle in time slice t-1. This represents the value function of the executor network.

[0054] The loss L(α) of the commenter network i,t As shown below:

[0055]

[0056] In the formula, Let λ represent the dominance function. λ is a hyperparameter. Let represent the commentator network parameters for the i-th intelligent vehicle at time slice t. This indicates the update strategy of the commentator network.

[0057] 6.2) The target network of the vehicle edge computing network system is periodically updated using a soft update method, as shown below:

[0058]

[0059]

[0060] In the formula, ξ represents the target network update parameter of the vehicle edge computing network system. β i α i β' represents the executor network parameters and commenter network parameters before the update, respectively. i α' i These represent the updated executor network parameters and commenter network parameters, respectively.

[0061] The technical effectiveness of this invention is undeniable. This invention designs a competitive and cooperative computation offloading model for dense heterogeneous tasks in a vehicle edge computing network, aiming to effectively reduce task execution latency. This invention analyzes the costs of local execution and offloading to roadside units or task vehicles, and proposes a sequential task offloading decision problem.

[0062] This invention proposes a competitive and cooperative task offloading algorithm based on multi-agent reinforcement learning. Compared with other task offloading methods, this algorithm exhibits lower task execution latency and higher task completion rate, providing an innovative solution to the task offloading decision-making problem in the field of vehicle edge computing.

[0063] This invention provides a competitive and cooperative computational offloading decision optimization algorithm for dense heterogeneous tasks, which maximizes network benefits by taking into account task execution latency. Attached Figure Description

[0064] Figure 1Flowchart of the competitive and collaborative computational unloading algorithm;

[0065] Figure 2 This is a system model diagram. Detailed Implementation

[0066] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0067] Example 1:

[0068] See Figures 1 to 2 A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network includes the following steps:

[0069] 1) Construct a vehicle edge computing network system, including roadside units and multiple intelligent vehicles.

[0070] 2) Obtain information data of all intelligent vehicles, roadside units and tasks in the current vehicle edge computing network system at the same time.

[0071] 3) Initialize the parameters of the vehicle edge computing network system and initialize the iteration count episode = 1.

[0072] 4) Initialize the state of all intelligent vehicles and set time slice t = 1.

[0073] 5) Select the task vehicle at time slice t, select the corresponding action according to the current time slice strategy of the current vehicle, generate an experience tuple after the intelligent vehicle executes the corresponding action, and store the experience tuple in the replay pool.

[0074] The experience tuple includes the state of the intelligent vehicle in time slice t, the actions of the intelligent vehicle in time slice t, the reward of the intelligent vehicle in time slice t, and the state of the intelligent vehicle in time slice t+1.

[0075] 6) Update the target network of the vehicle edge computing network system using the data in the playback pool, and determine whether time t has reached the preset threshold. If yes, proceed to step 7); otherwise, set t = t + 1 and return to step 5.

[0076] 7) Determine if the iteration count (episode) has reached the preset threshold. If yes, output the vehicle unloading calculation method. If no, set episode = episode + 1 and return to step 4.

[0077] Example 2:

[0078] A method for competitive and collaborative computation offloading for dense heterogeneous tasks in a vehicle edge computing network is described in Example 1. Further, in the vehicle edge computing network system, the task execution strategy includes local execution, offloading to roadside unit execution, and offloading to task vehicle execution.

[0079] The mission vehicle is selected through mission vehicle selection logic.

[0080] Example 3:

[0081] A method for competitive and cooperative computation offloading of dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 1 and 2, further wherein the latency of the task execution locally is as follows:

[0082]

[0083] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Let be the execution delay of the local computation task performed by the i-th intelligent vehicle in time slice t. f represents the computational workload for generating computational tasks for the i-th intelligent vehicle in time slice t. i This represents the computing resources of the i-th intelligent vehicle. Represents a collection of intelligent vehicles. Indicates the iteration period.

[0084] If the execution delay of the i-th intelligent vehicle performing the computation task locally in time slice t is... The maximum tolerable execution latency of the computational task of the i-th intelligent vehicle in time slice t is not more than the maximum tolerable execution latency of the computational task in time slice t. The cost of local execution is Otherwise, it is Ω, where Ω is a constant.

[0085] The delay for the task to be offloaded to the roadside unit for execution is as follows:

[0086]

[0087]

[0088] In the formula, These represent task transmission delay, task waiting delay, and task execution delay, respectively. This indicates the execution delay when computational tasks are offloaded to roadside units. This represents the signal transmission rate from the i-th intelligent vehicle to the roadside unit. f represents the size of the computational task generated by the i-th intelligent vehicle in time slice t. RSU This represents the computing resources of the roadside unit.

[0089] Among them, task waiting delay As shown below:

[0090]

[0091] In the formula, This indicates the set of vehicles selected to offload computational tasks to the RSU within time slice t.

[0092] The delay in offloading the task to the task vehicle for execution is as follows:

[0093]

[0094] In the formula, This indicates the execution delay when the computation task is unloaded and executed on the task vehicle. This indicates that the computation task is transferred from the i-th intelligent vehicle to the task vehicle k. t The delay. This indicates the delay in the execution of the computation task. Indicates the mission vehicle k t Computing resources. This indicates the distance from the i-th intelligent vehicle to the task vehicle k. t The signal transmission rate.

[0095] Example 4:

[0096] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 1 to 3, further wherein the total cost C of the vehicle edge computing network system is... tot As shown below:

[0097]

[0098] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. This represents the iteration period. θ is a control variable. This indicates that the computation task is executed locally. This indicates that intelligent vehicles will offload computational tasks to roadside units for execution. This indicates that intelligent vehicles will offload computational tasks to task vehicles for execution. These represent the total cost of executing the computation task locally, the total cost of offloading the computation task to a roadside unit for execution, and the total cost of offloading the computation task to a task vehicle for execution, respectively.

[0099] Example 5:

[0100] A method for competitive and collaborative computation offloading of dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 1 to 4, further wherein the information data of all intelligent vehicles, roadside units and tasks in the current vehicle edge computing network system at the same time include the size of the heterogeneous tasks. CPU cycles required for the task Maximum tolerable execution delay for the task and vehicle location information

[0101] Example 6:

[0102] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 1 to 5, further comprising the vehicle's location information. As shown below:

[0103]

[0104]

[0105] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period. Let x and y represent the x and y coordinates of the i-th intelligent vehicle at time slice t-1, respectively. Let x and y represent the x and y coordinates of the i-th intelligent vehicle at time slice t, respectively. R represents the movable distance of the i-th intelligent vehicle based on its position at time slice t-1. Indicates the angle of movement.

[0106] Example 7:

[0107] A method for competitive and collaborative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 1 to 6. Further, the parameters of the vehicle edge computing network system include the learning rate of the commentator network and the executor network, as well as the experience replay pool.

[0108] Example 8:

[0109] A method for competitive and collaborative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 1 to 7, further comprising the following steps: selecting the task vehicle at time slice t, selecting the corresponding action according to the current time slice strategy of the current vehicle, generating an experience tuple after the intelligent vehicle executes the corresponding action, and storing the experience tuple in the replay pool:

[0110] 5.1) Select the task vehicle for the current time slice t using the task vehicle selection logic, and set the state of the i-th intelligent vehicle in time slice t to be...

[0111] 5.2) For the i-th intelligent vehicle, select an action based on the vehicle unloading method calculated in the current time slice t.

[0112] 5.3) The i-th intelligent vehicle performs an action And receive the corresponding reward. and the state of time slice t+1

[0113] 5.4) Empirical tuples Store it in the experience replay pool.

[0114] 5.5) Return to step 5.2) until all the experience tuples generated by all vehicles are stored in the experience replay pool.

[0115] Example 9:

[0116] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 1 to 8, and further, the task vehicle selection logic is as follows:

[0117]

[0118] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period. k t Indicates the vehicle on the mission.

[0119] Among them, the weight value As shown below:

[0120]

[0121] In the formula, j is the intelligent vehicle serial number. The weights of distance and computational resources, d i,j f represents the distance from intelligent vehicle i to intelligent vehicle j. i This represents the computing resources of intelligent vehicle i.

[0122] Example 10:

[0123] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 1 to 9, further wherein the step of updating the target network of the vehicle edge computing network system using data in the replay pool is as follows:

[0124] 6.1) Randomly sample from the experience replay pool and use the sampled data to update the parameters of the commentator network and the enforcer network.

[0125] The parameters of both the executor network and the commentator network are updated using a time difference algorithm. The loss L(β) of the executor network is... i As shown below:

[0126]

[0127] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. This indicates the iteration cycle. B represents the experience replay pool. Let represent the reward for the i-th intelligent vehicle in time slice t. This represents the state of the i-th intelligent vehicle at time slice t+1. This represents the vehicle unloading calculation method for the i-th intelligent vehicle in time slice t-1. This represents the value function of the executor network.

[0128] The loss L(α) of the commenter network t,t As shown below:

[0129]

[0130] In the formula, Let λ represent the dominance function. λ is a hyperparameter. Let represent the commentator network parameters for the i-th intelligent vehicle at time slice t. This indicates the update strategy of the commentator network.

[0131] 6.2) The target network of the vehicle edge computing network system is periodically updated using a soft update method, as shown below:

[0132]

[0133]

[0134] In the formula, ξ represents the target network update parameter of the vehicle edge computing network system. β i α i β' represents the executor network parameters and commenter network parameters before the update, respectively. i α' i These represent the updated executor network parameters and commenter network parameters, respectively.

[0135] Example 11:

[0136] See Figures 1 to 2A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network includes the following steps:

[0137] 1) Construct a vehicle edge computing network system, including roadside units (RSUs) and multiple intelligent vehicles.

[0138] 2) Obtain information data of all intelligent vehicles, roadside units and tasks in the current vehicle edge computing network system at the same time.

[0139] 3) Initialize the parameters of the vehicle edge computing network system and initialize the iteration count episode = 1.

[0140] 4) Initialize the state of all intelligent vehicles and set time slice t = 1.

[0141] 5) Select the task vehicle at time slice t, select the corresponding action according to the current time slice strategy of the current vehicle, generate an experience tuple after the intelligent vehicle executes the corresponding action, and store the experience tuple in the replay pool.

[0142] The experience tuple includes the state of the intelligent vehicle in time slice t, the actions of the intelligent vehicle in time slice t, the reward of the intelligent vehicle in time slice t, and the state of the intelligent vehicle in time slice t+1.

[0143] 6) Update the target network of the vehicle edge computing network system using the data in the playback pool, and determine whether time t has reached the preset threshold. If yes, proceed to step 7); otherwise, set t = t + 1 and return to step 5.

[0144] 7) Determine if the iteration count (episode) has reached the preset threshold. If yes, output the vehicle unloading calculation method. If no, set episode = episode + 1 and return to step 4.

[0145] Example 12:

[0146] A method for competitive and collaborative computation offloading for dense heterogeneous tasks in a vehicle edge computing network is described in Example 11. Further, in the vehicle edge computing network system, the task execution strategy includes local execution, offloading to roadside unit execution, and offloading to task vehicle execution.

[0147] The mission vehicle is selected through mission vehicle selection logic.

[0148] Example 13:

[0149] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 11 and 12, further wherein the latency of the task being executed locally is as follows:

[0150]

[0151] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Let be the execution delay of the local computation task performed by the i-th intelligent vehicle in time slice t. f represents the computational workload for generating computational tasks for the i-th intelligent vehicle in time slice t. i This represents the computing resources of the i-th intelligent vehicle. Represents a collection of intelligent vehicles. Indicates the iteration period.

[0152] If the execution delay of the i-th intelligent vehicle performing the computation task locally in time slice t is... The maximum tolerable execution latency of the computational task of the i-th intelligent vehicle in time slice t is not more than the maximum tolerable execution latency of the computational task in time slice t. The cost of local execution is Otherwise, it is Ω, which represents the penalty for not executing within the maximum tolerable execution delay of the task itself. Ω is a constant.

[0153] The execution latency of a computation task at the RSU consists of task transmission latency, task waiting latency, and RSU execution latency. A first-in-first-out queue is used at the RSU to simulate the scheduling of multiple offloading tasks. The latency of offloading tasks to the roadside unit for execution is as follows:

[0154]

[0155] In the formula, These represent task transmission delay, task waiting delay, and task execution delay, respectively. This indicates the execution delay when computational tasks are offloaded to roadside units. This represents the signal transmission rate from the i-th intelligent vehicle to the roadside unit. f represents the size of the computational task generated by the i-th intelligent vehicle in time slice t. RSU This represents the computing resources of the roadside unit.

[0156] Among them, task waiting delay As shown below:

[0157]

[0158] In the formula, This indicates the set of vehicles selected to offload computational tasks to the RSU within time slice t.

[0159] if but Otherwise, it is Ω.

[0160] The delay in offloading the task to the task vehicle for execution is as follows:

[0161]

[0162] In the formula, This indicates the execution delay when the computation task is unloaded and executed on the task vehicle. This indicates that the computation task is transferred from the i-th intelligent vehicle to the task vehicle k. t The delay. This indicates the delay in the execution of the computation task. Indicates the mission vehicle k t Computing resources. This indicates the distance from the i-th intelligent vehicle to the task vehicle k. t The signal transmission rate.

[0163] Example 14:

[0164] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 11 to 13, further wherein the total cost C of the vehicle edge computing network system is... tot As shown below:

[0165]

[0166] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. This represents the iteration period. θ is a control variable. This indicates that the computation task is executed locally. This indicates that intelligent vehicles will offload computational tasks to roadside units for execution. This indicates that intelligent vehicles will offload computational tasks to task vehicles for execution. These represent the total cost of executing the computation task locally, the total cost of offloading the computation task to a roadside unit for execution, and the total cost of offloading the computation task to a task vehicle for execution, respectively (cost = reward + penalty (negative value)).

[0167] when hour, and The coefficient will become 0, which is used to calculate the total task execution latency under the current unloading strategy. Therefore, the optimization problem can be formulated as:

[0168] C tot ,

[0169] stx i ∈0,1,2,

[0170]

[0171]

[0172] These represent the unloading decision, task execution delay constraints, and weight constraints, respectively. The problem described above is a continuous task unloading decision problem, which is an NP-hard problem.

[0173] Example 15:

[0174] A method for competitive and collaborative computation offloading of dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 11 to 14, further wherein the information data of all intelligent vehicles, roadside units and tasks in the current vehicle edge computing network system at the same time include the size of the heterogeneous tasks. CPU cycles required for the task Maximum tolerable execution delay for the task and vehicle location information

[0175] Example 16:

[0176] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 11 to 15, further comprising the vehicle's location information. As shown below:

[0177]

[0178]

[0179] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period. Let x and y represent the x and y coordinates of the i-th intelligent vehicle at time slice t-1, respectively. Let x and y represent the x and y coordinates of the i-th intelligent vehicle at time slice t, respectively. R represents the movable distance of the i-th intelligent vehicle based on its position at time slice t-1. Indicates the angle of movement.

[0180] Example 17:

[0181] A method for competitive and collaborative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 11 to 16. Further, the parameters of the vehicle edge computing network system include the learning rates of the commentator network and the executor network, as well as the experience replay pool B.

[0182] Example 18:

[0183] A method for competitive and collaborative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of embodiments 11 to 17, further wherein the steps of selecting the task vehicle at time slice t, selecting the corresponding action according to the current time slice strategy of the current vehicle, generating an experience tuple after the intelligent vehicle executes the corresponding action, and storing the experience tuple in the replay pool are as follows:

[0184] 5.1) Select the task vehicle for the current time slice t using the task vehicle selection logic, and set the state of the i-th intelligent vehicle in time slice t to be...

[0185] 5.2) For the i-th intelligent vehicle, select an action based on the vehicle unloading method calculated in the current time slice t.

[0186] 5.3) The i-th intelligent vehicle performs an action And receive the corresponding reward. and the state of time slice t+1

[0187] 5.4) Empirical tuples Store it in the experience replay pool.

[0188] 5.5) Return to step 5.2) until all the experience tuples generated by all vehicles are stored in the experience replay pool.

[0189] Example 19:

[0190] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 11 to 18, and further, the task vehicle selection logic is as follows:

[0191]

[0192] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period. k t Indicates the vehicle on the mission.

[0193] The selection of the task vehicle is based on a weight value calculated according to the average distance between vehicle i and other vehicles and its own computing resource size. As shown below:

[0194]

[0195] In the formula, j is the intelligent vehicle serial number. The weights of distance and computational resources, d i,j f represents the distance from intelligent vehicle i to intelligent vehicle j. i This represents the computing resources of intelligent vehicle i.

[0196] Example 20:

[0197] A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, the main technical contents of which are described in any one of Embodiments 11 to 19, further wherein the step of updating the target network of the vehicle edge computing network system using data in the playback pool is as follows:

[0198] 6.1) Randomly sample from the experience replay pool and use the sampled data to update the parameters of the commentator network and the enforcer network.

[0199] The parameters of both the executor network and the commentator network are updated using a time difference algorithm. The loss L(β) of the executor network is... i As shown below:

[0200]

[0201] In the formula, i is the intelligent vehicle serial number. t represents the time slice. Represents a collection of intelligent vehicles. This indicates the iteration cycle. B represents the experience replay pool. Let represent the reward for the i-th intelligent vehicle in time slice t. This represents the state of the i-th intelligent vehicle at time slice t+1. This represents the vehicle unloading calculation method for the i-th intelligent vehicle in time slice t-1. Let represent the value function of the actor network, which is trained to estimate the expected reward.

[0202] The loss L(α) of the commenter network i,t As shown below:

[0203]

[0204] In the formula, Let λ represent the dominance function. λ is a hyperparameter. Let represent the commentator network parameters for the i-th intelligent vehicle at time slice t. This indicates the update strategy of the commentator network.

[0205] 6.2) The target network of the vehicle edge computing network system is periodically updated using a soft update method, as shown below:

[0206]

[0207]

[0208] In the formula, ξ represents the target network update parameter of the vehicle edge computing network system. β i α i β' represents the executor network parameters and commenter network parameters before the update, respectively. i α' i These represent the updated executor network parameters and commenter network parameters, respectively.

[0209] After training, each agent can determine the most effective action strategy based on their own observations.

[0210] Example 21:

[0211] See Figures 1 to 2 A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network includes the following steps:

[0212] 1) Establish a competitive and cooperative vehicle edge computing network system model. The vehicle edge computing network system model includes one roadside unit (RSU) and multiple intelligent vehicles. Task execution latency modeling under different strategies is as follows:

[0213] 1.1) Local execution delay

[0214] Vehicle i performs the computation task locally at time t. The execution delay can be expressed as:

[0215]

[0216] Here f i This represents the computing resources of vehicle i. This represents the computational workload (in CPU cycles required) needed to complete the computational task generated by vehicle i at time t. Indicates a collection of vehicles. Indicates the iteration period. If ( (This represents the maximum tolerable execution delay for vehicle i at time t). The cost of local execution is then expressed as... Otherwise, use Ω to represent the local execution cost (Ω represents the penalty for not executing within the maximum tolerable execution latency of the task itself, and it is a constant).

[0217] 1.2) Delay in unloading to roadside units

[0218] The execution latency of a computation task at RSU consists of task transmission latency, task waiting latency, and RSU execution latency. We use a first-in, first-out queue at RSU to simulate the scheduling of multiple unloading tasks. Therefore, the task waiting latency of vehicle i at time t can be expressed as:

[0219]

[0220] in, This represents the set of vehicles selected to offload computational tasks to the RSU within time t. The delay between the final offloading of the computational task to the RSU and its execution can be expressed as:

[0221]

[0222] in, and These represent task transmission delay, waiting delay, and execution delay, respectively. f represents the signal transmission rate from the vehicle to the RSU and the computational task size generated by vehicle i at time t, respectively. RSU This refers to the RSU's computing resources. Finally, if but Otherwise, it is Ω.

[0223] 1.3) Delay in unloading to the task vehicle

[0224] The execution latency of unloading the computational task onto the task vehicle can be expressed as:

[0225]

[0226]

[0227] in, and These represent the transfer of tasks from vehicle i to task vehicle k, respectively. t Delay, task execution delay, task vehicle k t Computing resources and vehicle i to vehicle k t The signal transmission rate.

[0228] 2) Obtain information data of all vehicles, roadside units, and tasks in the current vehicle edge computing network at the same time. Specifically, the information data includes the size of heterogeneous tasks. CPU cycles required for the task Maximum tolerable execution delay for the task and vehicle location information in Depend on and This can be expressed by the following formula:

[0229]

[0230]

[0231] Here, R represents vehicle i based on its position in the previous round. The movable distance, where θ represents the movement angle; The initial position of the vehicle is determined by a Gaussian distribution within a specific area (a 500*100 meter road). Finally, the selection of the task vehicle is based on a weighted value calculated from the average distance of vehicle i to other vehicles and its own computational resource size. It can be represented as:

[0232]

[0233] in, The weights of distance and computational resources, d i,j f represents the distance from vehicle i to j. i This represents the computing resources of vehicle i. Subsequently, task vehicle k can then... To calculate.

[0234] 3) Model the task execution latency in the vehicle edge computing network as a reward for the system. We will characterize the parameters... This indicates that the computational task is executed locally on the vehicle. This indicates that the vehicle has chosen to offload the computational task to a roadside unit for execution. This indicates that the computational task will be offloaded to the task vehicle for execution. The total system cost can then be expressed as follows:

[0235]

[0236] Where θ is a control variable, when hour, and The coefficient will become 0, which we use to calculate the total task execution latency under the current unloading strategy. Therefore, we can formulate the optimization problem as:

[0237] C tot ,

[0238] stx i ∈0,1,2,

[0239]

[0240]

[0241] These represent the unloading decision, task execution delay constraints, and weight constraints, respectively. The problem described above is a continuous task unloading decision problem, which is an NP-hard problem.

[0242] 4) Initialize the vehicle edge computing network system parameters and begin iterative computation. Specifically, the system parameters of the vehicle edge computing network include the learning rate of the commentator network and the executor network, the number of iterations, and the experience replay pool B.

[0243] 5) Initialize episode = 1 and begin iterative calculation. The specific steps are as follows:

[0244] 5.1) Initialize all vehicles initial state

[0245] 5.2) Select the task vehicle for the current iteration round t using the task vehicle selection logic;

[0246] 5.3) For each vehicle i, use the current strategy Choose an action

[0247] 5.4) Perform the action And receive a reward and new observations

[0248] 5.5) Empirical tuples Store in Experience Replay Pool B;

[0249] 5.6) Repeat the process until i = N (N is the last vehicle).

[0250] 6) Update the algorithm parameters using data from experience replay pool B. The specific steps are as follows:

[0251] 6.1) Randomly sample from experience replay pool B.

[0252] 6.2) For vehicle i, update the parameters of the commentator network α and the actor network β as follows. The commentator network parameters are updated using a time difference algorithm, and the loss of the commentator network can be calculated as follows:

[0253]

[0254] Update the actor network parameters using the policy gradient algorithm, and calculate the loss of the actor network using the following method:

[0255]

[0256] Here, λ represents the hyperparameter used to adjust the entropy term, while the dominance function... This means the discount reward minus the baseline.

[0257] 7) Update the target network periodically using soft updates:

[0258]

[0259]

[0260] Here, ξ represents the target network update parameters. After training, each agent can determine the most effective action strategy based on its own observations.

[0261] 8) Return to step 5 and iterate repeatedly until the number of iterations is num_episodes. Output the vehicle calculation unloading strategy.

[0262] It is worth noting that this invention designs a competitive and cooperative computational offloading model for dense heterogeneous tasks in a vehicle edge computing network, aiming to effectively reduce task execution latency. This invention analyzes the costs of local execution and offloading to roadside units or task vehicles, and proposes a sequential task offloading decision problem. To solve this problem, this invention proposes a competitive and cooperative computational offloading algorithm based on multi-agent reinforcement learning. Compared with other task offloading methods, this algorithm exhibits lower task execution latency and higher task completion rate, providing an innovative solution to the task offloading decision problem in the field of vehicle edge computing.

[0263] This invention provides a competitive and cooperative computational offloading decision optimization algorithm for dense heterogeneous tasks, which maximizes network benefits by taking into account task execution latency.

Claims

1. A method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network, characterized in that, Includes the following steps: 1) Construct a vehicle edge computing network system, including roadside units and multiple intelligent vehicles; In a vehicle edge computing network system, the execution strategies for tasks include local execution, offloading to roadside units for execution, and offloading to the task vehicle for execution. The delay for executing the task locally is shown below: （1） In the formula, The vehicle serial number is t; t represents the time slice. For the first The execution latency of a smart vehicle performing a computational task locally in time slice t; For the first The computational workload of a smart vehicle generating computational tasks in time slice t; Indicates the first Computing resources for a single intelligent vehicle; Represents a collection of intelligent vehicles. Indicates the iteration period; If the first The execution latency of a smart vehicle performing a computational task locally in time slice t No more than the The maximum tolerable execution latency of a smart vehicle's computational task in time slice t. The cost of local execution is Otherwise , It is a constant; The delay for the task to be offloaded to the roadside unit for execution is as follows: （2） In the formula, , , These represent task transmission delay, task waiting delay, and task execution delay, respectively. This indicates the execution delay when computational tasks are offloaded to roadside units; Indicates the first The signal transmission rate from a smart vehicle to a roadside unit; Indicates the first The size of the computational tasks generated by a smart vehicle in time slice t; This represents the computing resources of the roadside unit; Among them, task waiting delay As shown below: （3） In the formula, This represents the set of vehicles selected to offload computational tasks to roadside units within time slice t; The delay in offloading the task to the task vehicle for execution is as follows: （4） In the formula, This indicates the execution delay when the computation task is unloaded and transferred to the task vehicle for execution. Indicates that the computation task starts from the first... Transmitted from intelligent vehicle to mission vehicle The delay; Indicates the delay in the execution of the computation task; Indicates the mission vehicle Computing resources; Indicates the first From intelligent vehicles to mission vehicles The signal transmission rate; The task vehicle is selected through task vehicle selection logic; The vehicle selection logic for the mission is as follows: （5） In the formula, The vehicle serial number is t; t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period; Indicates the vehicle on the mission; Among them, the weight value As shown below: （6） In the formula, For intelligent vehicle serial numbers; Represents the weights of distance and computational resources. Indicating intelligent vehicles To intelligent vehicles distance, Indicating intelligent vehicles Computing resources; 2) Obtain information data of all intelligent vehicles, roadside units, and tasks in the current vehicle edge computing network system at the same time; 3) Initialize the parameters of the vehicle edge computing network system and initialize the iteration count episode=1; 4) Initialize the state of all intelligent vehicles and set the time slice. =1; 5) Select For a task vehicle within a time slice, the corresponding action is selected according to the current time slice strategy of the current vehicle. After the intelligent vehicle executes the corresponding action, an experience tuple is generated and stored in the replay pool. The steps are as follows: 5.1) Select the current vehicle through the task vehicle selection logic. The mission vehicle of the time slice, and ordered the first The state of the intelligent vehicle in time slice t is ; 5.2) For the first A smart vehicle selects an action based on the vehicle's unloading calculation method in the current time slice t. ; 5.3) Section Intelligent vehicles perform actions And receive the corresponding reward. and +1 time slice status ; 5.4) Empirical tuples Store in the experience replay pool; 5.5) Return to step 5.2) until all experience tuples generated by all vehicles are stored in the experience replay pool; The empirical tuples include The status of time-slice intelligent vehicles The actions of time-slice intelligent vehicles Rewards for time-slice smart vehicles and The status of the intelligent vehicle in the time slice; 6) Update the target network of the vehicle edge computing network system using data from the playback pool, and determine the time. Has the preset threshold been reached? If yes, proceed to step 7; otherwise, let = +1, and return to step 5); The total cost of the vehicle edge computing network system As shown below: （7） In the formula, The vehicle serial number is t; t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period; It is a control variable; This indicates that the computation task is executed locally. This indicates that intelligent vehicles will offload computational tasks to roadside units for execution. This indicates that intelligent vehicles will offload computational tasks to task vehicles for execution; , , These represent the total cost of executing the computation task locally, the total cost of offloading the computation task to a roadside unit for execution, and the total cost of offloading the computation task to a task vehicle for execution, respectively. 7) Determine if the iteration count (episode) has reached the preset threshold. If yes, output the vehicle unloading calculation method. If no, set episode = episode + 1 and return to step 4.

2. The method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network according to claim 1, characterized in that, The information data of all intelligent vehicles, roadside units, and tasks in the current vehicle edge computing network system at the same moment includes the size of heterogeneous tasks. Number of CPU cycles required for the task Maximum tolerable execution delay for the task and vehicle location information .

3. The method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network according to claim 2, characterized in that, The vehicle's location information As shown below: （8） （9） In the formula, The vehicle serial number is t; t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period; , They represent the first The x and y coordinates of a smart vehicle at time slice t-1; , They represent the first The x and y coordinates of a smart vehicle at time slice t; Representing the The distance a smart vehicle can move based on its position at time slice t-1. Indicates the angle of movement.

4. The method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network according to claim 1, characterized in that, The parameters of the vehicle edge computing network system include the learning rates of the commentator network and the executor network, as well as the experience replay pool.

5. The method for competitive and cooperative computation offloading for dense heterogeneous tasks in a vehicle edge computing network according to claim 1, characterized in that, The steps for updating the target network of the vehicle edge computing network system using data from the playback pool are as follows: 6.1) Randomly sample from the experience replay pool and use the sampled data to update the parameters of the commentator network and the enforcer network; The parameters of both the executor network and the commentator network are updated using a time-difference algorithm, and the loss of the executor network is... As shown below: （10） In the formula, The vehicle serial number is t; t represents the time slice. Represents a collection of intelligent vehicles. Indicates the iteration period; Represents the experience replay pool; Indicates the first Rewards for intelligent vehicles in time slice t; Indicates the first Intelligent vehicles in The state of +1 time slice; Indicates the first Intelligent vehicles in -1 time slice vehicle unloading calculation method; Represents the value function of the executor network; The loss of the commentator network As shown below: （11） In the formula, Represents the dominance function; For hyperparameters; Indicates the first Commentator network parameters of a smart vehicle in time slice t; This indicates the update strategy of the commentator network; 6.2) The target network of the vehicle edge computing network system is periodically updated using a soft update method, as shown below: （12） （13） In the formula, This represents the target network update parameters of the vehicle edge computing network system; , These represent the executor network parameters and the commenter network parameters before the update, respectively. , These represent the updated executor network parameters and commenter network parameters, respectively.

Images