Task unloading and resource allocation optimization method for vehicle-assisted edge computing network

Abstract

The invention discloses a task unloading and resource allocation optimization method for a vehicle-assisted edge computing network. The method comprises the following steps: establishing a basic model of a vehicle auxiliary edge computing network; establishing a vehicle auxiliary edge calculation task cost evaluation and optimization model, quantifying the cost and solving an optimization problem; according to the system state information, obtaining an unloading decision of the sub-task by using a multi-agent deep reinforcement learning method based on MADDPG; according to the unloading decision, obtaining a transmitting power and computing resource allocation decision by using a PSO-based particle swarm optimization method; and after the decision is completed, system state information and buffer area information are updated, and a multi-agent neural network is sampled and trained. According to the method, the problems of limited vehicle equipment resources, heavy burden of the MEC server, complex task unloading decision and the like can be solved, the purposes of reducing task execution delay, optimizing the resource utilization rate and reducing the system cost are achieved, and the performance of the vehicle-assisted edge computing network is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle networking and mobile edge computing, and in particular to a method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network. Background Art

[0002] With the in-depth development of Internet of Things (IoT) technology in the field of transportation, the Internet of Vehicles (IoV) has emerged and has become a focus of common concern in academia and industry. By building a network topology between communication entities such as vehicles, pedestrians and roadside units (RSUs), the IoV aims to provide efficient and secure information services for the traffic environment to meet the growing traffic needs. Among them, vehicle edge computing (VEC), as a key technology in the IoV, significantly improves the vehicle's task processing capabilities by offloading computationally intensive and delay-sensitive tasks to mobile edge computing (MEC) servers or other vehicles with computing capabilities, providing strong support for applications such as intelligent driving and augmented reality (AR) navigation.

[0003] In the field of vehicle edge computing, many studies have been devoted to optimizing task offloading and resource allocation strategies to improve system performance. Some studies focus on the strategy of offloading tasks to RSUs. For example, Dai et al. (P. Dai, K. Hu, X. Wu, H. Xing, F. Teng, and Z. Yu, "A probabilistic approach for cooperative computation offloading in MEC-assisted vehicular networks," IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 2, pp. 899-911, 2022.) proposed a joint load balancing and task offloading scheme to maximize system utilization; Wang et al. (H. Wang, T. Lv, Z. Lin, and J. Zeng, "Energy-delay minimization of task migration based on game theory in MEC-assisted vehicular networks," IEEE Transactions on Vehicular Technology, vol. 71, no. 8, pp. 8175 - 8188, 2022.) used game theory methods to enable vehicles to determine task offloading strategies in real time. However, these studies often ignore the vehicle's own computing resources and tend to offload all tasks to edge nodes, which may lead to insufficient resource utilization.

[0004] Although existing research has made some progress in task offloading and resource allocation in vehicle edge computing, there are still many shortcomings. Many solutions fail to fully consider the complexity of tasks, such as dependencies between tasks, dynamic changes in data transmission and computing resources, resulting in inefficient task scheduling. Some studies are not flexible enough in the use of vehicle resources and have not fully explored the potential of idle vehicle resources, limiting the improvement of the overall system performance. At the same time, the lack of a vehicle cache update strategy makes data acquisition inefficient during task processing and increases task processing delays. Therefore, there is an urgent need for an optimization solution that comprehensively considers task complexity, flexibility in vehicle resource utilization, and cache updates to improve the performance of vehicle-assisted edge computing networks and meet the growing needs of intelligent transportation applications. Summary of the invention

[0005] The purpose of the present invention is to provide a method for optimizing task offloading and resource allocation in a vehicle-assisted edge computing network to solve the problems existing in the prior art, such as limited vehicle equipment resources, heavy burden on MEC servers, and complex task offloading decisions, so as to achieve the goals of reducing task execution delays, optimizing resource utilization, and reducing system costs.

[0006] The technical solution to achieve the purpose of the present invention is: on the one hand, a method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network is provided, and the method comprises the following steps:

[0007] Step 1: Build a basic model of the vehicle-assisted edge computing network, including system architecture, task characteristics, service cache management, and communication capabilities;

[0008] Step 2: Build a task cost evaluation and optimization model, quantify the cost and solve the optimization problem, clarify the goal and task completion time, and unloading constraints;

[0009] Step 3: Calculate the subtask scheduling order and completion time based on the system status, and use the multi-agent deep reinforcement learning method based on MADDPG to determine the subtask offloading decision, and define the relevant elements through the algorithm framework;

[0010] Step 4: Based on the unloading decision, the PSO-based particle swarm method is used to solve the transmission power and computing resource allocation decision, and the optimal solution is determined through particle encoding, definition of fitness function and iterative update;

[0011] Step 5: After completing the above decision, update the system status, buffer information and train the multi-agent neural network, including the vehicle service program and the multi-agent reinforcement learning algorithm update process to achieve continuous optimization of system performance.

[0012] On the other hand, a task offloading and resource allocation optimization system for a vehicle-assisted edge computing network is provided, the system comprising:

[0013] The first module is used to build the basic model of the vehicle-assisted edge computing network, including system architecture, task characteristics, service cache management and communication capabilities;

[0014] The second module is used to build a task cost evaluation and optimization model, quantify the cost and solve the optimization problem, clarify the goal and task completion time, and unloading constraints;

[0015] The third module is used to calculate the subtask scheduling order and completion time according to the system status, and use the multi-agent deep reinforcement learning method based on MADDPG to determine the subtask offloading decision, which is implemented by defining relevant elements through the algorithm framework;

[0016] The fourth module is used to solve the transmission power and computing resource allocation decision based on the unloading decision using the PSO-based particle swarm method, and determine the optimal solution through particle encoding, definition of fitness function and iterative update;

[0017] The fifth module is used to update the system status, buffer information and train the multi-agent neural network after completing the above decisions, including the vehicle service program and the multi-agent reinforcement learning algorithm update process to achieve continuous optimization of system performance.

[0018] Compared with the prior art, the present invention has the following significant advantages:

[0019] (1) In task processing, the complexity of the task is fully taken into consideration. The basic model constructed comprehensively covers the system architecture, task characteristics, cache management, and communication capabilities. The cost evaluation and optimization model is accurately established, and a variety of costs and constraints are comprehensively considered, which makes up for the shortcomings of existing research in insufficient consideration of task dependencies and dynamic changes of resources.

[0020] (2) For the task offloading and resource allocation optimization method, a step-by-step solution is adopted. First, the subtask offloading decision is determined based on the MADDPG multi-agent deep reinforcement learning algorithm, and the corresponding service program is pulled from the edge server. Finally, the resource allocation decision is made based on the PSO particle swarm algorithm, which effectively improves the problems existing in the existing technology in offloading decision-making and resource utilization.

[0021] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flowchart of the task offloading and resource allocation optimization method of the vehicle-assisted edge computing network.

[0023] Figure 2 It is a schematic diagram of the system architecture model. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0025] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0026] The purpose of the present invention is to provide a method for optimizing task offloading and resource allocation in a vehicle-assisted edge computing network, so as to solve the problems existing in the prior art, such as limited vehicle equipment resources, heavy burden on MEC servers, and complex task offloading decisions, so as to achieve the goals of reducing task execution delays, optimizing resource utilization, and reducing system costs.

[0027] In one embodiment, in combination Figure 1 and Figure 2 , provides a method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network, the method comprising the following steps:

[0028] Step 1: Build a basic model of the vehicle-assisted edge computing network, covering key elements such as system architecture, task characteristics, service cache management and communication capabilities, to provide a basis for subsequent optimization.

[0029] Step 2: Build a task cost evaluation and optimization model to quantify the cost and solve the optimization problem, and clarify the goals and constraints such as task completion time and unloading.

[0030] Step 3: Based on the system status, calculate the subtask scheduling order and completion time point, and use the MADDPG-based multi-agent deep reinforcement learning method to determine the subtask offloading decision, and define the relevant elements through the algorithm framework.

[0031] Step 4: Based on the offloading decision, the PSO-based particle swarm method is used to solve the transmission power and computing resource allocation decision, and the optimal solution is determined through particle encoding, definition of fitness function and iterative update.

[0032] Step 5: After completing the above decision, update the system status, buffer information and train the multi-agent neural network, including the vehicle service program and the multi-agent reinforcement learning algorithm update process to achieve continuous optimization of system performance.

[0033] Furthermore, in one embodiment, step 1 specifically includes:

[0034] Step 1-1, build a vehicle-assisted edge computing network system model, specifically: obtain vehicle resource information in urban environment scenarios, divide time slots and define task vehicle sets, clarify RSU resource information, establish a decision module on the cloud server to process vehicle and RSU upload status information and generate decisions; specifically include:

[0035] Step 1-1-1, obtain vehicle resource information in the urban environment scene: Consider an urban environment scene with parallel lanes and vehicles running, the vehicle collection in the system , the i-th vehicle Expressed as ,in, , is the total number of vehicles, is the driving speed, is the maximum transmit power, is the maximum computation frequency, The maximum cache capacity.

[0036] Step 1-1-2, divide the time slots. Divide the time into These equal time slots, each vehicle generates at most one task in a time slot. In, define the set represents the set of vehicles that generate tasks, Indicates vehicles.

[0037] Step 1-1-3, obtain the RSU resource information in the urban environment scene. RSUs are deployed on one side of the road, denoted as , Represents the Mth RSU deployed, and adjacent RSUs are connected by optical fiber. According to the coverage of RSUs, the road is divided into sections , is the Mth road segment, and M is the total number of road segments. All RSUs are equipped with MEC servers to provide computing offload services for vehicles.

[0038] Preferably, for simplicity, a MEC server and a connected RSU are represented by a fixed edge node.

[0039] Step 1-1-4, establish a decision-making module. Deploy the intelligent agent decision-making module on the cloud server to process the status information uploaded by the vehicle and RSU and make decisions.

[0040] Step 1-2, establish the task model. This includes defining the task type and required cache capacity, determining the RSU service program leasing rules; establishing the vehicle task model, clarifying the number, information and dependencies of subtasks; introducing three unloading methods: local execution of subtasks, unloading to RSU and other vehicles.

[0041] Step 1-2-1, task type definition. Define the task type set as , the i-th task type The required cache capacity is ; Assuming that RSU has deployed service programs corresponding to all task types, after the vehicle makes a decision on subtask unloading, it updates the service program rental according to the subtask unloading situation to assist itself and other vehicles in unloading subtasks, and the unit price for leasing the service program from RSU is service fees.

[0042] Step 1-2-2, build the mission model. Vehicle The tasks generated in a time slot can be expressed as ,in is the number of subtasks in the task; Subtask information; is a matrix representing the subtask dependencies; Represents a subtask and There is a direct dependency between Represents a subtask and There is no direct dependency between them; For task type ; Submit the task at a certain time. The maximum tolerable time for the task.

[0043] Step 1-2-3, establish a subtask model. Assume that the subtasks cannot be divided any further and define the tasks No. The subtask is ,in Indicates the input data size of the subtask; Indicates the number of CPU cycles required to complete the subtask.

[0044] Step 1-2-4, subtask offloading method. In the ITS scenario, the vehicle Generate Tasks After that, the subtask can be processed in three ways: local execution, offloading to a fixed RSU node, or offloading to an auxiliary vehicle that meets the conditions. ,have offloading decisions, and their task offloading symbol factors are , , .

[0045] Step 1-2-4-1, uninstall to local. Subtask When executing locally ,otherwise .

[0046] Step 1-2-4-2, unload to RSU. Subtask When offloading to a fixed RSU node ,otherwise .

[0047] Step 1-2-4-3, unload to other vehicles. Subtask Unloading to other vehicles hour ,otherwise .

[0048] Step 1-2-4-4, subtask offloading decision definition. The uninstall decision is defined as , the relationship between the unloading symbol factor and the unloading decision is: if but ,like but ,like but ,show Decide on subtasks Uninstall to .

[0049] Step 1-3, establish a service cache model, including the vehicle leasing service programs based on subtask offloading information, and the storage space constraints of the service program cache.

[0050] Step 1-3-1, rental service conditions. After the subtask makes an unloading decision, the vehicle updates the service program rental according to the unloading information. When the vehicle processes a subtask, it first checks whether there is a service program in its own cache space to process the subtask. If so, it processes it directly; if not, it rents it from the RSU. The vehicle cache space has an upper limit. If there is no extra space and a rental service is required, the FIFO strategy is used to replace the saved service program.

[0051] Step 1-3-2, vehicle storage space limitation. For the cache of service programs, there are the following storage capacity limitations:

[0052] (1)

[0053] in, is a binary variable, indicating vehicle Is the cached capacity Service Procedure , Indicates that the service is cached, otherwise it is not cached, and the final result is , indicating whether each vehicle has cached a certain service program. Definition Binary variable, indicating whether it is a newly cached service program in the current time slot. Indicates the number of service program types. Indicates vehicle cache space.

[0054] Step 1-4: Establish a communication model, including vehicle-to-RSU communication and vehicle-to-vehicle communication.

[0055] Step 1-4-1, V2I communication model. Each vehicle in the system uses V2I communication to establish a communication connection with the RSU. The communication bandwidth between the vehicle and the RSU is , multiple subtasks are offloaded to edge server nodes .vehicle and fixed RSU nodes Uplink data rate between Indicates the number of service program types. Indicates vehicle The cache space calculation formula is:

[0056] (2)

[0057] in, yes The upload transmission power, yes and The instantaneous channel gain between is the instantaneous distance between the two, and Respectively and The path loss and fading factor between represents the background noise power, , is the total number of vehicles that generate tasks.

[0058] Step 1-4-2, V2V communication model. The i-th vehicle and the gth vehicle The bandwidth resources available are , and there are multiple subtasks offloaded to the vehicle .vehicle and vehicles The achievable transmission rate between is expressed as:

[0059] (3)

[0060] in yes and The instantaneous channel gain between is the instantaneous distance between the two, and Respectively and The path loss and fading factor between them.

[0061] Furthermore, in one embodiment, step 2 is to establish a vehicle-assisted edge computing task cost evaluation and optimization model. The establishment of this model needs to comprehensively consider multiple factors and clarify constraints such as goals and task completion time, unloading, etc. Specifically including:

[0062] Step 2-1, establish a calculation model. Mainly explain the execution delay, energy consumption calculation and its formulaic representation under different processing modes of vehicle subtasks.

[0063] Step 2-1-1, local processing. If the vehicle Generated subtasks If local execution is selected, the execution time is:

[0064] (4)

[0065] in, For vehicles Assign to subtask of computing resources, , .

[0066] Energy consumption of local execution:

[0067] (5)

[0068] in, is the effective switched capacitance in the chip.

[0069] Step 2-1-2, MEC server processing. If the vehicle Generated subtasks Offloaded to fixed RSU nodes Execute, if the vehicle When leaving the RSU coverage area during mission execution, The results will be transmitted through MEC server communication. Since the amount of returned data is small, the download delay can be ignored. The transmission time is expressed as:

[0070] (6)

[0071] Subtasks exist The execution time is expressed as:

[0072] (7)

[0073] in, for Select Assign to Subtask computing resources.

[0074] If the vehicle If you leave the road section where the current RSU coverage is located, the communication time between RSUs is:

[0075] (8)

[0076] in, is the number of road sections the vehicle passes through, is the communication time between adjacent RSUs, which is defined as a constant.

[0077] Therefore, the total time for offloading to the MEC server is:

[0078] (9)

[0079] vehicle The energy consumption generated by the transmission task is:

[0080] (10)

[0081] Step 2-1-3, SV processing. If the vehicle Generated subtasks Uninstall to ,The time consists of two parts: transmission time and execution time. and The transmission time between It is expressed as:

[0082] (11)

[0083] Subtasks Execution time It is expressed as:

[0084] (12)

[0085] in, For vehicles Assign to subtask computing resources.

[0086] Therefore, uninstall to Total time for:

[0087] (13)

[0088] Transmission energy consumption It is expressed as:

[0089] (14)

[0090] at the same time, Computational energy consumption generated by auxiliary computing offloading It is expressed as:

[0091] (15)

[0092] Step 2-1-4, uninstall the formulaic representation.

[0093] Step 2-1-4-1, time delay formula. By analyzing the calculation model, we can get the vehicle Generated subtasks Total execution time for:

[0094] (16)

[0095] Subtasks Task upload time :

[0096] (17)

[0097] Step 2-1-4-2, energy consumption formula. Subtask The energy consumption of vehicles is divided into two parts: Energy consumption :

[0098] (18)

[0099] vehicle Execution energy consumption generated by auxiliary calculation :

[0100] (19)

[0101] Step 2-2, establish a cost model. This includes the cost calculation of leasing service programs from RSU and the cost calculation of leasing computing resources between vehicles. Specifically, it includes:

[0102] Step 2-2-1: Lease service program fees from RSU. For vehicles, if they want to process tasks, they need to cache the service program for the corresponding tasks. Each vehicle needs to pay the rental service program fee. for:

[0103] (20)

[0104] in, Indicates the vehicles in the current time slot Whether the service program is newly cached , represents the unit price of the service program for leasing from RSU, Indicates the cache size of the service program.

[0105] Step 2-2-2, renting computing resources. According to the above calculation model, when the vehicle Deciding to offload a subtask When a resource provider is used, the resource provider must pay the resource provider for the computing resource usage. , and its computing resource unit price is set as :

[0106] (twenty one)

[0107] Unit price and vehicle Willing to do subtask Allocated computing resources Proportional, is the correlation coefficient.

[0108] For vehicles with mission requirements :like Allocate more computing resources to subtasks , which can reduce delay cost, but will increase Finally, for the subtask , Resource providers The service fee paid can be expressed as :

[0109] (twenty two)

[0110] Step 2-3, construct and solve the problem of minimizing the average execution cost of the task initiator in the system, which specifically includes:

[0111] Step 2-3-1, define the task initiator cost and optimization problem, and clarify the system optimization goals and constraints.

[0112] Costs to the task initiator Defined as:

[0113] (twenty three)

[0114] in, , and represent the relative weights of latency, energy consumption, and service cost, respectively, and , Indicates the exit subtask of task i, that is, the completion time of the last subtask. Total energy consumption (generating tasks and providing services to other vehicles) are:

[0115] (twenty four)

[0116] Finally, the optimization problem is expressed as:

[0117] (25)

[0118] in, , , represents the task offloading symbolic factor. In addition, the offloading decision of all tasks is defined as ,in . Indicates the computing resources allocated to all subtasks in the system. Indicates the computing resources allocated to each subtask; Indicates the transmission power allocated to all subtasks in the system, Indicates the transmit power allocated to each subtask.

[0119] Constraints:

[0120] (26)

[0121] (27)

[0122] (28)

[0123] (29)

[0124] (30)

[0125] (31)

[0126] From formula (25), we can get that the objective function is the average execution cost of all tasks in the system. To reduce the average execution cost, we need to minimize the sum of the execution costs of all tasks. Formula (26) represents the task completion time constraint; formula (27) represents the subtask offloading constraint; formula (28) represents the offloading decision constraint; formula (29) represents the transmission power constraint; formulas (30) and (31) respectively represent that the computing resources allocated by RSU and vehicle for all subtasks do not exceed their total computing resources.

[0127] Step 2-3-2, using a step-by-step solution strategy to solve the optimization problem, integrating multi-agent reinforcement learning and particle swarm optimization to optimize task offloading and resource allocation, as follows:

[0128] In the first step, multi-agent reinforcement learning is used to solve the subtask offloading decision, and the optimal offloading location of the subtask is determined through agent interaction and learning, laying the foundation for resource allocation.

[0129] In the second step, the particle swarm algorithm is used to solve the resource allocation decision, with the goal of minimizing the system cost, optimizing the transmission power and computing resource allocation, and improving the resource utilization efficiency.

[0130] The third step is to update information based on the current execution results, including system status information, buffer information, and agent learning buffer data, so that the agent can update the neural network based on the latest information, continuously optimize the decision-making process, and improve system performance. This step-by-step solution strategy effectively copes with the complexity of the problem and improves the optimization effect of task offloading and resource allocation in the vehicle-assisted edge computing network.

[0131] Furthermore, in one embodiment, step 3 uses a multi-agent reinforcement learning method based on MADDPG to solve the offloading decision, and the implementation process is as follows: first, the subtask scheduling order and completion time point are calculated according to the system state, and then the subtask offloading decision is determined by a multi-agent deep reinforcement learning method based on MADDPG. Specifically, it includes:

[0132] Step 3-1, subtask scheduling order.

[0133] Step 3-1-1, according to Internal dependencies , and get the subtask The set of predecessor subtasks and the set of subsequent subtasks .

[0134] Step 3-1-2, subtask execution delay.

[0135] Step 3-1-2-1, local execution delay. Substitute into formula (4) to obtain the subtask The time for local processing, where For local vehicles The maximum calculation frequency provided.

[0136] Step 3-1-2-2, in RSU The execution delay is Substituting into formula (7), Substitute into formula (2) and call formula (6) to obtain the subtask In RSU The transmission and computation delays of for The maximum computing resources provided. Therefore, the total execution delay :

[0137] (32)

[0138] Step 3-1-2-3, calculate the execution delay on other vehicles. Substituting into formula (12), Substitute into formula (3) and call formula (11) to obtain the subtask Unloading to vehicle Therefore, the total execution delay of unloading to other vehicles is obtained by formula (13).

[0139] Step 3-1-3, calculate the subtask execution cost. Substitute the execution delay obtained by formula (4) (32) (13) , and Substitute the following formula:

[0140] (33)

[0141] Get the execution cost of the subtask, where Indicates the number of vehicles in the system.

[0142] Step 3-1-4, execution priority. Subtasks calculated according to step 3-1-3 The execution cost of the task is used to determine its execution priority. Defined as exit subtasks, each subtask has at least one path leading to the exit subtask. The priority of the subtask is calculated using the reverse recursive method:

[0143] (34)

[0144] in Represented as a subtask The set of successor subtasks of For subtask The priority of For subtask priority.

[0145] Step 3-1-5, build a subtask scheduling queue. According to the subtask priority obtained in step 3-1-4, all subtasks are sorted to form a scheduling queue, which is expressed as .

[0146] Step 3-2, calculate the completion time of the subtask.

[0147] definition Represents a subtask The task completion time, Represents a subtask The time point when the task upload is completed;

[0148] Subtasks The task completion time can be expressed as:

[0149] (35)

[0150] in, It is a subtask The set of predecessor subtasks, when hour, .

[0151] Subtasks The time when the task upload is completed The calculation is expressed as:

[0152] (36)

[0153] Among them, when hour, .

[0154] Step 3-3, vehicle-assisted edge computing network task offloading optimization algorithm based on multi-agent reinforcement learning.

[0155] Formula (25) aims to optimize the execution cost of all tasks in the vehicle-assisted edge computing network. Since this problem involves resource competition and cooperation among multiple vehicles and a dynamic environment, the traditional single-agent reinforcement learning algorithm cannot solve it. Therefore, a multi-agent reinforcement learning algorithm is proposed to solve the task offloading problem of the vehicle-assisted edge computing network. In this algorithm, the intelligent vehicle that generates tasks is the agent, and the vehicle-assisted edge computing network is the reinforcement learning environment. The agent maximizes its utility function by learning different strategies, thereby optimizing system performance.

[0156] The following introduces the algorithm framework based on multi-agent reinforcement learning.

[0157] In reinforcement learning, the environment and the agent are the core elements. In each time slot, the agent makes an action decision based on the current state of the environment. The action will change the environment, and the environment will then feed back a reward signal of the new state to the agent. There are three key elements in the reinforcement learning algorithm: state, action, and reward. The state represents the current situation of the system, the action is the decision made by the agent based on the observed state, and the reward is the feedback signal from the environment, which is used to evaluate the behavior of the agent. The agent maximizes the cumulative reward by continuously adjusting the action strategy. The following will elaborate on the application of these three key elements in the vehicle-assisted edge computing network in combination with the system model and research questions.

[0158] Step 3-4-1, state space. In order to make the agent take appropriate actions according to the environmental information, Agents in time slot The state space The definition is as follows:

[0159] (37)

[0160] (1) Vehicle status information. Determine the vehicle set , for each vehicle Set its initial travel speed , Maximum transmit power , Maximum calculation frequency and maximum cache capacity .

[0161] (38)

[0162] (2) RSU status information. Clearly identify the current RSU Location coordinates , Coverage And the total computing resources of the MEC servers equipped .

[0163] (39)

[0164] (3) Task and subtask status information. Define a task set .in Indicates the number of subtasks contained in the task. Indicates the scheduling order of subtasks (calculated by step 3-1-5), Represents the entire task Type of task ; Representation Task The submission time point; Representation Task The maximum tolerance time.

[0165] (40)

[0166] (4) Service program status information, including:

[0167] Determine the status information of all vehicle cache services in the current time slot, where Indicates whether this service is cached.

[0168] (41)

[0169] Determine relevant information of the service program, such as unit price and capacity.

[0170] (42)

[0171] The service program status information is:

[0172] (43)

[0173] Step 3-4-2, action space. Mainly solve the offloading decision of subtasks. Agents in time slot The action space is

[0174] (44)

[0175] In the formula, They represent the three processing modes of each task generated by agent n in time slot t: local execution, offloading to RSU, and offloading to other vehicles;

[0176] Step 3-4-3, reward function. For the vehicle , if the current state satisfies the constraints in formulas (26), (27), and (28), then Agents in time slot The reward function It is expressed as:

[0177] (45)

[0178] in Negative parameter.

[0179] In a multi-agent system, the reward function comprehensively considers the behaviors of all agents, promotes system collaboration or coordination, balances cooperation and competition among agents, and maximizes overall performance. The reward is defined as the sum of all agent rewards. Extending formula (45) to a multi-agent environment, the reward value of the system in the current state is It can be expressed as:

[0180] (46)

[0181] Further, in one embodiment, in step 4, based on the offloading decision determined in step 3, a particle swarm method based on PSO is used to solve the transmission power and computing resource allocation decision. This process involves particle encoding, fitness function definition, and iterative update operations to determine the optimal solution. Specifically as follows:

[0182] Step 4-1, resource allocation.

[0183] The present invention adopts the particle swarm (PSO) algorithm to solve the problem of transmitting power resource allocation. The PSO algorithm simulates the predation behavior of bird flocks in nature, abstracts the bird flocks into particle swarms, and regards the food source as the optimal solution to be found. Within a spatial range, particles share information, and each particle searches the space based on self-cognition and social experience to obtain the optimal solution.

[0184] Step 4-1-1, particle encoding. Assume the number of particles is , particle number , number of tasks ,Task The number of subtasks , using the matrix and Represents the moving position of the particle, where the matrix Describe the transmit power resource allocation decision, the matrix Describes computing resource allocation decisions. Matrix The element value of Represents a subtask Transmit power resource allocation decision, indicating the amount of transmit power allocation, matrix The element value of Describe the subtask Computing resource allocation decision, indicating the number of computing resources allocated, in GHz.

[0185] In the resource allocation matrix and In the above example, all particles are coded with real numbers. and They represent the emission power allocation decision and computing resource allocation decision of the particle respectively. Value Represents a subtask The motion trend of allocating transmission power, the matrix Value Represents a subtask The movement trend of allocating computing resources.

[0186] Step 4-1-2, fitness function. The total cost of the system Size:

[0187] (47)

[0188] Step 4-2, designing a particle swarm algorithm, including:

[0189] In the particle swarm algorithm, the position of a particle represents a feasible solution. The next position of each particle is calculated through each iteration until it converges to the optimal position. The update of the particle position is determined by the last position and the particle's speed. The core of the algorithm is the iterative update method of the particle position and speed. The speed update is affected by three aspects: inertial speed, self-cognitive experience, and social experience. Each aspect corresponds to a factor, namely the inertia factor. and learning factor , By dynamically changing the value of each factor during the iteration process, the purpose of optimizing the particle swarm algorithm is achieved.

[0190] Step 4-2-1, parameter update.

[0191] If the particle The speed of the iteration is , and its update formula is:

[0192] (48)

[0193] in, represents the number of update iterations, is a random function, is the optimal solution currently searched by the particle, is the optimal solution currently searched by all particles, is the inertia factor, For the The speed of iterations; , is the learning factor. Let the inertia factor Dynamic update, the update formula is:

[0194] (49)

[0195] in, and are the current number of iterations and the maximum number of iterations respectively, is the number of particles, , are coefficients, which can be calculated based on The update formula can adjust the inertia factor of the particle in the early stage of motion. Larger, so that the algorithm has a stronger global convergence ability, as the number of iterations in the late stage of the movement increases, The nonlinearity is reduced, which makes the algorithm have stronger local convergence ability. At the same time, the number of particles Influence :When the number of particles is large, reduce To prevent the particle path from being repeated; when the number of particles is small, increase it appropriately The value of can increase the global convergence ability and prevent the algorithm from local convergence due to insufficient particle path length.

[0196] Learning Factor , Describe the influence of personal cognitive experience and social experience on particle speed respectively. This part is updated dynamically. , According to the comparison of the particle fitness values ​​in each iteration, the particle fitness value in the next iteration is dynamically reduced or increased. , The update formulas are:

[0197] (50)

[0198] (51)

[0199] In the formula, Respectively Iteration, The learning factor of the iteration , Respectively Iteration, The learning factor of the iteration , , and The particles are The fitness value at the time of the first update, the fitness value of the particle's current optimal solution, and the fitness value of the global optimal solution, Indicates the number of updates. , All are coefficients, used to adjust the incremental ratio;

[0200] Step 4-2-2, location update.

[0201] The particle position update formula is:

[0202] (52)

[0203] in, , and The particles are The fitness value at the time of the first update, the fitness value of the particle's current optimal solution, and the fitness value of the global optimal solution, Indicates the number of updates. , are coefficients, which can adjust the increment ratio. Formula (52) shows that when the particle If the fitness value of the next update is smaller than the fitness value of the last individual optimal solution, appropriately increase the particle Updated (Right now ), otherwise reduce appropriately This operation means that good particles will increase the influence of their own experience, while other particles rely on the social experience of the particle group. If the fitness value of the individual optimal solution is less than the fitness value corresponding to the global optimal solution during the first update, appropriately reduce the number of particles. No. Updated (Right now ), otherwise it increases This operation means that good particles will reduce the influence of social experience on the next speed of particles, trust their own cognitive experience, and the influence of social experience of other particles will also increase, making particles move closer to the global optimal solution. To prevent particles from being overconfident or over-reliant, set , The range of change, that is , , and .

[0204] Step 4-2-3, the specific process of the algorithm is as follows:

[0205] Step 4-2-3-1, initialization, random generation in the solution space A particle swarm of particles, including the position matrix , and the velocity matrix , To avoid the particle speed calculated by the formula being too large or too small, the speed , set the minimum speed limit to , the maximum speed limit is ; For speed , set the minimum speed limit to , the maximum speed limit is When calculating the next position of a particle based on the velocity formula, the position may exceed the boundary. To avoid this phenomenon, , set the minimum position limit to , the maximum position limit is ; For location , set the minimum position limit to , the maximum position limit is .

[0206] Step 4-2-3-2, calculate the fitness value of each particle according to formula (47).

[0207] Step 4-2-3-3, update individual historical optimal position and the global optimal particle position .

[0208] Step 4-2-3-4, update the particle swarm position matrix and , so that the transmit power resource allocation matrix The out-of-bounds element value in is located at and between The out-of-bounds element value is located at and between.

[0209] Step 4-2-3-5, update the particle swarm velocity matrix and , so that the velocity matrix The out-of-bounds element value is located at and between, The same applies to out-of-bounds element values.

[0210] Step 4-2-3-6, let .

[0211] Step 4-2-3-7, if the fitness difference between two iterations is less than the fixed value, or the number of iterations When the fixed value is reached, the algorithm ends and outputs the global optimal particle position , otherwise return to step 4-2-3-2 to continue iterating.

[0212] Further, in one embodiment, in step 5, after the decision in step 3 and step 4 is completed, the system status and buffer information are updated, and the multi-agent neural network is trained, including the vehicle service program operation and the multi-agent reinforcement learning algorithm update process, so as to achieve continuous optimization of system performance. Specifically:

[0213] Step 5-1, vehicle service program update;

[0214] Step 5-1-1, according to the subtask unloading situation (Formula (44)), check the subtask service type unloaded to the current vehicle during the unloading process;

[0215] Step 5-1-2: If the current vehicle already has this service program, there is no need to lease the current service from the RSU.

[0216] Step 5-1-3, if the current vehicle does not have this service program:

[0217] Step 5-1-3-1, determine whether the size of the current service program pulled exceeds the cache capacity limit of the current vehicle (Formula (1)). If not, cache it directly in the vehicle;

[0218] Step 5-1-3-2: if the cache capacity of the current vehicle is exceeded, the service program in the vehicle cache space is replaced by FIFO;

[0219] Step 5-2, multi-agent reinforcement learning algorithm update process, specifically includes:

[0220] Step 5-2-1, initialize global policy network parameters , target network parameters And the corresponding target network parameters and ;

[0221] Step 5-2-2, initialize the global experience playback memory buffer ;

[0222] Step 5-2-3, set the number of iterations , the initial value is 0;

[0223] Step 5-2-4, initialize the observations of all agents ;

[0224] Step 5-2-5, initialize the time slot is 0;

[0225] Step 5-2-6, each agent generates an independent random number , if the random number Less than the greed factor , with probability Choose to explore, i.e. randomly choose an action , otherwise, with probability According to the strategy Select Action ;

[0226] Step 5-2-7, each agent performs its own actions , and observe the current reward value and the next state ;

[0227] Step 5-2-8, the sample Upload to buffer ;

[0228] Step 5-2-9, update the current state to the next state: ;

[0229] Step 5-2-10, if The number of samples in the experience pool reaches the scale , then randomly select Samples ;

[0230] Step 5-2-11, calculate the target value ;

[0231] Step 5-2-12, minimize the loss function and update the value network parameters ;

[0232] Step 5-2-14, update the policy network parameters by deterministic policy gradient method ;

[0233] Step 5-2-15, using soft update to update the target network parameters to a smoothed version of the policy network parameters;

[0234] Step 5-3, start a new round of training until the preset end condition is reached.

[0235] In one embodiment, a task offloading and resource allocation optimization system for a vehicle-assisted edge computing network is provided, the system comprising:

[0236] The first module is used to build the basic model of the vehicle-assisted edge computing network, including system architecture, task characteristics, service cache management and communication capabilities;

[0237] The second module is used to build a task cost evaluation and optimization model, quantify the cost and solve the optimization problem, clarify the goal and task completion time, and unloading constraints;

[0238] The third module is used to calculate the subtask scheduling order and completion time according to the system status, and use the multi-agent deep reinforcement learning method based on MADDPG to determine the subtask offloading decision, which is implemented by defining relevant elements through the algorithm framework;

[0239] The fourth module is used to solve the transmission power and computing resource allocation decision based on the unloading decision using the PSO-based particle swarm method, and determine the optimal solution through particle encoding, definition of fitness function and iterative update;

[0240] The fifth module is used to update the system status, buffer information and train the multi-agent neural network after completing the above decisions, including the vehicle service program and the multi-agent reinforcement learning algorithm update process to achieve continuous optimization of system performance.

[0241] For the specific limitations on the task offloading and resource allocation optimization system of the vehicle-assisted edge computing network, please refer to the limitations on the task offloading and resource allocation optimization method of the vehicle-assisted edge computing network mentioned above, which will not be repeated here. Each module in the above-mentioned task offloading and resource allocation optimization system of the vehicle-assisted edge computing network can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0242] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0243] Step 1: Build a basic model of the vehicle-assisted edge computing network, including system architecture, task characteristics, service cache management, and communication capabilities;

[0244] Step 2: Build a task cost evaluation and optimization model, quantify the cost and solve the optimization problem, clarify the goal and task completion time, and unloading constraints;

[0245] Step 3: Calculate the subtask scheduling order and completion time based on the system status, and use the multi-agent deep reinforcement learning method based on MADDPG to determine the subtask offloading decision, and define the relevant elements through the algorithm framework;

[0246] Step 4: Based on the unloading decision, the PSO-based particle swarm method is used to solve the transmission power and computing resource allocation decision, and the optimal solution is determined through particle encoding, definition of fitness function and iterative update;

[0247] Step 5: After completing the above decision, update the system status, buffer information and train the multi-agent neural network, including the vehicle service program and the multi-agent reinforcement learning algorithm update process to achieve continuous optimization of system performance.

[0248] For the specific limitations of each step, please refer to the limitations of the task offloading and resource allocation optimization method of the vehicle-assisted edge computing network mentioned above, which will not be repeated here.

[0249] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0250] Step 1: Build a basic model of the vehicle-assisted edge computing network, including system architecture, task characteristics, service cache management, and communication capabilities;

[0251] Step 2: Build a task cost evaluation and optimization model, quantify the cost and solve the optimization problem, clarify the goal and task completion time, and unloading constraints;

[0252] Step 3: Calculate the subtask scheduling order and completion time based on the system status, and use the multi-agent deep reinforcement learning method based on MADDPG to determine the subtask offloading decision, and define the relevant elements through the algorithm framework;

[0253] Step 4: Based on the unloading decision, the PSO-based particle swarm method is used to solve the transmission power and computing resource allocation decision, and the optimal solution is determined through particle encoding, definition of fitness function and iterative update;

[0254] Step 5: After completing the above decision, update the system status, buffer information and train the multi-agent neural network, including the vehicle service program and the multi-agent reinforcement learning algorithm update process to achieve continuous optimization of system performance.

[0255] For the specific limitations of each step, please refer to the limitations of the task offloading and resource allocation optimization method of the vehicle-assisted edge computing network mentioned above, which will not be repeated here.

[0256] The method of the present invention can solve the problems of limited vehicle equipment resources, heavy burden on MEC servers, and complex task offloading decisions, achieve the goals of reducing task execution delays, optimizing resource utilization, and reducing system costs, and improve the performance of vehicle-assisted edge computing networks.

[0257] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for optimizing task offloading and resource allocation in a vehicle-assisted edge computing network, characterized in that: The method comprises the following steps: Step 1: Build a basic model of the vehicle-assisted edge computing network, including system architecture, task characteristics, service cache management, and communication capabilities; Step 2: Build a task cost evaluation and optimization model, quantify the cost and solve the optimization problem, clarify the goal and task completion time, and unloading constraints; Step 3: Calculate the subtask scheduling order and completion time based on the system status, and use the multi-agent deep reinforcement learning method based on MADDPG to determine the subtask offloading decision, and define the relevant elements through the algorithm framework; Step 4: Based on the unloading decision, the PSO-based particle swarm method is used to solve the transmission power and computing resource allocation decision, and the optimal solution is determined through particle encoding, definition of fitness function and iterative update; Step 5: After completing the above decision, update the system status, buffer information and train the multi-agent neural network, including the vehicle service program and the multi-agent reinforcement learning algorithm update process to achieve continuous optimization of system performance.

2. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 1, characterized in that: Step 1 specifically includes: Step 1-1, building a vehicle-assisted edge computing network system model, specifically including: obtaining vehicle resource information in urban environment scenarios, dividing time slots and defining task vehicle sets, determining RSU resource information, and establishing a decision module on the cloud server to process vehicle and RSU uploaded status information and generate decisions; Step 1-2, establish a task model, including defining the task type and required cache capacity, and determining the RSU service program leasing rules; establish a vehicle task model, clarify the number, information and dependencies of subtasks; define three unloading methods: local execution of subtasks, unloading to RSU and other vehicles; Step 1-3, establishing a service cache model, including the vehicle leasing service programs based on subtask offloading information, and storage space constraints for service program cache; Steps 1-4, establish a communication model, including vehicle-to-RSU communication and vehicle-to-vehicle communication, specifically including: Step 1-4-1, establish a V2I communication model; Each vehicle in the system uses V2I communication to establish a communication connection with the RSU. The communication bandwidth between the vehicle and the RSU is , multiple subtasks are offloaded to the edge server nodes, i.e. fixed RSU nodes ; The i-th vehicle and fixed RSU nodes Uplink data rate between The calculation formula is: ； in, yes The upload transmission power, yes and The instantaneous channel gain between is the instantaneous distance between the two, and Respectively and The path loss and fading factor between represents the background noise power, , The total number of vehicles that generate tasks; Step 1-4-2, establish a V2V communication model; The i-th vehicle and the gth vehicle The bandwidth resources available are , and there are multiple subtasks offloaded to the vehicle , then the vehicle and vehicles The transmission rate between : ； in, yes and The instantaneous channel gain between is the instantaneous distance between the two, and Respectively and The path loss and fading factor between , is the total number of vehicles.

3. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 2, characterized in that: Step 1-1 specifically includes: Step 1-1-1, obtain vehicle resource information in the urban environment scene: Consider an urban environment scene with parallel lanes and vehicles running, the vehicle collection in the system , the i-th vehicle Expressed as ,in, , is the total number of vehicles, is the driving speed, is the maximum transmit power, is the maximum computation frequency, is the maximum cache capacity; Step 1-1-2, divide time slots: divide time into These equal time slots, each vehicle generates at most one task in a time slot; In, define the set represents the set of vehicles that generate tasks, , , Indicates Vehicles; Step 1-1-3, obtain RSU resource information in the urban environment scenario: RSUs are deployed on one side of the road, denoted as , Represents the Mth RSU deployed. Adjacent RSUs are connected by optical fiber. The road is divided into sections according to the coverage of RSUs. , is the Mth road section, where M is the total number of road sections; all RSUs are equipped with MEC servers to provide computing offloading services for vehicles; Step 1-1-4, establish a decision-making module: deploy the intelligent agent decision-making module on the cloud server to process the status information uploaded by the vehicle and RSU and make decisions.

4. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 3, characterized in that: Steps 1-2 specifically include: Step 1-2-1, Task type definition: define the task type set as , the i-th task type The required cache capacity is ; Assume that RSU has deployed service programs corresponding to all task types. After the vehicle makes a decision on subtask unloading, it updates the service program leasing according to the subtask unloading situation to assist itself and other vehicles in unloading subtasks. The unit price for leasing the service program from RSU is service fees; Step 1-2-2, build the mission model: vehicle The tasks generated in a time slot are represented as , here ,in, is the number of subtasks in the task; is the subtask information, For the task No. subtasks, is a matrix representing the subtask dependencies, Represents a subtask and There is a direct dependency between Represents a subtask and There is no direct dependency between them; is the task type, ; Submit the task at a certain time. is the maximum tolerable time for the task; Step 1-2-3, establish a subtask model: assume that subtasks cannot be divided any further, define tasks No. The subtask is ,in Indicates The input data size of each subtask; Indicates completion of The number of CPU cycles required for each subtask, ; Step 1-2-4, subtask offloading method: In the ITS scenario, the vehicle Generate Tasks After that, the subtasks are processed by three methods: local execution, unloading to fixed RSU nodes, and unloading to auxiliary vehicles that meet the conditions; for subtasks ,have offloading decisions, the symbolic factors of the three task offloading methods are , , ; Specifically include: (1) Unloading to local machine: subtask When executing locally ,otherwise ; (2) Offloading to RSU: Subtask When offloading to a fixed RSU node ,otherwise ; (3) Unloading to an auxiliary vehicle that meets the conditions: Subtask Unload to a qualified auxiliary vehicle hour ,otherwise ,in ; (4) Subtask offloading decision definition: The uninstall decision is defined as , the relationship between the task offloading mode symbol factor and the offloading decision is: if but ,like but ;like but ,show Decide on subtasks Uninstall to .

5. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 4, characterized in that: Steps 1-3 specifically include: Step 1-3-1, establish rental service conditions; After the subtask makes an unloading decision, the vehicle updates the service program leasing according to the unloading information; when the vehicle processes a subtask, it first checks whether there is a service program to process the subtask in its own cache space. If so, it processes it directly; if not, it leases it from the RSU; the vehicle cache space has an upper limit. If there is no extra space and a rental service is required, the FIFO strategy is used to replace the saved service program; Step 1-3-2, establish vehicle storage space restrictions; For the service program cache, there are the following storage capacity limitations: ； in, is a binary variable, indicating vehicle Is the cached capacity Service Procedure , Indicates that the service program is cached , otherwise it means it is not cached, and finally forms , indicating whether each vehicle has cached a certain service program; definition Binary variable, indicating whether it is a newly cached service program in the current time slot. Indicates the number of service program types. Indicates vehicle cache space.

6. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 5, characterized in that: Step 2 specifically includes: Step 2-1, establishing a calculation model, including: Step 2-1-1, uninstall to local execution; If the vehicle Generated subtasks If you choose local execution, the execution time for: ； in, For vehicles Assign to subtask of computing resources, , ; Execution Energy Consumption for: ； in, is the effective switching capacitance in the chip; Step 2-1-2, the subtask is processed by the MEC server and offloaded to the fixed RSU node; If the vehicle Generated subtasks Offloaded to fixed RSU nodes Execute, if the vehicle When leaving the RSU coverage area during mission execution, The results will be transmitted through MEC server communication, subtask Transmission time It is expressed as: ； Subtask exist Execution time It is expressed as: ； in, for Select Assign to Subtask computing resources; If the vehicle If you leave the road section where the current RSU coverage is located, the communication time between RSUs for: ； in, is the number of road sections the vehicle passes through, is the communication time between adjacent RSUs, which is defined as a constant; Therefore, the total execution time of offloading to the MEC server is for: ； vehicle Energy consumption of transmission tasks for: ； Step 2-1-3, unloading to an auxiliary vehicle that meets the conditions for execution, that is, SV processing; If the vehicle Generated subtasks Uninstall to ,The time consists of two parts: transmission time and execution time; and The transmission time between It is expressed as: ； Subtasks Execution time It is expressed as: ； in, For vehicles Assign to subtask computing resources; Therefore, uninstall to Total time for: ； Transmission energy consumption It is expressed as: ； at the same time, Computational energy consumption generated by auxiliary computing offloading It is expressed as: ； Step 2-1-4, unloading the formulaic representation; (1) Time delay formulation; By analyzing the calculation model, the vehicle Generated subtasks Total execution time for: ； Subtasks Task upload time for: ； (2) Energy consumption formula; Subtasks The energy consumption of vehicles is divided into two parts: Energy consumption for: ； vehicle Execution energy consumption generated by auxiliary calculation for: ； Step 2-2, establish a cost model, including the cost calculation of leasing service programs from RSU and the cost calculation of leasing computing resources between vehicles; specifically including: Step 2-2-1, establish a cost calculation model for the RSU leasing service program; For vehicles, if you want to process tasks, you need to cache the service program for the corresponding task, and each vehicle needs to pay the rental service program fee. for: ； in, Indicates the vehicles in the current time slot Whether the service program is newly cached , represents the unit price of the service program for leasing from RSU, Indicates the cache size of the service program; Step 2-2-2, establish a cost model for leasing computing resources; According to the above cost calculation model, when the vehicle Deciding to offload a subtask When using the resource provider, you need to pay the resource provider for the computing resource usage fee; for the vehicle that provides the resource , and its computing resource unit price is set as : ； Unit price and vehicle Willing to do subtask Allocated computing resources Proportional, is the correlation coefficient; For vehicles with mission requirements :like Allocate relatively more computing resources to subtasks , then for the subtask , Resource providers The service fee paid is expressed as : ； Step 2-3, construct and solve the problem of minimizing the average execution cost of the task initiator in the system, which specifically includes: Step 2-3-1, define the task initiator cost and optimization problem, and clarify the system optimization goals and constraints; Costs to the task initiator Defined as: ； in, , and represent the relative weights of latency, energy consumption, and service cost, respectively, and , represents the exit subtask of task i, that is, the completion time of the last subtask; Total energy consumption for: ； Finally, the optimization problem is expressed as: ； in, , , represents the task offloading symbol factor; is the offloading decision of all tasks, where ; Indicates the computing resources allocated to all subtasks in the system. Indicates the computing resources allocated to each subtask; Indicates the transmission power allocated to all subtasks in the system, Indicates the transmission power allocated to each subtask; Constraints: ； ； ； ； ； ； In the formula, , Represent the total computing resources of RSU and vehicle respectively; Step 2-3-2, using a step-by-step solution strategy to solve the optimization problem, integrating multi-agent reinforcement learning and particle swarm algorithm to optimize task offloading and resource allocation, specifically including: The first step is to use multi-agent reinforcement learning to solve the subtask offloading decision and determine the optimal offloading location of the subtask through agent interaction and learning; In the second step, the particle swarm algorithm is used to solve the resource allocation decision, with the goal of minimizing the system cost and optimizing the allocation of transmission power and computing resources; The third step is to update information based on the current execution results, including system status information, buffer information, and agent learning buffer data, so that the agent can update the neural network based on the latest information and continuously optimize the decision-making process.

7. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 6, characterized in that: Step 3 specifically includes: Step 3-1, calculate the subtask scheduling order, specifically including: Step 3-1-1, according to Internal dependencies , get the subtask The set of predecessor subtasks and the set of subsequent subtasks ; Step 3-1-2, calculate the subtask execution delay, specifically including: Step 3-1-2-1, calculate the local execution delay: Substitute the execution time formula in step 2-1-1 to obtain the subtask The time for local processing, where For local vehicles The maximum calculation frequency provided; Step 3-1-2-2, calculate Execution delay: Substitute the execution time formula of step 2-1-2 into the formula, Substitute the formula in step 1-4-1 and call the transmission time formula in step 2-1-2 to obtain the subtask exist The transmission and computation delays of for The maximum computing resources provided, so the total execution delay is : ； Step 3-1-2-3, calculate the execution delay on other vehicles: Substitute the execution time formula of step 2-1-3, Substitute the transmission rate formula in step 1-4-2 and call the transmission time formula in step 2-1-3 to obtain the subtask Unloading to vehicle The transmission and calculation delays on the , so the total execution delay of unloading to other vehicles is obtained by the total time formula in step 2-1-3; Step 3-1-3, calculate the subtask execution cost; The execution delays obtained by respectively combining the execution time formula in step 2-1-1, the total execution delay formula in step 3-1-2-2, and the total time formula in step 2-1-3 , and Substitute the following formula: ； Get subtask Execution cost ,in Indicates the number of vehicles in the system; Step 3-1-4, execution priority; Subtasks calculated according to step 3-1-3 Execution cost of the , and obtain its execution priority; In the task described by DAG, the last subtask Defined as exit subtasks, each subtask has at least one path leading to the exit subtask, and the priority of the subtask is calculated using the reverse recursive method: ； in, Represented as a subtask The set of successor subtasks of For subtask The priority of For subtask Priority; Step 3-1-5, build a subtask scheduling queue; According to the subtask priority obtained in step 3-1-4, all subtasks are sorted to form a scheduling queue, which is expressed as ; Step 3-2, calculate the completion time of the subtask; definition Represents a subtask The task completion time, Represents a subtask The time point when the task upload is completed; Subtasks The task completion time is expressed as: ； in, It is a subtask The set of predecessor subtasks of hour, ; Subtasks The time when the task upload is completed It is expressed as: ； Among them, when hour, ; Step 3-3, design a vehicle-assisted edge computing network task offloading optimization algorithm based on multi-agent reinforcement learning to optimize the task offloading problem of the vehicle-assisted edge computing network; in the algorithm, the smart car that generates the task is the agent, and the vehicle-assisted edge computing network is the reinforcement learning environment.

8. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 7, characterized in that: The vehicle-assisted edge computing network task offloading optimization algorithm based on multi-agent reinforcement learning described in step 3-3 specifically includes: (1) State space; For Agents in time slot The state space The definition is as follows: ； in, is the vehicle status information, expressed as: ； In the formula, , , , are the i-th vehicle The initial driving speed, maximum transmission power, maximum calculation frequency and maximum cache capacity; is the RSU status information, expressed as: ； In the formula, for The location coordinates, For coverage, is the total computing resources of the configured MEC servers; is the task and subtask status information, expressed as: ； In the formula, Indicates the number of subtasks contained in the task. Indicates the scheduling order of subtasks, calculated by step 3-1-5. Represents the entire task The type of task; Indicates the task The submission time point; Indicates the task The maximum tolerance time; It is the service program status information, expressed as: ； in, Indicates the status information of all vehicle cache services in the current time slot. Indicates vehicle Whether the service program k is cached in Represents relevant information of the service program; (2) Action space; No. Agents in time slot The action space is: ； In the formula, They represent the three processing modes of each task generated by agent n in time slot t: local execution, offloading to RSU, and offloading to other vehicles; (3) Reward function; For vehicles , if the current state satisfies the first three formulas of the constraints in step 2-3-1, then Agents in time slot The reward function It is expressed as: ； in is a negative parameter; Expand the above formula to a multi-agent environment, and the reward value of the system in the current state It is expressed as: 。 9. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 8, characterized in that: Step 4 specifically includes: Step 4-1, allocate resources, including: Step 4-1-1, particle encoding: Assume the number of particles is , particle number , number of tasks ,Task The number of subtasks , using the matrix and Represents the moving position of the particle, where the matrix Describe the transmit power resource allocation decision, the matrix Describes computing resource allocation decisions; matrix The element value of Represents a subtask Transmit power resource allocation decision, indicating the amount of transmit power allocation, matrix The element value of Describe the subtask Computing resource allocation decision, which indicates the number of computing resources allocated, in GHz; In the resource allocation matrix and In the code, all particles are encoded with real numbers; and Respectively represent the emission power allocation decision and computing resource allocation decision of the particle; the matrix Value Represents a subtask The motion trend of allocating transmission power, the matrix Value Represents a subtask The movement trend of allocating computing resources; Step 4-1-2, construct the fitness function, which is the total cost of the system : ； Step 4-2, designing a particle swarm algorithm, including: Step 4-2-1, parameter update: If the particle The speed of the iteration is , and its update formula is: ； in, represents the number of update iterations, is a random function, is the optimal solution currently searched by the particle, is the optimal solution currently searched by all particles, is the inertia factor, For the The speed of iterations; , is the learning factor; The inertia factor Dynamic update, the update formula is: ； in, and are the current number of iterations and the maximum number of iterations respectively, is the number of particles, , are coefficients, The learning factor , The update formulas are: ； ； In the formula, Respectively Iteration, The learning factor of the iteration , Respectively Iteration, The learning factor of the iteration , , and The particles are The fitness value at the time of the first update, the fitness value of the particle's current optimal solution, and the fitness value of the global optimal solution, Indicates the number of updates. , All are coefficients, used to adjust the incremental ratio; Step 4-2-2, location update; The particle position update formula is: ； In the formula, Respectively Iteration, The position of the particle at the iteration; Step 4-2-3, the specific process of the algorithm includes: Step 4-2-3-1, initialization, random generation in the solution space A particle swarm of particles, including the position matrix , and the velocity matrix , ; For speed , set the minimum speed limit to , the maximum speed limit is ; For speed , set the minimum speed limit to , the maximum speed limit is ; For location , set the minimum position limit to , the maximum position limit is ; For location , set the minimum position limit to , the maximum position limit is ; Step 4-2-3-2, calculate the fitness value of each particle according to the fitness function of step 4-1-2; Step 4-2-3-3, update individual historical optimal position and the global optimal particle position ; Step 4-2-3-4, update the particle swarm position matrix and , so that the transmit power resource allocation matrix The out-of-bounds element value in is located at and between The out-of-bounds element value is located at and between; Step 4-2-3-5, update the particle swarm velocity matrix and , so that the velocity matrix The out-of-bounds element value is located at and between, The same applies to out-of-bounds element values; Step 4-2-3-6, let ; Step 4-2-3-7, if the fitness difference between two iterations is less than the preset value, or the number of iterations When the preset value is reached, the algorithm ends and outputs the global optimal particle position , otherwise return to step 4-2-3-2 to continue iterating.

10. The method for optimizing task offloading and resource allocation of a vehicle-assisted edge computing network according to claim 1, characterized in that: Step 5 specifically includes: Step 5-1, update the vehicle service program, including: Step 5-1-1, according to the subtask unloading situation, i.e., the action space, checking the subtask service type unloaded to the current vehicle during the unloading process; Step 5-1-2: If the current vehicle already has a service program corresponding to this subtask service type, there is no need to rent the current service from the RSU; if the current vehicle does not have a corresponding service program, execute step 5-1-3; Step 5-1-3 determines whether the size of the current service program pulled exceeds the cache capacity limit of the current vehicle. If not, it is directly cached in the vehicle; if it exceeds the cache capacity of the current vehicle, the service program in the vehicle cache space is replaced by FIFO; Step 5-2, update the multi-agent reinforcement learning algorithm, including: Step 5-2-1, initialize global policy network parameters , target network parameters And the corresponding target network parameters and ; Step 5-2-2, initialize the global experience playback memory buffer ; Step 5-2-3, set the number of iterations , the initial value is 0; Step 5-2-4, initialize the observations of all agents ; Step 5-2-5, initialize the time slot is 0; Step 5-2-6, each agent generates an independent random number , ; If the random number Less than the greed factor , with probability Choose to explore, i.e. randomly choose an action , otherwise, with probability According to the strategy Select Action ; Step 5-2-7, each agent performs its own actions , and observe the current reward value and the next state ; Step 5-2-8, the sample Upload to buffer ; Step 5-2-9, update the current state to the next state: ; Step 5-2-10, if When the number of samples in the experience pool reaches the preset size, a random sample is drawn from it. Samples ; Step 5-2-11, calculate the target value ; Step 5-2-12, minimize the loss function and update the value network parameters ; Step 5-2-14, update the policy network parameters by deterministic policy gradient method ; Step 5-2-15, using soft update to update the target network parameters to a smoothed version of the policy network parameters; Step 5-3, start a new round of training until the preset end condition is reached.

Images