Parking-assisted cooperative task unloading and resource allocation method

Abstract

The invention belongs to the technical field of mobile edge computing and deep reinforcement learning, and particularly relates to a deep reinforcement learning VEC collaborative task unloading and resource allocation optimization method based on edge computing under parking assistance. According to the method, the parking probability and resource availability of the parked vehicle are considered, and a plurality of candidate unloading nodes are selected between the vehicle and the MEC server by adopting an unloading node selection algorithm. According to the method, a system model for multi-parking vehicle auxiliary edge calculation is analyzed. Under the delay constraint, an unloading strategy is provided to solve the problems of load balancing and unloading optimization, including unloading node selection and workload distribution, so as to minimize the total system cost. Unloading decision selection and computing resource allocation can be effectively carried out, so that the convergence speed is improved, and the system time delay and energy consumption are reduced. The problems that the computing resources unloaded to the MEC server are overloaded and the time delay and the energy consumption of the vehicle terminal user are too high are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile edge computing and deep reinforcement learning, and specifically relates to a method for optimizing VEC collaborative task offloading and resource allocation based on edge computing for parking assistance. Background Art

[0002] In recent years, with the increase in the number of cars on the road and the advancement of technology, cars are also moving towards intelligence. The Internet of Vehicles (IOV) is the application of the Internet of Things in the field of intelligent transportation and is also an important organic part of the intelligent transportation system. However, with the continuous development of artificial intelligence technology and 6G technology, autonomous driving, high-definition video, AR (Augmented New resource-intensive Internet of Vehicles applications such as Virtual Reality (VR) and Virtual Reality (VR) continue to emerge. The limited computing resources configured in the vehicle itself are far from meeting the explosively growing computing resource requirements in the Internet of Vehicles. The shortage of computing resources has become a bottleneck problem that hinders the development of the Internet of Vehicles. It not only seriously affects the driving experience, but also poses a serious threat to driving safety.

[0003] The introduction of mobile edge computing (MEC) has become an effective solution to the problem of scarce vehicular network resources. Generally, MEC can enhance network resources and realize localized data processing by deploying computing and storage resources at the edge of the network close to users. Compared with remote cloud computing, it can use resource-rich servers at roadside units (RSUs) to provide users with low-latency, high-bandwidth application services. So far, many studies on VEC have been proposed. In these works, edge servers are placed near roadside units (RSUs). However, most of these studies have performance limitations, which can be explained from the following reasons. First, the offloading performance of these studies is largely affected by the number of edge servers and RSUs. However, due to the high cost of infrastructure construction, they cannot cover every place in the city. Second, the transmission range of RSU is actually limited. Third, when a disaster occurs, RSU may fail. Finally, the processing capacity of edge servers in VEC is limited. When many users try to offload tasks at the same time, the edge servers and wireless channels occupy too many resources, resulting in a long task response time.

[0004] In response to these problems, a deep reinforcement learning VEC collaborative task offloading and resource allocation optimization method based on edge computing under parking assistance is proposed. The present invention introduces the idea of ​​parking edge computing, which uses externally parked vehicles to assist edge servers in processing unloaded tasks. How to design the unloading decision of vehicle-mounted terminal users and the allocation of computing resources in the entire mobile edge computing system to minimize the latency and energy consumption of all vehicle-mounted terminal users is the main problem solved by the present invention. Summary of the invention

[0005] The present invention provides a deep reinforcement learning VEC collaborative task offloading and resource management based on edge computing in parking assistance. The resource allocation optimization method solves the problem of computing resource overload offloaded to MEC servers and the latency and energy consumption of vehicle end users. The problem of excessive consumption.

[0006] Firstly, the present invention takes roadside parked vehicles with idle computing resources as task offloading platforms and constructs a mobile edge computing task offloading strategy based on roadside parking collaboration.

[0007] Secondly, the offloading decision and resource allocation issues in a multi-user and multi-server parking vehicle assisted MEC environment are studied. In order to ensure the service quality of end users, the weighted total delay cost and energy consumption are taken as optimization targets. The problem is formulated as a Markov decision process, and a joint computation offloading and resource allocation optimization algorithm based on deep reinforcement learning is proposed. Optimization algorithm is developed to minimize the total system cost (the weighted sum of latency and energy consumption).

[0008] Finally, experimental simulation results verify the effectiveness and feasibility of the algorithm. The proposed algorithm can effectively reduce the total system cost during task offloading.

[0009] The purpose of the present invention is to solve the problem of overload of computing resources offloaded to MEC servers and delay of vehicle end users. In order to solve the problems of high energy consumption and joint computing offloading and resource allocation optimization based on deep reinforcement learning of edge computing, a novel edge computing joint computing offloading and resource allocation optimization method is proposed. Optimized algorithms can reduce the latency and energy consumption of vehicle-mounted terminal users, thereby improving system performance gains.

[0010] The parking assistance collaborative task unloading and resource allocation method is characterized by comprising the following steps: S1. For parked vehicles in road scenarios, the on-board units carried by the parked vehicles themselves have communication and computing resources. Mobile vehicles offload tasks to parked vehicles with idle resources to relieve the load pressure of MEC servers. The vehicle network consists of cloud servers, roadside parked vehicles, M RSUs, and N mobile vehicle users. MEC servers are deployed in each RSU to provide computing resources for vehicles. Roadside parked vehicles without computing tasks, RSUs, and cloud servers help vehicles that need computing. Tasks on mobile vehicles are calculated locally or offloaded to roadside parked vehicles, migrated to RSUs, or loaded to cloud servers through cellular networks for processing. In the absence of RSUs or insufficient local computing resources in vehicles, roadside parked vehicles are added as unloading platforms. S2. Establish a communication model, establish data transmission between mobile vehicles and RSU, and monitor the transmission delay and energy consumption of mobile vehicles offloading tasks to MEC; build a communication and computing model for parked vehicles, MEC servers, and cloud servers to collaboratively process tasks.

[0011] S3. Establish a computing offloading and resource allocation model to allocate tasks on mobile vehicles to local computing or offload to roadside parking vehicles; S4, all tasks are offloaded to local, MEC, cloud server or parked vehicle to minimize the total system cost, that is, the weighted sum of delay and energy consumption. Experimental simulation verifies that the proposed scheme has higher system performance than several other benchmark schemes under the same parameter conditions.

[0012] Compared with the existing research technology, the beneficial effects of the present invention are: 1: The present invention takes into account the parking probability and resource availability of parked vehicles and adopts an offloading node selection algorithm to select between vehicles and MEC. Multiple candidate offloading nodes are selected among the servers.

[0013] 2: This paper analyzes the system model of multi-parked vehicle assisted edge computing. Under the delay constraint, an offloading strategy is proposed to solve the load balancing and offloading optimization problems, including offloading node selection and workload distribution, to minimize the total system cost.

[0014] 3: This paper proposes a deep reinforcement learning algorithm that can effectively perform offloading decision selection and computing resource allocation, thereby improving the convergence speed and reducing system latency and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0016] Figure 1 A flow chart specifically allocated to the present invention; Figure 2 It is a schematic diagram of the system model of the present invention; Figure 3 Schematic diagram for comparing the solution of the present invention with four benchmark solutions under the same parameters. DETAILED DESCRIPTION

[0017] Step 1: In view of the large-scale access and massive data transmission of mobile vehicle users in modern urban road scenarios, The server resources are limited, and it is difficult to ensure the quality of user experience. However, there are always parked vehicles on both sides of the city roads. The vehicle-mounted unit has communication and computing resources. Therefore, in order to prevent a large amount of resource waste in the Internet of Vehicles, the mobile Vehicles can offload tasks to parked vehicles with idle resources, alleviating the load pressure on the MEC server. The network consists of cloud servers, roadside parked vehicles, M RSUs, and N mobile vehicle users. MEC servers are deployed in each RSU to provide computing resources for vehicles. Roadside parked vehicles, RSUs, and cloud servers that do not have computing tasks can help vehicles that need computing. Tasks on mobile vehicles can be computed locally, offloaded to roadside parked vehicles, and migrated to RSU, or loaded to the cloud server for processing via the cellular network.

[0018] Step 2: Build a communication model Communication model: The data transmission rate between the mobile vehicle and the RSU is as follows: (1) in represents the channel bandwidth between the vehicle and the RSU, is the bandwidth allocation factor, p represents the vehicle’s transmission power, represents the channel gain from the mobile vehicle to the RSU on the channel, Represents white noise.

[0019] The transmission rates between moving vehicles and parked vehicles on the roadside are as follows: (2) Where B2 represents the channel bandwidth between the vehicle and the roadside parked vehicles, Bandwidth allocation factor, p represents the vehicle’s Radiated power, It represents the channel gain from the moving vehicle to the roadside parked vehicle on the channel. Represents white noise.

[0020] Computational model: When the task of a mobile vehicle is calculated locally, the latency and energy consumption are expressed as: (3) (4) in represents the device power of mobile vehicle i. Since the task is only computed locally, there is no communication delay.

[0021] The transmission delay and energy consumption of mobile vehicles offloading tasks to MEC are expressed as: (5) (6) in Indicates the transmission power of the mobile vehicle uploading task Represents the power of the mobile edge computing server.

[0022] The transmission delay and energy consumption of the mobile vehicle offloading the task to the parked vehicle are expressed as: (7) (8) The transmission delay and energy consumption of the mobile vehicle offloading the task to the cloud server are expressed as: (9) (10) in represents the computing resources provided by the cloud server to the mobile vehicle, Indicates the device power of the cloud server.

[0023] Step 3: Build a computation offloading and resource allocation model The total cost of the system is minimized under the maximum tolerable delay Tmax and resource constraints. Therefore, the computation offloading and resource allocation of the system are modeled as follows: (11) Model (11) will be solved by obtaining the optimal offloading decision vector S and computing resource allocation vector F. More specifically, the offloading decision vector S will determine which layer to select to perform the computing task; the computing resource allocation vector F will determine how much computing resources to allocate to the task.

[0024] Due to the parallel processing of tasks, the total task processing delay is the maximum delay of local processing and task offloading processing, which can be expressed as: (12) in, Indicates the maximum delay of local task processing. The maximum latency for task offloading processing.

[0025] The total energy consumption of task processing can be written as: (13) in, Indicates the maximum energy consumption of local task processing, Processing maximum energy consumption for task offloading.

[0026] Since the optimal offloading decision vector S is a binary vector, the feasible set and objective function of model (11) are not convex. In addition, the size of the optimal S and F will increase rapidly with the number of computing tasks. This problem will be more serious due to the unstable topology of the IoV network. The traditional method is to transform non-convex problems into convex problems. However, these methods may not be suitable for the data size of the computing tasks, especially for multi-objective optimization problems. Therefore, we use deep reinforcement learning to obtain the optimal S and F, thereby obtaining the optimal solution to the problem. The detailed flowchart of the DRL algorithm based on task offloading and resource allocation proposed in the present invention is shown in Algorithm 1: Algorithm 1: DRL-based task offloading and resource allocation algorithm 1: Initialization: Use ω to initialize the parameters of DNN and initialize the experience playback memory 2: Input: Channel resources and vehicle computing resource information for each time slot t, including 3: Output: task offloading decision and resource allocation results, including 4: for each stable traffic topology do 5: Observe and obtain the status of the system 6: for each time slot t = 1, 2, . . . , do 7: Select Action according to Usage strategy . 8: Otherwise select an action. 9: Store the acquired data in the form of tuples in the experience replay memory . 10: Calculate the value of the target network . 11: Construct the error function. 12: Use experience to replay data tuples in memory To train the Q network. 13: Update the parameters of the main network based on the gradient descent algorithm. 14: end for 15: Update the parameters of the target network every κ steps. 16: Get task offloading decision and resource allocation results. 17: end for Step 4: In order to better evaluate the solution proposed by the present invention, we compare the solution proposed by the present invention with the other four benchmark solutions. For comparison, for baseline solution 1: all tasks only need to be performed on the mobile vehicle; for baseline solution 2: all The tasks are offloaded to the MEC server equipped with the current RSU; for baseline solution 3: no parked vehicles are introduced, and the vehicle tasks are offloaded to the local, MEC server, and cloud server for processing; for baseline solution 4: all tasks are randomly offloaded to the local, MEC, cloud server, and parked vehicles; Figure 3 It shows the system cost comparison of five unloading strategies under the same parameters. They are: (1) All-local strategy: All tasks only need to be performed on the mobile vehicle.

[0027] (2) All-MEC strategy: offload all tasks to the MEC server equipped with the current RSU.

[0028] (3) LMC strategy: No parked vehicles are introduced, and vehicle tasks are processed locally or offloaded to MEC servers or cloud servers.

[0029] (4) Random strategy: Computing tasks are randomly processed locally or offloaded to parked vehicles, MEC servers, and cloud servers for processing.

[0030] (5) Proposed strategy: This is the strategy proposed by this paper, which considers jointly computing the computing tasks locally or offloading them to parked vehicles, MEC and cloud servers, and dividing the tasks of specific vehicles according to the optimization results.

[0031] In the simulation scenario, 5 MEC servers, 10 parked vehicles, and a cloud server are set up to provide unloading services for the mission vehicles. Figure 3 It is reflected that the local strategy has the highest system cost, which is due to the low computing power and large computing delay of the local vehicle when performing the task. Compared with the local strategy, the tasks can be offloaded to the MEC server and parked vehicles, which have larger computing resources, so the system costs of the other four strategies are reduced.

[0032] In addition, it can be seen that the solution of the present invention has a lower system cost than the other four strategies under the same parameters. Therefore, the solution can reduce system cost, improve system performance, and thus reduce system delay and energy consumption.

[0033] The solution proposed in this invention is to jointly consider the computing tasks for local computing or offloading to parked vehicles, MEC and cloud servers, and divide the tasks of specific vehicles according to the optimization results. By setting the same parameters, we compared the performance differences between the solution proposed in this invention and the other four benchmark solutions.

[0034] The present invention constructs an offloading strategy for roadside parking in collaboration with mobile edge computing tasks. In this scheme, roadside parking is added as a task offloading platform, and three offloading platforms, namely local, RSU, and cloud server, are combined for task collaborative processing. A deep reinforcement learning algorithm for joint task offloading and resource allocation is proposed to solve the problems of latency and energy consumption. Simulation results show that compared with other offloading schemes, the proposed scheme can effectively reduce system overhead, latency and energy consumption, and is superior to the other four offloading schemes. Therefore, the task offloading strategy scheme of mobile edge computing in collaboration with roadside parking can more effectively reduce the total system cost in the task offloading process.

[0035] Finally, the details of the above examples of the present invention are only examples for explaining the present invention. For those skilled in the art, any modifications, improvements and substitutions of the above examples should be included in the protection scope of the claims of the present invention.

[0036] Glossary VEC collaborative task offloading: refers to offloading network function tasks from the virtualized evolved packet core (VEC) to the software-defined network (SDN) controller to achieve more flexible and efficient network traffic management and resource allocation. This technology can enable network operators to reduce network deployment and operation and maintenance costs, accelerate the deployment and update of network functions, and improve network flexibility and performance.

Claims

1. Parking assistance collaborative task offloading and resource allocation method, characterized in that The steps include: S1. For parked vehicles in road scenarios, the on-board units carried by the parked vehicles themselves have communication and computing resources. Mobile vehicles offload tasks to parked vehicles with idle resources to relieve the load pressure of MEC servers. The vehicle network consists of cloud servers, roadside parked vehicles, M RSUs, and N mobile vehicle users. MEC servers are deployed in each RSU to provide computing resources for vehicles. Roadside parked vehicles, RSUs, and cloud servers without computing tasks help vehicles that need computing. Tasks on mobile vehicles are computed locally or offloaded to roadside parked vehicles, migrated to RSUs, or loaded to cloud servers for processing via cellular networks; S2. Establish a communication model, establish data transmission between mobile vehicles and RSU, and monitor the transmission delay and energy consumption of mobile vehicles offloading tasks to MEC; S3, establish a computing offloading and resource allocation model to allocate tasks on mobile vehicles to local computing or offload to roadside parking vehicles; S4, offload all tasks to local, MEC, cloud servers or parked vehicles to minimize the total system cost, i.e., the weighted sum of latency and energy consumption.

2. The parking assistance collaborative task offloading and resource allocation method according to claim 1 is characterized in that The communication model is established in S2 Communication model: The data transmission rate between the mobile vehicle and the RSU is as follows: in represents the channel bandwidth between the vehicle and the RSU, is the bandwidth allocation factor, p represents the vehicle’s transmission power, represents the channel gain from the mobile vehicle to the RSU on the channel, represents white noise; The transmission rates between moving vehicles and parked vehicles on the roadside are as follows: （2） in represents the channel bandwidth between the vehicle and the parked vehicles on the roadside, Bandwidth allocation factor, p represents the vehicle’s Radiated power, It represents the channel gain from the moving vehicle to the roadside parked vehicle on the channel. represents white noise; Computational model: When the task of a mobile vehicle is calculated locally, the latency and energy consumption are expressed as: （3） （4） in represents the device power of mobile vehicle i. Since the task is only calculated locally, there is no communication delay; The transmission delay and energy consumption of mobile vehicles offloading tasks to MEC are expressed as: （5） （6） in represents the transmission power of the mobile vehicle uploading task, represents the power of the mobile edge computing server; the transmission delay and energy consumption of the mobile vehicle offloading the task to the parked vehicle are respectively expressed as: （7） （8） The transmission delay and energy consumption of the mobile vehicle offloading the task to the cloud server are expressed as: （9） （10） in represents the computing resources provided by the cloud server to the mobile vehicle, Indicates the device power of the cloud server.

3. The parking assistance collaborative task offloading and resource allocation method according to claim 1, characterized in that The computation offloading and resource allocation model is established in S3 The total cost of the system is minimized under the maximum tolerable delay Tmax and resource constraints. Therefore, the computation offloading and resource allocation of the system are modeled as follows: （11） (11) is solved by obtaining the optimal offloading decision vector S and computing resource allocation vector F. More specifically, the offloading decision vector S will determine which layer is selected to perform the computing task; the computing resource allocation vector F will determine how much computing resources are allocated to the task; Due to the parallel processing of tasks, the total task processing delay is the maximum delay of local processing and task offloading processing, expressed as: （12） in, Indicates the maximum delay of local task processing. Maximum latency for task offloading processing; The total energy consumption of task processing is written as: （13） in, Indicates the maximum energy consumption of local task processing, Processing maximum energy consumption for task offloading.

Images