A low-energy task migration and resource scheduling method

Abstract

The present application relates to a kind of low energy consumption task migration and resource scheduling method, belong to communication field.For the scene of multiple computing services coexistence in Internet of Vehicles, due to the difference of different service performance indicators, it is difficult to migrate and schedule tasks, which further leads to excessive energy consumption of vehicle system, reducing the endurance capability.Among them, according to the characteristics of different services generated by vehicles, the energy consumption and delay in multi-service scenario are modeled and analyzed.Then, a V2I link allocation, power control and task migration method is designed to reduce the energy consumption of vehicle system by coordinating local vehicle computing, auxiliary vehicle computing and edge computing.Finally, according to the nature of the problem, the problem is modeled as a Markov decision process, and multi-agent deep reinforcement learning is used to solve it to ensure that vehicles can achieve adaptive real-time task migration and resource scheduling in multi-service and dynamic edge environment.

Description

Technical Field

[0001] This invention belongs to the field of communications and relates to a low-energy task migration and resource scheduling method. Background Technology

[0002] Currently, with the development of wireless communication technology, humanity is being invited into a new era of the Internet of Vehicles (IoV), where vehicles can not only access the internet but also interact with each other. Therefore, Vehicular Edge Computing (VEC) has been proposed, supporting in-vehicle entertainment services such as video-on-demand and online conferencing through collaboration between vehicles and RSUs (Roadside Units). In particular, with increasingly powerful computing capabilities being introduced into vehicles, vehicle collaboration is expected to provide more types of services for the IoV, such as mixed reality, autonomous driving, and the metaverse. In this regard, VEC has evolved into a new computing paradigm called edge-end collaboration, where the edge refers to the RSU or auxiliary vehicle, and the end refers to the vehicle under consideration for the task.

[0003] A fundamental challenge in achieving efficient collaboration between terminals and the edge in the Internet of Things (IoV) is ensuring the performance requirements of various services by collaboratively optimizing network resources and task scheduling. On one hand, future vehicle services will be highly sensitive to both task latency and throughput. Due to the mobility of vehicles, optimizing edge-device collaboration in IoV is more challenging than in traditional IoT. On the other hand, energy consumption due to communication and computing accounts for a significant proportion of global energy consumption. To sustain real-time or latency-sensitive tasks in IoV edge-device collaboration, upcoming B5G / 6G and AI technologies will consume even more energy, as automotive energy storage is typically limited and lacks external replenishment.

[0004] The common approach to addressing the aforementioned vehicle-mounted tasks is to unify task modeling, process computational tasks using edge-end collaboration or cloud-edge collaboration, and schedule tasks and resources using rule-based or heuristic algorithms. However, inappropriate task migration and resource scheduling can lead to channel congestion and wasted computing resources, resulting in excessive energy consumption. This invention proposes a low-energy task migration and resource scheduling method. By reasonably considering the characteristics of different vehicle-mounted tasks, it models task latency and models separately, and employs deep reinforcement learning to enable vehicles to adaptively and in real-time make scheduling strategies in dynamic IoV environments. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a low-energy task migration and resource scheduling method.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A low-energy task migration and resource scheduling method, comprising the following steps:

[0008] S1: Based on the latency requirements of services in the Internet of Vehicles, vehicle-mounted tasks are divided into ordinary tasks and emergency tasks;

[0009] Ordinary tasks are computationally intensive, and the task processing method is collaborative computation between the local unit and the roadside unit (RSU).

[0010] Emergency tasks are time-sensitive tasks, and the task processing method is collaborative computing between local and neighboring auxiliary vehicles;

[0011] Emergency missions use vehicle-to-vehicle (V2V) wireless links and share vehicle-to-infrastructure (V2I) wireless links used by normal missions. Latency and energy consumption are modeled for both types of missions.

[0012] S2: Based on the latency and energy consumption in S1, design a multi-service energy consumption model;

[0013] S3: Vehicle-mounted tasks are constantly generated, and the vehicle's handling of these tasks constitutes a sequential decision problem. This problem is modeled as a Markov decision process, and multi-agent deep reinforcement learning is used to solve the problem and train the model.

[0014] S4: Based on the model trained in S3, deploy the model to the vehicle for application.

[0015] Optionally, S1 specifically includes the following steps:

[0016] S11: Discretize time, all tasks are divisible, and the vehicle dispatcher starts scheduling and processing emergency subtasks and normal subtasks at the beginning of each time slot.

[0017] S12: Model the latency of emergency tasks and normal tasks respectively;

[0018] The delay for emergency tasks consists of two parts, expressed as follows:

[0019]

[0020] in, This indicates the latency of executing an emergency task locally. This indicates the delay in the auxiliary vehicle's task processing;

[0021] The latency of a normal task consists of two parts, represented as follows:

[0022]

[0023] in, This indicates the latency of a normal task executing locally. This indicates the latency of processing tasks at RSU.

[0024] S13: Based on the task latency of S12, combined with the computing power of each edge and the power of different link transmitters, construct the energy consumption model for different tasks.

[0025] The energy consumption of an emergency mission comes from local computing power consumption and the energy consumption incurred during mission migration, and is expressed as:

[0026]

[0027] in, This indicates the energy consumption of performing emergency tasks locally. This indicates the energy consumption of auxiliary vehicles in handling tasks; by controlling transmitter power, V2V link selection, and task scheduling, the energy consumption caused by ineffective scheduling methods is reduced.

[0028] The energy consumption of a normal task comes from local computing energy consumption and the energy consumption caused by task migration, and is expressed as:

[0029]

[0030] in, This represents the energy consumption of a normal task executed locally. This indicates the energy consumption caused by task migration.

[0031] Optionally, S3 specifically includes the following steps:

[0032] S31: Define the state space of the agent as wireless channel information, the computing power of the assisted vehicle, the current onboard computing power, the power gain of the V2I link, the power gain of the V2V link, and the number of training rounds; the action space is the offloading strategy for emergency tasks, spectrum resource allocation and power selection, and the offloading strategy and power selection for ordinary tasks. The reward function is given by the optimization problem in S2.

[0033] S32: Treat the vehicle as an intelligent agent, design and initialize a reinforcement learning model for actors and critics;

[0034] S33: Design optimization objectives for each network in the model and train the multi-agent system until the rewards tend to converge and are free from fluctuations due to noise.

[0035] Optionally, in S4, the scheduler algorithm comes from the deep reinforcement learning network trained and converged in S3. The corresponding policy network is deployed in the vehicle system. In the actual driving environment, the vehicle uses the information collected by the sensors as the input of the algorithm in the state mode given in S31 to obtain the task migration and resource scheduling strategies for different services.

[0036] The beneficial effects of this invention are as follows: First, based on the characteristics of different services generated by vehicles, energy consumption and latency in multi-service scenarios are modeled and analyzed; second, a method for V2I link allocation, power control, and task migration is designed in collaboration with local on-board computing, auxiliary vehicle computing, and edge computing to reduce the energy consumption of the vehicle system; finally, based on the nature of the problem, the problem is modeled as a Markov decision process, and multi-agent reinforcement learning is used to enable the vehicle to adaptively perform task migration and resource scheduling in complex and dynamic IoV scenarios.

[0037] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0038] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0039] Figure 1 This invention provides a multi-service task migration framework.

[0040] Figure 2 This is a diagram of the task migration and resource scheduling architecture of the present invention. Detailed Implementation

[0041] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0042] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0043] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0044] like Figure 1 and Figure 2 As shown, the low-energy task migration and resource scheduling method proposed in this invention specifically includes the following steps:

[0045] Step 1: Based on the latency requirements of services in the Internet of Vehicles (IoV), onboard tasks are divided into ordinary tasks and emergency tasks. Ordinary tasks include a large number of computationally intensive tasks, so the task processing method is designed as collaborative computing between the local vehicle and roadside units (RSUs). Emergency tasks include a large number of latency-sensitive tasks, so the task processing method is designed as collaborative computing between the local vehicle and neighboring auxiliary vehicles. This invention does not design separate wireless transmission channels for the two types of tasks, but instead uses the vehicle-to-vehicle (V2V) wireless link used for emergency task migration to share the vehicle-to-infrastructure (V2I) wireless link used for ordinary task migration, thereby improving spectrum efficiency. The latency and energy consumption of the two types of tasks are then modeled separately, specifically including the following steps:

[0046] Step 11: Discretize the time. All tasks are divisible, so the vehicle scheduler needs to schedule and process urgent and normal subtasks at the beginning of each time slot. The vehicle set is represented as follows: The representation of a V2V link is the same as that of a vehicle, because each vehicle corresponds to only one V2V link. The set of V2I links is represented as follows: Each vehicle will be assigned a separate V2I link, while multiple V2V links may share the same V2I link;

[0047] Step 12: Since the computational complexity and latency requirements of different tasks vary, latency models are created for urgent tasks and normal tasks separately. Based on the task processing method designed in Step 1, the latency of an urgent task consists of two parts, which can be expressed as:

[0048]

[0049] in, This indicates the latency of executing an emergency task locally. This represents the latency of task processing in the auxiliary vehicle. The local execution of emergency subtasks only needs to consider the local real-time computing power and the computational load of the subtask. However, migrating the task to the auxiliary vehicle requires transmitting the computational task to the neighboring vehicle via a V2V wireless link. Because the V2V link reuses the V2I link, interference from other V2V and V2I links exists in the wireless link and needs to be included in the transmission rate modeling. Since the amount of data after computation is often much smaller than the original data, the latency of returning the computation result is not considered. Specifically, the local execution latency of the emergency task is expressed as:

[0050]

[0051] in, For time slot interval, For time slot set, It is a binary indicator variable. This indicates that the subtask is computed locally; otherwise... Migrate to auxiliary vehicles.

[0052] If the defined time slot interval guarantees that a subtask transmitted within a time slot can complete its calculation within that time slot interval, then the execution delay of the emergency task on the auxiliary vehicle is given by the calculation delay and the transmission delay:

[0053]

[0054] in, It is a binary indicator variable, if the vehicle V2V link sharing The spectrum resources of each V2I link, ,otherwise , This indicates the computing resources required to process 1 bit of this task. Indicates that it can be used to process vehicles Computational resources of adjacent vehicles in the unloading task

[0055]

[0056] For vehicles V2V links in Occupying the first time slot Data transmission rate of a V2I link, For channel bandwidth, The signal-to-noise ratio is expressed as:

[0057]

[0058] in, Indicates car Transmission power on V2V links, This represents noise power. Additionally, it represents interference power. Represented as:

[0059]

[0060] in, express Vehicles within the time slot Power transmitted between the RSU and the RSU. and Indicates vehicle V2V links in Time slot number Transmission power and interference channel gain on a V2I link.

[0061] Similarly, the processing latency of ordinary tasks also comes from two parts, which can be expressed as:

[0062]

[0063] in, This indicates the latency of a normal task executing locally. This represents the latency of task processing at the RSU. Local computation considerations are similar to those for emergency tasks. The task migration method involves allocating an independent V2I link for each vehicle in the system. However, this can lead to wireless interference as other vehicles reuse this link for V2V purposes, so this also needs to be considered in the design. Specifically, the latency of local computation for normal tasks is expressed as:

[0064]

[0065] in, As an indicator variable, This indicates that the corresponding subtask is processed locally. This indicates that the task will be unloaded to RSU.

[0066] Subtasks in If the time shifts to RSU, the execution latency of this subtask includes transmission latency and computation latency.

[0067] The delay is represented as:

[0068]

[0069] in, For vehicles In the The data transmission rate on a V2I link is expressed as:

[0070]

[0071] in, For vehicles In the time slot Through the first The received signal-to-interference-plus-noise ratio (SIR) of a V2I link can be expressed as:

[0072]

[0073] Interference generated by reusing this V2V link is represented as follows:

[0074]

[0075] Step 13: Based on the task latency in Step 12, and considering the computing power of each endpoint and the power of different link transmitters, construct energy consumption models for different tasks. The energy consumption of emergency tasks, consisting of local computing energy consumption and energy consumption due to task migration, is represented as follows:

[0076]

[0077] in, This indicates the energy consumption of performing emergency tasks locally. This refers to the energy consumption of auxiliary vehicles in handling tasks. By controlling transmitter power, V2V link selection, and task scheduling, energy consumption caused by ineffective scheduling can be effectively reduced. Specifically, the vehicle... The energy consumption for performing emergency tasks locally is:

[0078]

[0079] in, This indicates the local computing power required to perform urgent tasks. and As a constant related to the hardware characteristics of the computing chip, energy is consumed during data transfer and task computation when the task is offloaded to an adjacent vehicle. At this time, the energy consumption for task execution is:

[0080]

[0081] Similarly, vehicles The local computing power for performing ordinary tasks is ,in, f represents a vehicle Allocate computing resources for handling routine tasks. Based on this, the vehicle's performance can be calculated. The energy consumption when performing a normal task locally is:

[0082]

[0083] Since the RSU has an external power supply, only the vehicle's energy consumption is considered when handling normal tasks. In this case, the energy consumption for performing the task is:

[0084]

[0085] Step 2: To optimize the energy consumption of the vehicle system under latency constraints and improve the vehicle's range, based on the latency and energy consumption in Step 1, a multi-service energy consumption model is designed as follows:

[0086]

[0087] Step 3: Since the vehicle-mounted tasks are continuously generated and the vehicle's handling of these tasks constitutes a sequential decision-making problem, the problem is modeled as a Markov decision process, and multi-agent deep reinforcement learning is used to solve the problem. Specifically, this includes the following steps:

[0088] Step 31: Define the agent's state space as wireless channel information, the assisted vehicle's computing power, the current onboard computing power, V2I link power gain, V2V link power gain, and the number of training rounds; define the action space as the offloading strategy for emergency tasks, spectrum resource allocation and power selection, and the offloading strategy and power selection for ordinary tasks. The reward function is given by the optimization problem in Step 2. ,in, for Energy consumption of all vehicles in the system within the time slot. This is a penalty item; if the task processing time exceeds a threshold, a constant will be applied as a penalty. If the task processing time does not exceed the threshold, there is no penalty. and This is to balance the relationship between energy consumption and penalties;

[0089] Step 32: Treat the vehicle as an intelligent agent, design and initialize the actor-critic reinforcement learning model;

[0090] Step 33: Design optimization objectives for each network in the model and train the multi-agent system until the rewards tend to converge and are less affected by noise fluctuations. Specifically, design the value network loss function as follows:

[0091]

[0092] in, For policy network parameters, These are parameters related to the algorithm, called temperature parameters. The policy network loss function is:

[0093]

[0094] in, These are the parameters for the policy network. During training, each agent maintains an experience replay pool. When a certain amount of data has accumulated, the agent uses the data from the experience replay pool to update the parameters of the value network using gradient descent, and to update the parameters of the policy network using gradient ascent.

[0095] Step 4: Based on the model trained in Step 3, deploy the model to the vehicle for use in a real-world environment. Simply deploy the corresponding policy network in the vehicle system. In a real-world driving environment, the vehicle uses the information collected by the sensors, presented in Step 3.1, as input to the algorithm to obtain task migration and resource scheduling strategies for different services.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A low-energy task migration and resource scheduling method, characterized in that: The method includes the following steps: S1: Based on the latency requirements of services in the Internet of Vehicles, vehicle-mounted tasks are divided into ordinary tasks and emergency tasks; Ordinary tasks are computationally intensive, and the task processing method is collaborative computation between the local unit and the roadside unit (RSU). Emergency tasks are time-sensitive tasks, and the task processing method is collaborative computing between local and neighboring auxiliary vehicles; Emergency missions use vehicle-to-vehicle (V2V) wireless links and share vehicle-to-infrastructure (V2I) wireless links used by normal missions. Latency and energy consumption are modeled for both types of missions. Step 12: Different tasks have different computational complexity and latency requirements. Latency models are created for urgent tasks and regular tasks separately. Based on the task processing method designed in Step 1, the latency of an urgent task consists of two parts, represented as follows: in, This indicates the latency of executing an emergency task locally. This represents the latency of the auxiliary vehicle processing the task; the local execution method of the emergency subtask only considers the local real-time computing power and the computational load of the subtask, and migrates the task to the auxiliary vehicle to transmit the computation task to the neighboring vehicle through the V2V wireless link. The interference from other V2V links and V2I links in the wireless link is included in the transmission rate modeling, and the latency of the computation result return is not considered; the local execution latency of the emergency task is expressed as: in, For time slot interval, For time slot set, It is a binary indicator variable. This indicates that the subtask is computed locally; otherwise... Migrate to auxiliary vehicles; If the defined time slot interval guarantees that a subtask transmitted within a time slot can complete its calculation within that time slot interval, then the execution delay of the emergency task on the auxiliary vehicle is given by the calculation delay and the transmission delay: in, It is a binary indicator variable, if the vehicle V2V link sharing The spectrum resources of each V2I link, ,otherwise , This indicates the computing resources required to process 1 bit of this task. Indicates that it can be used to process vehicles Computational resources of adjacent vehicles in the unloading task: For vehicles V2V links in Occupying the first time slot Data transmission rate of a V2I link, For channel bandwidth, The signal-to-noise ratio is expressed as: in, Indicates car Transmission power on V2V links, Noise power; additionally, interference power. Represented as: in, express Vehicles within the time slot Power transfer between the RSU and the RSU; and Indicates vehicle V2V links in Time slot number Transmission power and interference channel gain on a V2I link; The processing latency of a normal task can be expressed as: in, This indicates the latency of a normal task executing locally. This represents the latency of processing tasks at RSU; the latency of local computation for ordinary tasks is represented as: in, As an indicator variable, This indicates that the corresponding subtask is processed locally. This indicates that the task will be offloaded to the RSU; Subtasks in If the time shifts to RSU, the execution latency of this subtask includes transmission latency and computation latency. The delay is represented as: in, For vehicles In the The data transmission rate on a V2I link is expressed as: in, For vehicles In the time slot Through the first The received signal-to-interference-plus-noise ratio (SIR) of a V2I link is expressed as: Interference generated by reusing this V2V link is represented as follows: Step 13: Based on the task latency in Step 12, and considering the computing power of each endpoint and the power of different link transmitters, construct energy consumption models for different tasks; the energy consumption of emergency tasks, consisting of local computing energy consumption and energy consumption due to task migration, is represented as follows: in, This indicates the energy consumption of performing emergency tasks locally. This indicates the energy consumption of auxiliary vehicles in handling tasks; by controlling transmitter power, V2V link selection, and task scheduling, energy consumption caused by ineffective scheduling methods is reduced; vehicle The energy consumption for performing emergency tasks locally is: in, This indicates the local computing power required to perform urgent tasks. and As a constant related to the hardware characteristics of the computing chip, energy is consumed during data transmission and task computation when the task is offloaded to an adjacent vehicle; at this time, the energy consumption for task execution is: vehicle The local computing power for performing ordinary tasks is ,in, f represents a vehicle Allocate computing resources for handling routine tasks; calculate vehicle... The energy consumption when performing a normal task locally is: The RSU has an external power supply, and only the vehicle's energy consumption is considered when handling normal tasks; the energy consumption for performing tasks is: S2: Based on the latency and energy consumption in step 1, design a multi-service energy consumption model as follows: S3: Vehicle-mounted tasks are constantly generated, and the vehicle's handling of these tasks constitutes a sequential decision problem. This problem is modeled as a Markov decision process, and multi-agent deep reinforcement learning is used to solve the problem and train the model. S4: Based on the model trained in S3, deploy the model to the vehicle for application.

Images