Fine-grained task unloading method and device based on directed acyclic graph dependence

Abstract

The invention discloses a fine-grained task unloading method and device based on directed acyclic graph dependency, and the method comprises the steps: 1) carrying out the modeling of a complex application in an Internet of Vehicles environment through a directed acyclic graph, and determining a task topology mapping rule based on a dependency relationship in the graph; and 2) modeling is carried out on a fine-grained task unloading problem by taking user satisfaction as an optimization target, and in calculation unloading, a calculation task is unloaded to an edge server or locally executed in a vehicle. 3) expressing an optimization problem as a Markov decision process, and defining a state space, an action space and a reward function; and 4) designing an S2S neural network architecture based on a recurrent neural network, representing the DAG tasks after priority ranking as a series of task vector sequences, training a network model by applying a deep reinforcement learning algorithm, and gradually optimizing a strategy through iterative training so as to obtain good performance in actual implementation.

Description

Technical Field

[0001] The present invention relates to a directed acyclic graph dependency-based fine-grained task offloading method and device, and relates to cross-technical fields such as reinforcement learning and smart transportation. Background Art

[0002] With the rapid development of Internet of Vehicles technology, computing-intensive and latency-sensitive applications such as autonomous driving, collision detection, and augmented reality continue to emerge, placing higher demands on the computing power of vehicles. Behind the rise of this technology is people's pursuit of safer and smarter transportation methods, as well as the demand for continuous improvement of vehicle performance and intelligent functions. However, insufficient computing power has become the key to its development. In this context, mobile edge computing, as the intersection of wireless network edge and infrastructure edge, aims to sink powerful computing and data analysis capabilities to a location close to the data source through the deep integration of cloud computing and IoT technology, so as to achieve ultra-low latency and ultra-high efficiency data processing and exchange. In particular, with the support of artificial intelligence algorithms, mobile edge computing will further develop into edge intelligence, giving wireless network edge devices stronger intelligent analysis and decision-making capabilities. By leveraging the collaborative learning between the cloud and edge servers, vehicles can obtain big data support from the cloud, while performing real-time data processing and analysis on the edge servers. In this way, future in-vehicle applications are expected to obtain more efficient and convenient intelligent services.

[0003] However, with the rapid development of this field, it still faces a series of challenges. Most of the existing studies explore the Internet of Vehicles scenario where the internal modules are tightly coupled. In this scenario, the system tends to treat all tasks as a complete unit for unified processing. Taking autonomous vehicles as an example, there is a deep interaction between multiple modules involved, such as path planning, environmental perception, and vehicle control. If these tasks are simply offloaded to edge computing nodes or cloud servers, it is impossible to fully utilize the advantages of distributed computing resources, nor can it effectively respond to rapidly changing and diversified user needs. In addition, when different subtasks are assigned to different execution units for parallel processing, the offloading decision problem is intertwined with the task scheduling problem in a heterogeneous environment, forming a more complex optimization problem. Not only is it necessary to determine the best execution location for each task, that is, whether to execute locally or remotely, but it is also necessary to design the execution order and time schedule of the tasks to ensure that user satisfaction is maximized while meeting the task execution efficiency.

[0004] Therefore, in order to better cope with these challenges, future research needs to take into account the dependencies between different tasks and the dynamics in heterogeneous environments. Through more precise task offloading strategies, the overall performance and efficiency of the Internet of Vehicles system can be improved, thereby better meeting user needs and promoting the development of technology. Summary of the invention

[0005] The purpose of the present invention is to address the shortcomings of existing methods and to propose a fine-grained task offloading method and device based on directed acyclic graph dependencies.

[0006] The present invention uses directed acyclic graphs to model complex applications, constructs a topological sorting sequence of directed acyclic graphs, adds priority information of task nodes, and meets the requirement of "the higher the execution cost, the higher the priority". At the same time, the fine-grained task offloading problem is modeled as an optimization problem with user satisfaction as the goal, and the optimization problem is expressed as a Markov decision process, and a deep reinforcement learning algorithm is introduced to optimize the task offloading decision.

[0007] A first aspect of the present invention relates to a fine-grained task offloading method based on directed acyclic graph dependency, comprising the following steps:

[0008] S1: Use directed acyclic graphs to model complex applications and formulate task topology mapping rules based on graph dependencies;

[0009] S2: Modeling the fine-grained task offloading problem with user satisfaction as the optimization objective;

[0010] S3: Using the optimization problem described in S2 to establish a Markov decision process model, and optimize the decision process;

[0011] S4: Using the Markov decision process model mentioned in S3, a reinforcement learning algorithm is used to determine the task offloading strategy and implement it.

[0012] S1 specifically includes the following steps:

[0013] S1.1: According to the task node t i The time cost and task node t of local calculation i Fixed transfer cost calculated locally Calculate the execution cost of the task node to process the local calculation

[0014] S1.2: According to the task node t i The time cost and task node t calculated on the remote server i Fixed transfer cost of execution on remote server to calculate the execution cost of task node on remote offload

[0015] S1.3: Calculate the SLA index SLA(i) of each task node according to S1.1 and S1.2;

[0016] S1.4: Arrange all task nodes in the directed acyclic graph in descending order of SLA priority to obtain a topological sorting sequence index of the DAG, where index[1] represents the task node with the highest priority and index[|V|] represents the task node with the lowest priority.

[0017] S2 specifically includes the following steps:

[0018] S2.1: When the vehicle's computing task is executed on the local server, the vehicle's CPU will provide computing resources to complete the processing. First, define the computing rate of the vehicle's own CPU as f local , since the task is only executed locally, there will be no transmission delay, so executing the task locally i The resulting delay can be expressed as follows:

[0019]

[0020] Among them, C i Represents task t i The CPU cycles required.

[0021] S2.2: In the remote execution scenario, when the vehicle needs to offload the task to the edge server, the total delay of task processing includes not only the time required to complete the calculation on the server, that is, the calculation delay, but also the time to transmit the data from the vehicle to the edge server, that is, the transmission delay. Considering that the MBS and RSUs are connected via wired optical fiber, this article does not take the transmission delay into consideration. Therefore, when the task is offloaded to the edge server via the uplink, the specific formula for the transmission delay is as follows:

[0022]

[0023] Among them, data i represents the size of the task data generated by vehicle i, Indicates the uplink transmission rate.

[0024] When the edge service receives the task transmitted by the vehicle, it can start calculating. The specific formula for calculating the delay is as follows:

[0025]

[0026] Among them, C i Represents task t i The CPU cycles required, f server Indicates the CPU computing rate of the server.

[0027] S2.3: Determine the task offloading optimization problem based on S2.1 and S2.2. The goal of the optimization problem is to minimize the total delay, that is, to maximize the satisfaction.

[0028] S3 specifically includes the following steps:

[0029] S3.1: Determine the state space and use the symbol S to represent the state of MEC resources;

[0030] S3.2: Determine the action space. The system obtains the status information of the vehicle and the server by observing the vehicle edge network in time slot t, and makes task offloading decisions based on this. The action space is represented by the symbol a(t);

[0031] S3.3: Define the reward function, which defines a reward or cost value for each state and action pair, using the symbol r i It indicates that the reward function is expressed as user satisfaction.

[0032] S4 specifically includes the following steps:

[0033] S4.1: A recurrent neural network is used to construct encoder and decoder units in the network. The encoder unit is responsible for receiving the task embedding sequence [t1, t2, ..., t n ] as input, and further transforms these sequences to extract feature descriptions corresponding to directed acyclic graphs. At the same time, the decoder initializes its hidden state based on the feature output provided by the encoder and begins to gradually process historical unloading decision information. Through this process, the decoder continuously generates estimates of the strategy and value function, and then gradually derives the next action until the final action sequence [a1, a2, ..., a n ];

[0034] S4.2: Initialize the policy network π θ and value network V φ , where θ and φ represent the parameters of their respective networks;

[0035] S4.3: Adopt current strategy π θ Perform several actions in the environment and collect a series of data trajectories {s t ,a t ,r t ,s t+1};

[0036] S4.4: At the same time, for each time step t, the algorithm calculates an advantage estimate to assess the additional value of taking a specific action relative to the average reward;

[0037] S4.5: Update the policy network parameters θ to optimize the objective function L(θ);

[0038] S4.6: Update the value network V φ to more accurately estimate the value of the state;

[0039] S4.7: Continue to update the above process until the maximum number of iterations is reached.

[0040] The second aspect of the present invention relates to a fine-grained task offloading device based on directed acyclic graph dependencies, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a fine-grained task offloading method based on directed acyclic graph dependencies of the present invention.

[0041] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a directed acyclic graph dependency-based fine-grained task offloading method of the present invention.

[0042] The working principle of the present invention is (the reason why the advantages of the present invention arise):

[0043] The present invention uses a directed acyclic graph (DAG) to model tasks in a connected vehicle environment, subdividing the overall task into multiple subtasks. By analyzing the dependencies between different tasks, a task topology sequence based on the subtask dependencies is established.

[0044] On this basis, we model the task offloading problem with user satisfaction as the optimization goal. We formulate the optimization problem as a Markov decision process and design a network structure based on a recurrent neural network (RNN) to solve the problem. At the same time, we integrate the attention mechanism to better capture the long-term dependencies between task sequences, thereby improving the performance and efficiency of the model.

[0045] The advantages of the present invention are:

[0046] 1. Reduced task complexity: The overall task is subdivided into multiple subtasks through a directed acyclic graph (DAG), and a topological sequence between tasks is established, which significantly reduces the task complexity in the Internet of Vehicles environment.

[0047] 2. Maximize user satisfaction: Take user satisfaction as the optimization goal to ensure that task offloading, task execution delay, and resource allocation can meet user needs and improve service quality.

[0048] 3. Capturing long-term dependencies: The model based on recurrent neural network (RNN) and attention mechanism can better capture the long-term dependencies between task sequences and make more reasonable task offloading decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1is a structural diagram of the method of the present invention;

[0050] Figure 2 It is a directed acyclic graph task schematic diagram of the present invention;

[0051] Figure 3 is a system architecture diagram of the present invention;

[0052] Figure 4 This is a network architecture diagram of the present invention.

[0053] Figure 5 It is a structural diagram of the device of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present invention more clear, the specific implementation modes of the present invention will be further described in detail below.

[0055] Example 1

[0056] Reference Figure 1-Figure 4 , the embodiment of the present invention provides a fine-grained task offloading method based on directed acyclic graph dependency. The structure diagram is as follows Figure 1 As shown, the method includes:

[0057] S1: Considering the dependencies between DAG task nodes, the node offloading decision must follow the topological order of DAG. The task diagram is as follows: Figure 2 As shown. This strategy can ensure that all decisions of the predecessor nodes are completed before the successor nodes. The service level agreement (SLA) calculation rule adopted is based on the execution cost: the higher the execution cost, the higher the service level of the task node. The specific steps are as follows:

[0058] S1.1: Calculate the execution cost of local computing processing: task node t i The execution cost of local computation processing is expressed as:

[0059]

[0060] in, Represents task node t i The time cost of local calculation, Represents task node t i The fixed transfer cost calculated locally, ο(i) represents the task node t i All successor nodes;

[0061] S1.2: Calculate the execution cost of remote offloading processing: Task node t i The execution cost of remote offloading is expressed as:

[0062]

[0063] in, Represents task node t i The time cost of computing on the remote server, Represents task node t i Fixed delivery cost for execution on remote servers.

[0064] S1.3: Calculate the SLA index: Based on the execution cost calculated in this article, the SLA index of each task node can be obtained, which can be expressed as:

[0065]

[0066] Among them, v i ∈v exit Represents a collection of exit tasks. Indicates the final execution cost of the task node.

[0067] S1.4: By arranging all task nodes in a directed acyclic graph (DAG) in descending order of SLA priority, a topological sorting sequence index of the DAG is obtained, where index[1] represents the task node with the highest priority, and index[|V|] represents the task node with the lowest priority.

[0068] S2: In computation offloading, the computation task can be offloaded to the edge server or executed locally in the vehicle. i To be uninstalled, execute i There are three steps. First, the vehicle transmits t i The edge server then runs the received task. Finally, i The running results are returned to the vehicle. Therefore, task offloading can be divided into two cases, namely local execution and offload execution. The system architecture diagram is as follows: Figure 3 The specific steps are as follows:

[0069] S2.1: Calculate the task t executed locally i The delay caused C i Represents task t i The CPU cycles required, f local Indicates the calculation rate of the vehicle's own CPU;

[0070] S2.2: Transmission delay caused when computing tasks are offloaded to edge servers via uplink And the delay caused by calculation data irepresents the size of the task data generated by vehicle i, Indicates the uplink transmission rate, C i Represents task t i The CPU cycles required, f server Indicates the CPU computing rate of the server.

[0071] S2.5: Determine the task offloading optimization problem based on S2.1 and S2.2. The goal of the optimization problem is to minimize the total delay, that is, to maximize the satisfaction.

[0072] The constraints are as follows:

[0073] (1) If the task node t i If it is decided to be executed locally, there is no edge server to offload the computation and transmission, that is, the execution completion time ET i up =0, ET i server =0;

[0074] (2) If the task node t i If it is decided to be executed remotely, there is no local computation completion time, i.e., ET i local =0.

[0075] The available time of a resource depends on the completion time of the previous task node on the resource. i The previous task node did not use this resource, so the completion time on this resource is set to 0. i Offload to the MEC host, only when all its parent tasks are completed and the uplink is available, the task node t i Therefore, the uplink transmission completion time is expressed as:

[0076]

[0077] When the task node is transmitted to the edge server, it is processed by the edge server. i When the edge server is processing, the CPU of the edge server must be available and the task node t i The data has been uploaded successfully, and the task node t i All the previous task nodes of have completed the computation execution. Therefore, task node t i The processing completion time on the edge server is expressed as:

[0078]

[0079] If the task node ti When scheduling locally, the start time of a task depends on the completion time of its parent task and the local available time, which means that the vehicle’s local CPU must be available and all previous task nodes have been calculated and executed. Therefore, the task node t i The processing completion time on the local is expressed as:

[0080]

[0081] In this paper, the offloading decision sequence of directed acyclic graph (DAG) is represented as A = {a1, a2, ..., a n}, where a i represents the offloading decision of the i-th task. i The value of is based on the scheduling plan, that is, a i =0 indicates task node t i Calculated locally in the vehicle, a i =1 indicates task node t i The calculation is performed on the remote MEC server. The overall goal of the optimization problem is to find an effective offloading plan so that the directed acyclic graph (DAG) obtains the minimum total delay, which can be expressed as:

[0082]

[0083] S3: Use the optimization problem described in S2 to establish a Markov decision process model and optimize the decision process. The specific steps are as follows:

[0084] S3.1: In the scheduling task t i When ti is set, the latency of the task depends on the number of CPU cycles required by the task node, the data size, the DAG topology, the wireless transmission rate, and the state of the MEC resources. According to the constraints on the offloading decision in the problem modeling, the state of the MEC resources is related to the offloading decision arranged before the task ti. Therefore, the state space can be defined as follows:

[0085] S={s i |s i =G(T,E),A 1:i} (8)

[0086] Among them, s i =G(T,E) represents task node t i The task sequence that has completed the uninstallation decision before, A 1:i It represents the decision action of offloading the previous i tasks that have completed the decision.

[0087] S3.2: Considering that the offloading destination of the task is limited to the local or edge server, the action space can be defined accordingly as follows:

[0088] ai ={0,1} (9)

[0089] Among them, "0" means that the task is calculated locally, and "1" means that the task is offloaded to the edge server for calculation.

[0090] S3.3: The optimization goal of this paper is to minimize T total To achieve this goal, we define the reward function as the estimated negative delay increment after making the offloading decision for the task. Specifically, when the task node t i When taking action, increments are defined as:

[0091]

[0092] Among them, T i total Represents considering the task node t i After the offloading decision, from the first task to task node t i The total delay of It represents the total delay before making the offloading decision. In this way, the impact of each offloading decision on the overall task execution time can be evaluated, so as to gradually optimize the delay performance in the entire decision-making process.

[0093] Therefore, in order to maximize user satisfaction, this paper defines the reward function as follows:

[0094]

[0095] S4: Using the Markov decision process model mentioned in S3, the PPO algorithm is used to determine the task offloading decision and implement it, such as Figure 4 The specific steps are as follows:

[0096] S4.1: A recurrent neural network (RNN) is used in the network to construct encoder and decoder units. The network architecture is shown in the figure below: Figure 4 The encoder unit is responsible for receiving the task embedding sequence [t1, t2, ..., t n ] as input, and further transforms these sequences to extract feature descriptions corresponding to directed acyclic graphs. At the same time, the decoder initializes its hidden state based on the feature output provided by the encoder and begins to gradually process historical unloading decision information. Through this process, the decoder continuously generates estimates of the strategy and value function, and then gradually derives the next action until the final action sequence [a1, a2, ..., a n ];

[0097] S4.2: Initialize the policy network π θ and value network V φ, where θ and φ represent the parameters of their respective networks;

[0098] S4.3: Adopt current strategy π θ Perform several actions in the environment and collect a series of data trajectories {s t ,a t ,r t ,s t+1 Each trajectory contains the state, the action taken, the reward value obtained, and the next state, which is used to estimate the performance of the strategy and perform subsequent strategy updates;

[0099] S4.4: For each time step t, the algorithm computes an advantage estimate to assess the additional value of taking a particular action relative to the average reward, as follows:

[0100] A t =r t +γV φ (s t+1 )-V φ (s t ) (12)

[0101] S4.5: Update the policy network parameters θ to optimize the objective function L(θ) as follows:

[0102]

[0103] Among them, ρ t (θ) represents the probability ratio of the new and old strategies. This objective function aims to prevent excessive strategy updates and ensure the stability of the learning process by clipping the changes in the probability ratio or adding a penalty term;

[0104] S4.6: Update the value network V φ to more accurately estimate the value of the state;

[0105] S4.7: The above iterative process is repeated to continuously collect new data trajectories, recalculate advantage estimates, and update policy and value network parameters until the maximum number of iterations is reached.

[0106] Example 2

[0107] This embodiment relates to an unmanned driving method in a vehicle network using a fine-grained task offloading method based on directed acyclic graph dependency of the present invention, the method comprising:

[0108] S1: Considering the dependencies between DAG task nodes, the node offloading decision must follow the topological order of DAG. The task diagram is as follows: Figure 2As shown. This strategy can ensure that all decisions of the predecessor nodes are completed before the successor nodes. The service level agreement (SLA) calculation rule adopted is based on the execution cost: the higher the execution cost, the higher the service level of the task node. The specific steps are as follows:

[0109] S1.1: Calculate the execution cost of local computing processing: task node t i The execution cost of local computation processing is expressed as:

[0110]

[0111] in, Represents task node t i The time cost of local calculation, Represents task node t i The fixed transfer cost calculated locally, ο(i) represents the task node t i All successor nodes;

[0112] S1.2: Calculate the execution cost of remote offloading processing: Task node t i The execution cost of remote offloading is expressed as:

[0113]

[0114] in, Represents task node t i The time cost of computing on the remote server, Represents task node t i Fixed delivery cost for execution on remote servers.

[0115] S1.3: Calculate the SLA index: Based on the execution cost calculated in this article, the SLA index of each task node can be obtained, which can be expressed as:

[0116]

[0117] Among them, v i ∈v exit Represents a collection of exit tasks. Indicates the final execution cost of the task node.

[0118] S1.4: By arranging all task nodes in a directed acyclic graph (DAG) in descending order of SLA priority, a topological sorting sequence index of the DAG is obtained, where index[1] represents the task node with the highest priority, and index[|V|] represents the task node with the lowest priority.

[0119] S2: In computation offloading, the computation task can be offloaded to the edge server or executed locally in the vehicle. i To be uninstalled, execute i There are three steps. First, the vehicle transmits t i The edge server then runs the received task. Finally, i The running results are returned to the vehicle. Therefore, task offloading can be divided into two cases, namely local execution and offload execution. The system architecture diagram is as follows: Figure 3 The specific steps are as follows:

[0120] S2.1: Calculate the task t executed locally i The delay caused C i Represents task t i The CPU cycles required, f local Indicates the calculation rate of the vehicle's own CPU;

[0121] S2.2: Transmission delay caused when computing tasks are offloaded to edge servers via uplink And the delay caused by calculation data i represents the size of the task data generated by vehicle i, Indicates the uplink transmission rate, C i Represents task t i The CPU cycles required, f server Indicates the CPU computing rate of the server.

[0122] S2.5: Determine the task offloading optimization problem based on S2.1 and S2.2. The goal of the optimization problem is to minimize the total delay, that is, to maximize the satisfaction.

[0123] The constraints are as follows:

[0124] (3) If the task node t i If it is decided to be executed locally, there is no edge server to offload the computation and transmission, that is, the execution completion time ET i up =0, ET i server =0;

[0125] (4) If the task node t i If it is decided to be executed remotely, there is no local computation completion time, i.e., ET i local =0.

[0126] The available time of a resource depends on the completion time of the previous task node on the resource. i The previous task node did not use this resource, so the completion time on this resource is set to 0. i Offload to the MEC host, only when all its parent tasks are completed and the uplink is available, the task node t i Therefore, the uplink transmission completion time is expressed as:

[0127]

[0128] When the task node is transmitted to the edge server, it is processed by the edge server. i When the edge server is processing, the CPU of the edge server must be available and the task node t i The data has been uploaded successfully, and the task node t i All the previous task nodes of have completed the computation execution. Therefore, task node t i The processing completion time on the edge server is expressed as:

[0129]

[0130] If the task node t i When scheduling locally, the start time of a task depends on the completion time of its parent task and the local available time, which means that the vehicle’s local CPU must be available and all previous task nodes have been calculated and executed. Therefore, the task node t i The processing completion time on the local is expressed as:

[0131]

[0132] In this paper, the offloading decision sequence of directed acyclic graph (DAG) is represented as A = {a1, a2, ..., a n}, where a i represents the offloading decision of the i-th task. i The value of is based on the scheduling plan, that is, a i =0 indicates task node t i Calculated locally in the vehicle, a i =1 indicates task node t i The calculation is performed on the remote MEC server. The overall goal of the optimization problem is to find an effective offloading plan so that the directed acyclic graph (DAG) obtains the minimum total delay, which can be expressed as:

[0133]

[0134] S3: Use the optimization problem described in S2 to establish a Markov decision process model and optimize the decision process. The specific steps are as follows:

[0135] S3.1: In the scheduling task t i When ti is set, the latency of the task depends on the number of CPU cycles required by the task node, the data size, the DAG topology, the wireless transmission rate, and the state of the MEC resources. According to the constraints on the offloading decision in the problem modeling, the state of the MEC resources is related to the offloading decision arranged before the task ti. Therefore, the state space can be defined as follows:

[0136] S={s i |s i =G(T,E),A 1:i} (twenty one)

[0137] Among them, s i =G(T,E) represents task node t i The task sequence that has completed the uninstallation decision before, A 1:i It represents the decision action of offloading the previous i tasks that have completed the decision.

[0138] S3.2: Considering that the offloading destination of the task is limited to the local or edge server, the action space can be defined accordingly as follows:

[0139] a i ={0,1} (22)

[0140] Among them, "0" means that the task is calculated locally, and "1" means that the task is offloaded to the edge server for calculation.

[0141] S3.3: The optimization goal of this paper is to minimize T total To achieve this goal, we define the reward function as the estimated negative delay increment after making the offloading decision for the task. Specifically, when the task node t i When taking action, increments are defined as:

[0142]

[0143] Among them, T i total Represents considering the task node t i After the offloading decision, from the first task to task node t i The total delay of It represents the total delay before making the offloading decision. In this way, the impact of each offloading decision on the overall task execution time can be evaluated, so as to gradually optimize the delay performance in the entire decision-making process.

[0144] Therefore, in order to maximize user satisfaction, this paper defines the reward function as follows:

[0145]

[0146] S4: Using the Markov decision process model mentioned in S3, the PPO algorithm is used to determine the task offloading decision and implement it, such as Figure 4 The specific steps are as follows:

[0147] S4.1: A recurrent neural network (RNN) is used in the network to construct encoder and decoder units. The network architecture is shown in the figure below: Figure 4 The encoder unit is responsible for receiving the task embedding sequence [t1, t2, ..., t n ] as input, and further transforms these sequences to extract feature descriptions corresponding to directed acyclic graphs. At the same time, the decoder initializes its hidden state based on the feature output provided by the encoder and begins to gradually process historical unloading decision information. Through this process, the decoder continuously generates estimates of the strategy and value function, and then gradually derives the next action until the final action sequence [a1, a2, ..., a n ];

[0148] S4.2: Initialize the policy network π θ and value network V φ , where θ and φ represent the parameters of their respective networks;

[0149] S4.3: Adopt current strategy π θ Perform several actions in the environment and collect a series of data trajectories {s t ,a t ,r t ,s t+1 Each trajectory contains the state, the action taken, the reward value obtained, and the next state, which is used to estimate the performance of the strategy and perform subsequent strategy updates;

[0150] S4.4: For each time step t, the algorithm computes an advantage estimate to assess the additional value of taking a particular action relative to the average reward, as follows:

[0151] A t =r t +γV φ (s t+1 )-V φ (s t ) (25)

[0152] S4.5: Update the policy network parameters θ to optimize the objective function L(θ) as follows:

[0153]

[0154] Among them, ρ t (θ) represents the probability ratio of the new and old strategies. This objective function aims to prevent excessive strategy updates and ensure the stability of the learning process by clipping the changes in the probability ratio or adding a penalty term;

[0155] S4.6: Update the value network V φ to more accurately estimate the value of the state;

[0156] S4.7: The above iterative process is repeated to continuously collect new data trajectories, recalculate advantage estimates, and update policy and value network parameters until the maximum number of iterations is reached.

[0157] S5: In the unmanned driving scenario, perception tasks, path planning tasks and control tasks are the core of vehicle decision-making and operation. Their real-time requirements and inter-task dependencies directly determine the task offloading strategy, as follows:

[0158] S5.1: Perception tasks require data collection, object detection and classification, environmental perception and other tasks. The execution of such tasks usually requires a large amount of computing resources and cannot be completed by the vehicle itself. It is suitable to be offloaded to the edge server for processing. When such tasks occur, the vehicle will make an offloading decision according to the above offloading method.

[0159] S5.2: Path planning problem The vehicle develops a safe and optimal driving route. It considers multiple factors such as road conditions, traffic rules, obstacles, and dynamic environmental changes. The tasks of the path planning problem include global path planning and local path planning. Global path planning has low real-time requirements and is suitable for offloading to the edge or cloud processing; while local path planning may be more suitable for execution on the on-board computing unit because it needs to respond frequently to environmental changes. When such tasks occur, according to the above offloading method, the vehicle will make corresponding decisions based on the tasks.

[0160] S5.3: The control task is to adjust the vehicle's state in real time based on the results generated by path planning, including speed control, steering control, etc. This type of control task directly affects the safety of the vehicle and must be responded to in a very short time. Due to its extremely high requirements for real-time performance, the control task is usually processed locally in the vehicle first. When such a task occurs, according to the above-mentioned unloading method, the vehicle will make a decision to execute it locally.

[0161] Example 3

[0162] Reference Figure 5This embodiment relates to a fine-grained task offloading device based on directed acyclic graph dependency, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a fine-grained task offloading method based on directed acyclic graph dependency of embodiment 1.

[0163] Example 4

[0164] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, a fine-grained task offloading method based on directed acyclic graph dependency of embodiment 1 is implemented.

[0165] The various embodiments of the present invention are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0166] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A fine-grained task offloading method based on directed acyclic graph dependency, comprising the following steps: S1: Use directed acyclic graphs to model complex applications and formulate task topology mapping rules based on graph dependencies; S2: Modeling the fine-grained task offloading problem with user satisfaction as the optimization objective; S3: Using the optimization problem described in S2 to establish a Markov decision process model, and optimize the decision process; S4: Using the Markov decision process model mentioned in S3, a reinforcement learning algorithm is used to determine the task offloading strategy and implement it.

2. A method for fine-grained task offloading based on directed acyclic graph dependency as claimed in claim 1, characterized in that: The step S1 comprises the following steps: S1.1: Calculate the execution cost of local computing processing: task node t i The execution cost of local computation processing is expressed as in, Represents task node t i The time cost of local calculation, Represents task node t i The fixed transfer cost calculated locally, ο(i) represents the task node t i All successor nodes; S1.2: Calculate the execution cost of remote offloading processing: Task node t i The execution cost of remote offloading is expressed as: in, Represents task node t i The time cost of computing on the remote server, Represents task node t i Fixed delivery cost for execution on remote servers; S1.3: Calculate the SLA index: Based on the calculated execution cost, the SLA index of each task node can be obtained, which can be expressed as: Among them, v i ∈v exit Represents a collection of exit tasks. Indicates the final execution cost of the task node; S1.4: By arranging all task nodes in a directed acyclic graph (DAG) in descending order of SLA priority, a topological sorting sequence index of the DAG is obtained, where index[1] represents the task node with the highest priority, and index[|V|] represents the task node with the lowest priority.

3. A method for fine-grained task offloading based on directed acyclic graph dependency as claimed in claim 1, characterized in that: Step S2 includes the following steps: S2.1: Calculate the task t executed locally i The delay caused C i Represents task t i The CPU cycles required, f local Indicates the calculation rate of the vehicle's own CPU; S2.2: Transmission delay caused when computing tasks are offloaded to edge servers via uplink And the delay caused by calculation data i represents the size of the task data generated by vehicle i, Indicates the uplink transmission rate, C i Represents task t i The CPU cycles required, f server Indicates the CPU computing rate of the server; S2.5: Determine the task offloading optimization problem based on S2.1 and S2.

2. The goal of the optimization problem is to minimize the total delay, that is, to maximize the satisfaction; The constraints are as follows: (1) If the task node t i is decided to be executed locally, then there is no edge server to offload the computation and transmission execution completion time, i.e. (2) If the task node t i is decided to be executed remotely, then there is no local computation completion time, that is, The available time of a resource depends on the completion time of the previous task node on the resource; if the task node t i The previous task node did not use this resource, so the completion time on this resource is set to 0. i Offload to the MEC host, only when all its parent tasks are completed and the uplink is available, the task node t i Before it can start sending its data; Therefore, the uplink transmission completion time is expressed as: When the task node is transmitted to the edge server, it is processed by the edge server; the task node t i When the edge server is processing, the CPU of the edge server must be available and the task node t i The data has been uploaded successfully, and the task node t i All the previous task nodes of have completed the computation execution; therefore, task node t i The processing completion time on the edge server is expressed as: If the task node t i In local scheduling, the start time of a task depends on the completion time of its parent task and the local available time, that is, the vehicle’s local CPU must be available and all previous task nodes have been calculated and executed; therefore, the task node t i The processing completion time on the local is expressed as: The offloading decision sequence of the directed acyclic graph (DAG) is represented as A = {a1, a2, ..., a n }, where a i represents the offloading decision of the i-th task; a i The value of is based on the scheduling plan, that is, a i =0 indicates task node t i Calculated locally in the vehicle, a i =1 indicates task node t i The calculation is performed on the remote MEC server; the overall goal of the optimization problem is to find an effective offloading plan so that the directed acyclic graph (DAG) obtains the minimum total delay, which is expressed as:

4. A method for fine-grained task offloading based on directed acyclic graph dependency as claimed in claim 1, characterized in that: Step S3 includes the following steps: S3.1: In the scheduling task t i When ti is set, the task latency depends on the number of CPU cycles required by the task node, the data size, the DAG topology, the wireless transmission rate, and the state of the MEC resources. According to the constraints on the offloading decision in the problem modeling, the state of the MEC resources is related to the offloading decision arranged before the task ti. Therefore, the state space can be defined as follows: S={s i |s i =G(T,E),A 1:i } (8) Among them, s i =G(T,E) represents task node t i The task sequence that has completed the uninstallation decision before, A 1:i Indicates the decision action of offloading the previous i tasks that have completed the decision; S3.2: Considering that the offloading destination of the task is limited to the local or edge server, the action space can be defined accordingly as follows: a i ＝{0,1} (9) Among them, "0" means that the task is calculated locally, and "1" means that the task is offloaded to the edge server for calculation; S3.3: The optimization goal is to minimize T total ; To achieve this goal, the reward function is defined as the estimated negative delay increment after the task is offloaded; specifically, when the task node t i When taking action, the increment is defined as in, Represents considering the task node t i After the offloading decision, from the first task to task node t i The total delay of It represents the total delay before making the offloading decision. In this way, the impact of each offloading decision on the overall task execution time can be evaluated, so as to gradually optimize the delay performance in the entire decision-making process. Therefore, in order to maximize user satisfaction, the reward function is defined as follows:

5. A method for fine-grained task offloading based on directed acyclic graph dependency as claimed in claim 1, characterized in that: Step S4 comprises the following steps: S4.1: A recurrent neural network (RNN) is used in the network to construct encoder and decoder units; the encoder unit is responsible for receiving the task embedding sequence [t1, t2, ..., t n ] as input, and further transform these sequences to extract the feature description corresponding to the directed acyclic graph; at the same time, the decoder initializes its hidden state based on the feature output provided by the encoder, and begins to gradually process the historical unloading decision information; through this process, the decoder continuously generates estimates of the strategy and value function, and then gradually derives the next action until the final action sequence [a1, a2, ..., a n ]; S4.2: Initialize the policy network π θ and value network V φ , where θ and φ represent the parameters of their respective networks; S4.3: Adopt current strategy π θ Perform several actions in the environment and collect a series of data trajectories {s t ,a t ,r t ,s t+1 }; Each trajectory contains the state, the action taken, the reward value obtained, and the next state, which is used to estimate the performance of the strategy and perform subsequent strategy updates; S4.4: For each time step t, the algorithm computes an advantage estimate to assess the additional value of taking a particular action relative to the average reward, as follows: From t =r t +γV φ (s t+1 )-V φ (s t ) (12) S4.5: Update the policy network parameters θ to optimize the objective function L(θ) as follows: Among them, ρ t (θ) represents the probability ratio of the new and old strategies; this objective function aims to prevent excessive strategy updates and ensure the stability of the learning process by clipping the changes in the probability ratio or adding a penalty term; S4.6: Update the value network V φ to more accurately estimate the value of the state; S4.7: The above iterative process is repeated to continuously collect new data trajectories, recalculate advantage estimates, and update policy and value network parameters until the maximum number of iterations is reached.

6. A fine-grained task offloading device based on directed acyclic graph dependency, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement a fine-grained task offloading method based on directed acyclic graph dependency as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a fine-grained task offloading method based on directed acyclic graph dependency as described in any one of claims 1 to 5 is implemented.

Images