A method, system, and medium for offloading tasks in collaboration with a vehicle edge

Abstract

The application discloses a kind of vehicle side cooperative task unloading method, system and medium, method includes: with the task abstraction and modeling of the vehicle-mounted application of associated subtask, it is split into workflow application;Through list scheduling method, the priority of each subtask of the workflow application is calculated, and is sorted according to priority, generates workflow subtask scheduling sequence and calculates task embedding vector;Using the task scheduling method of sequence-to-sequence deep reinforcement learning, according to sequence, each workflow subtask is carried out binary unloading decision, and the task unloading mode is obtained;Vehicle terminal uploads reinforcement learning strategy network parameter to edge server for training, and the trained network parameter is transmitted back to local for the update of reinforcement learning strategy network.The application realizes quickly and effectively executing task unloading, effectively reduces delay and cost, improves execution efficiency.

Description

Technical Field

[0001] This application relates to the field of vehicle networking technology, and more specifically, to a method, system, and medium for unloading vehicle-side collaborative tasks. Background Technology

[0002] With rapid economic development and advancements in Internet of Things (IoT) technology, intelligent vehicles have become an indispensable mode of transportation in people's daily lives. Intelligent vehicle applications and services, including image-assisted navigation, intelligent vehicle control, augmented reality (AR), online games, and social media applications, are computationally intensive and latency-sensitive, requiring significant computing and storage resources, which puts considerable pressure on vehicle computing capabilities.

[0003] To address the shortage of computing resources in the Internet of Vehicles (IoV), besides developing the manufacturing industry and improving the computing resource allocation of vehicles themselves, offloading computing tasks is considered an effective solution. This involves transferring the vehicle's workload to other devices with richer computing resources. Mobile Edge Computing (MEC) is one such solution. By moving services from the "cloud" to the access network closer to the user, it localizes applications, services, and content in the edge network. Based on the motivation and foundation of edge computing, Vehicular Edge Computing (VEC) has become a promising technology for supporting Intelligent Traffic System (ITS) services, smart city applications, and urban computing in the IoV scenario.

[0004] Furthermore, the internal functional modules of applications are becoming increasingly complex, such as various machine learning-based applications, path planning, and augmented reality tasks. These mobile applications, characterized by high complexity and logically divisible internal modules into sets of related tasks, can be abstracted as heterogeneous workflows. By implementing a computational task offloading plan, modular subtasks can be distributed across different devices for parallel execution, improving the efficiency of mobile devices. Heterogeneous workflows have complex internal structures, with strict data or control dependencies between subtasks. These subtasks can only begin after all preceding tasks are completed. Offloading decisions must consider these dependencies; otherwise, subtasks may acquire computational resources while their preceding tasks are not yet executed, forcing them to wait idly. Therefore, implementing offloading decisions for complex and structurally diverse heterogeneous workflows in a vehicle-side collaborative work environment, while ensuring latency constraints and reducing vehicle energy consumption, remains a significant challenge.

[0005] Effective technical solutions are urgently needed to address the above problems. Summary of the Invention

[0006] The purpose of this application is to provide a vehicle-side collaborative task unloading method, system, and medium. By using list scheduling and deep reinforcement learning to make decisions on the task unloading method, the task unloading can be executed quickly and effectively, reducing latency and cost, and improving execution efficiency.

[0007] Firstly, this application provides a method for unloading vehicle-side collaborative tasks, the method comprising:

[0008] Task abstraction and modeling are performed on in-vehicle applications with related sub-tasks, and they are broken down into workflow applications;

[0009] The priority of each subtask in the workflow application is calculated using a list scheduling method, and the subtasks are sorted according to their priority to generate a workflow subtask scheduling sequence and calculate the task embedding vector.

[0010] A task scheduling method using sequence-to-sequence deep reinforcement learning is employed to perform binary offloading decisions on each workflow subtask in sequence, resulting in a task offloading method. The task offloading method includes either offloading the task to a local machine for execution or offloading the task to an edge server for execution.

[0011] The vehicle-mounted terminal uploads the reinforcement learning policy network parameters to the edge server for training, and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network.

[0012] In this solution, in-vehicle applications with related subtasks are abstracted and modeled, and then broken down into workflow applications, specifically:

[0013] Identify sub-module tasks with independent functions in automotive applications and estimate the input data volume for each sub-module task. Output data volume Total CPU cycles required for execution X i ;

[0014] Each of the aforementioned sub-module tasks is modeled as a directed acyclic graph, represented as follows: in Given the set of vertices in a directed acyclic graph, each vertex... Corresponding to logically divisible sub-module tasks in the application, This represents the edge set in the WDG, where each bidirectional edge... For two subtasks t i and t j Data and control dependencies between them, edge weights This represents the amount of data that the execution result of the predecessor task is passed to the successor task. τ is the maximum allowable latency for this workflow application, in milliseconds.

[0015] In this solution, the step of calculating the priority of each subtask of the workflow application using a list scheduling method, sorting them according to priority, generating a workflow subtask scheduling sequence, and calculating the task embedding vector specifically includes:

[0016] For the exit node φ in the directed acyclic graph i Define the priority of the subtask as rank(φ) i ) = T i,e ,in This indicates the processing time for uploading the subtask to the edge server for execution and then sending the result back to the vehicle terminal.

[0017] From the exit node of the directed acyclic graph, recursively calculate the priority of other subtasks upwards.

[0018]

[0019] In the formula, succ(φ) i ) represents the subtask φ i For subsequent tasks, calculate the priority of each task node, sort them from high to low, and obtain a scheduling sequence that meets the workflow dependency constraints.

[0020] The scheduling sequence is encoded by embedding the data size and estimated overhead characteristics of each task node.

[0021] In this scheme, task node φ i Embedded ψ i The features include: the subtask's own ID i, and the vector of its direct predecessor nodes. Direct successor node vector Estimated latency for task execution locally and energy consumption The estimated latency for offloading the edge server includes: upload latency. Execution delay and latency of returning results Execution energy consumption

[0022] In this scheme, a sequence-to-sequence deep reinforcement learning task scheduling method is used to perform binary offloading decisions on each workflow subtask in sequence to obtain the task offloading method, specifically including:

[0023] A seq2seq network is used as the main body of the policy network and value network in the deep reinforcement learning model. The embedding sequences of each task are used as the input of the seq2seq network encoder, which calculates the hidden state h of the current step. i =f e (ψ i ,h i-1 ), where f e For the encoder network, the hidden states at all time steps are eventually transformed into context vectors c;

[0024] The decoder network parameters of the seq2seq network are f. d The hidden layer transform of the computation decoder is h′ j =f d (a j-1 ,c j ,h′ i-1 ), where a j-1 The output of the decoder at the previous time step is denoted as subtask φ. j-1 The binary scheduling action;

[0025] After obtaining the decoder's hidden state, the output layer and softmax are used to compute the output scheduling action a at time step j. j The conditional probability distribution P(a) j |a1,a2,…,a j-1 c), the unloading decision can be obtained using conditional probability.

[0026] In this solution, the vehicle-mounted terminal uploads the reinforcement learning policy network parameters to the edge server for training, and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network. Specifically, this includes:

[0027] During off-peak hours for computational tasks, vehicle terminals upload the network parameters of sequence-to-sequence models to edge servers with sufficient computing resources for training via road units.

[0028] The edge server uses the collected vehicle application workflow scheduling trajectory to calculate the reinforcement learning advantage, reward value and other values, and uses the near-end policy optimization method to update the network parameters, thereby optimizing the workflow application execution time and vehicle energy consumption.

[0029] The edge server sends the trained network parameters back to the vehicle terminal, so that the updated network parameters can be used for inference when the in-vehicle application task is unloaded.

[0030] A second aspect of this application also provides a vehicle-side collaborative task unloading system, the system comprising: a memory and a processor, wherein the memory includes a program for a vehicle-side collaborative task unloading method, and when the program for the vehicle-side collaborative task unloading method is executed by the processor, the following steps are implemented:

[0031] Task abstraction and modeling are performed on in-vehicle applications with related sub-tasks, and they are broken down into workflow applications;

[0032] The priority of each subtask in the workflow application is calculated using a list scheduling method, and the subtasks are sorted according to their priority to generate a workflow subtask scheduling sequence and calculate the task embedding vector.

[0033] A task scheduling method using sequence-to-sequence deep reinforcement learning is employed to perform binary offloading decisions on each workflow subtask in sequence, resulting in a task offloading method. The task offloading method includes either offloading the task to a local machine for execution or offloading the task to an edge server for execution.

[0034] The vehicle-mounted terminal uploads the reinforcement learning policy network parameters to the edge server for training, and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network.

[0035] In this solution, in-vehicle applications with related subtasks are abstracted and modeled, and then broken down into workflow applications, specifically:

[0036] Identify sub-module tasks with independent functions in automotive applications and estimate the input data volume for each sub-module task. Output data volume Total CPU cycles required for execution X i ;

[0037] Each of the aforementioned sub-module tasks is modeled as a directed acyclic graph, represented as follows: in Given the set of vertices in a directed acyclic graph, each vertex... Corresponding to logically divisible sub-module tasks in the application, This represents the edge set in the WDG, where each bidirectional edge... For two subtasks t i and t j Data and control dependencies between them, edge weights This represents the amount of data that the execution result of the predecessor task is passed to the successor task. τ is the maximum allowable latency for this workflow application, in milliseconds.

[0038] In this solution, the step of calculating the priority of each subtask of the workflow application using a list scheduling method, sorting them according to priority, generating a workflow subtask scheduling sequence, and calculating the task embedding vector specifically includes:

[0039] For the exit node φ in the directed acyclic graph i Define the priority of the subtask as rank(φ) i ) = T i,e ,in This indicates the processing time for uploading the subtask to the edge server for execution and then sending the result back to the vehicle terminal.

[0040] From the exit node of the directed acyclic graph, recursively calculate the priority of other subtasks upwards. In the formula, succ(φ) i ) represents the subtask φ i For subsequent tasks, calculate the priority of each task node, sort them from high to low, and obtain a scheduling sequence that meets the workflow dependency constraints.

[0041] The scheduling sequence is encoded by embedding the data size and estimated overhead characteristics of each task node.

[0042] Thirdly, this application also provides a computer-readable storage medium including a vehicle-side collaborative task unloading method program, which, when executed by a processor, implements the steps of the vehicle-side collaborative task unloading method.

[0043] As described above, the recycling method, system, and medium for recyclable items provided in this application abstract and model the in-vehicle application with related sub-tasks, breaking it down into workflow applications. Using a list scheduling method, the priority of each sub-task in the workflow application is calculated and sorted according to priority, generating a workflow sub-task scheduling sequence and calculating task embedding vectors. A sequence-to-sequence deep reinforcement learning task scheduling method is used to sequentially perform binary offloading decisions for each workflow sub-task, obtaining the task offloading method. The in-vehicle terminal uploads the reinforcement learning policy network parameters to an edge server for training and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network. This achieves task offloading decision-making through list scheduling and deep reinforcement learning, enabling fast and efficient task offloading, effectively reducing latency and cost, and improving execution efficiency.

[0044] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart of the vehicle-side collaborative task unloading method provided in the embodiments of this application;

[0047] Figure 2 The embodiments of this application provide a task abstraction and modeling method for in-vehicle applications with related sub-tasks, which is then broken down into flowcharts of workflow applications.

[0048] Figure 3 A flowchart for generating a workflow subtask scheduling sequence using a list scheduling method is provided in this embodiment of the application.

[0049] Figure 4 This application provides a flowchart of a task scheduling method utilizing sequence-to-sequence deep reinforcement learning in its embodiments.

[0050] Figure 5 A schematic diagram of the seq2seq model structure provided in this application embodiment;

[0051] Figure 6 This is a schematic diagram of a vehicle-side collaborative task unloading system provided in an embodiment of this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0053] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0054] Please refer to Figure 1 This is a flowchart illustrating a vehicle-side collaborative task unloading method in some embodiments of this application. This method for recycling recyclable items is used in connected vehicles and vehicle networks, including traditional fuel vehicles and new energy vehicles.

[0055] The method for unloading the vehicle-side collaborative task includes the following steps:

[0056] S101 performs task abstraction and modeling on in-vehicle applications with related sub-tasks, and breaks them down into workflow applications;

[0057] S102 calculates the priority of each subtask of the workflow application using a list scheduling method, sorts them according to priority, generates a workflow subtask scheduling sequence, and calculates the task embedding vector.

[0058] S103 utilizes a sequence-to-sequence deep reinforcement learning task scheduling method to perform binary offloading decisions on each workflow subtask in sequence, thereby obtaining a task offloading method; the task offloading method includes: offloading the task to be executed locally or offloading the task to be executed on an edge server.

[0059] The S104 vehicle terminal uploads the reinforcement learning policy network parameters to the edge server for training, and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network.

[0060] It should be noted that this application first abstracts and models the in-vehicle application with related subtasks, breaking them down into workflow applications. Then, using a list scheduling method, the priority of each subtask in the workflow application is calculated and sorted according to priority to generate a workflow subtask scheduling sequence and calculate task embedding vectors. Next, using a sequence-to-sequence deep reinforcement learning task scheduling method, a binary offloading decision is performed on each workflow subtask in sequence to obtain the task offloading method. The task offloading method includes: offloading the task to the local machine for execution or offloading the task to an edge server for execution. Finally, the in-vehicle terminal uploads the reinforcement learning policy network parameters to the edge server for training and transmits the trained network parameters back to the local machine to update the reinforcement learning policy network. This application achieves joint optimization of task offloading processing latency and vehicle energy consumption, enabling fast and efficient task offloading and effectively reducing latency and cost. The task offloading method is not only highly scalable and adaptable to complex vehicle networking environments, but also has high execution efficiency.

[0061] Please refer to Figure 2 This application provides a method for abstracting and modeling in-vehicle applications with related sub-tasks, and breaking them down into workflow application flowcharts.

[0062] According to an embodiment of the present invention, specifically:

[0063] The S201 identifies sub-module tasks with independent functions in automotive applications and estimates the amount of input data for each sub-module task. Output data volume Total CPU cycles required for execution X i ;

[0064] S202 models each of the sub-module tasks as a directed acyclic graph, represented as follows: in Given the set of vertices in a directed acyclic graph, each vertex... Corresponding to logically divisible sub-module tasks in the application, This represents the edge set in the WDG, where each bidirectional edge... For two subtasks t i and t j Data and control dependencies between them, edge weights This represents the amount of data that the execution result of the predecessor task is passed to the successor task. τ is the maximum allowable latency for this workflow application, in milliseconds.

[0065] It should be noted that, in a specific embodiment, it is assumed that there are two relatively independent but dependent subtasks A and B, where the input data size of A is 500KB and the output result is 128B, and the clock cycles required to execute the subtask are 5×10⁻⁶. 9 Then A can be represented as A(500K, 128, 5×10). 9 ).

[0066] In a specific embodiment, it is assumed that there are two relatively independent subtasks A(100K, 128, 5×10) that are dependent on each other. 9 → B(20K, 16, 10×10) 9 In a directed acyclic graph G, where → represents a dependency, then... Assume τ = 10,000 ms.

[0067] Figure 3 A flowchart illustrating the generation of a workflow subtask scheduling sequence using a list scheduling method is provided. According to an embodiment of the present invention, the step of calculating the priority of each subtask of the workflow application using the list scheduling method, sorting them according to priority, generating a workflow subtask scheduling sequence, and calculating a task embedding vector specifically includes:

[0068] S301 For the exit node φ in the directed acyclic graph i Define the priority of the subtask as rank(φ) i ) = T i,e ,in This indicates the processing time in seconds for uploading the subtask to the edge server for execution and sending the result back to the vehicle terminal.

[0069] It should be noted that, in a specific embodiment, the exit node is B(20K, 16, 10×10). 9 The task transmits results wirelessly to and from the edge server at a rate of 4 Mbps, and the edge server's CPU frequency is 10 GHz. Therefore, rank(B) = T. B,e =0.04+1+0.000032≈1.04.

[0070] From the exit node of the self-directed acyclic graph S302, the priority of other subtasks is recursively calculated as follows:

[0071]

[0072] In the formula, succ(φ) i ) represents the subtask φ i The subsequent tasks. After calculating the priority of each task node, sort them from high to low to obtain a scheduling sequence that meets the workflow dependency constraints;

[0073] It should be noted that, in a specific instance, the only predecessor node of B is A(100K, 128, 5×10). 9 ), calculate its priority rank(A) = rank(B) + (0.2 + 0.5 + 0.000256) ≈ 1.74. Next, the node set The nodes in the array are sorted from highest to lowest priority.<A,B> This sequence corresponds to the next task scheduling sequence.

[0074] S303 encodes the scheduling sequence by encoding the data size and estimated overhead characteristics of each task node into the embedding.

[0075] It should be noted that in this scheme, task node φ i Embedded ψ i The features include: the subtask's ID i; and the vector of its direct predecessor nodes. Direct successor node vector Estimated latency for task execution locally and energy consumption The estimated latency for offloading the edge server includes: upload latency. Execution delay and latency of returning results Execution energy consumption

[0076] Figure 4 A flowchart of a task scheduling method utilizing sequence-to-sequence deep reinforcement learning is shown.

[0077] According to an embodiment of the present invention, a task scheduling method using sequence-to-sequence deep reinforcement learning is employed to perform binary offloading decisions on each workflow subtask sequentially, resulting in a task offloading method, specifically including:

[0078] S401 uses a seq2seq network as the core of the policy and value networks in a deep reinforcement learning model. The embedding sequences of each task serve as the input to the seq2seq network encoder, which computes the hidden state h of the current step. i =f e (ψ i ,h i-1 / , where f e For the encoder network, the hidden states at all time steps are eventually transformed into context vectors c;

[0079] It should be noted that, as Figure 5 The image shows a seq2seq model structure in this embodiment, where φ1, φ2, ..., φ nThe task scheduling sequence obtained in the previous step, sorted by priority from high to low, is transformed into a sequence of embedding vectors for related tasks after being represented by the embedding layer. The task embedding vector ψ i It contains the task's feature information: the task's ID i, and the vector of its direct predecessor nodes. Direct successor node vector Estimated latency for task execution locally and energy consumption The estimated latency for offloading the edge server includes: upload latency. Execution delay and latency of returning results Execution energy consumption

[0080] Furthermore, to enhance the seq2seq network's ability to capture dependency information related to tasks, a bidirectional LSTM unit is used in the encoder, and the generated hidden states are weighted and summed using an attention mechanism to obtain the context vector, resulting in:

[0081] The decoder network parameters of the S402seq2seq network are f d The hidden layer transform of the computation decoder is h′ j =f d (a j-1 ,c j ,h′ i-1 ), where a j-1 The output of the decoder at the previous time step is denoted as subtask φ. j-1 The binary scheduling action;

[0082] After obtaining the hidden state of the decoder, S403 uses the output layer and softmax to calculate the output scheduling action a at time step j. j The conditional probability distribution P(a) j |a1,a2,…,a j-1 c), the unloading decision can be obtained using conditional probability.

[0083] It should be noted that in the seq2seq model structure of this embodiment, after processing by the fully connected layer and the Softmax layer, it is assumed that for a certain related task scheduling sequence...<A,B> If the uninstallation decision is <0,1>, then according to the binary uninstallation decision representation method, uninstalling to the local machine is represented as 0, and uninstalling to the edge server is represented as 1. Therefore, these two tasks will be uninstalled to the local machine and the edge server for execution, respectively.

[0084] According to an embodiment of the present invention, the vehicle-mounted terminal uploads the reinforcement learning policy network parameters to an edge server for training, and transmits the trained network parameters back to the local machine to update the reinforcement learning policy network, specifically including:

[0085] During off-peak hours for computational tasks, such as at night, vehicle terminals upload the network parameters of sequence-to-sequence models to edge servers with sufficient computing resources for training via road units.

[0086] The edge server uses the collected vehicle application workflow scheduling trajectory to calculate the reinforcement learning advantage, reward value and other values, and uses the near-end policy optimization method to update the network parameters, thereby optimizing the workflow application execution time and vehicle energy consumption.

[0087] It should be noted that, in order to improve training efficiency, speed, and stability, the seq2seq algorithm is trained based on the Proximal Policy Optimization (PPO) algorithm. The optimization objective of PPO is... Where L CLIP The PPO algorithm adds targets through pruning to avoid significant policy changes, and has... In the formula It is an estimate of the dominance function, and ∈ is a hyperparameter that controls the clipping range. It is the ratio of the current strategy to the old strategy. And... The mean squared error is the expected value of the state and the expected return.

[0088] The training process requires collecting old policy networks π. old The scheduling trajectory generated during execution, including the unloading plan. By executing an unloading plan during training, a sequence of reward values ​​[r1, r2, ..., r] from environmental feedback can be obtained. n To calculate the time difference error (td-error) δ t It is also necessary to collect the output state value sequence of the value network from the trajectory [v] π (s1),v π (s2),…,v π (s n )], and has δ t =r t +γv π (s t+1 )-v π (s t Then, the dominant factor is obtained through calculation. Next, utilizing the state value, reward value, payout value, and advantage estimate of the mobile phone trajectory, a mini-batch sampling method is employed in each iteration, and the Adam algorithm is used to optimize the objective function.

[0089] Finally, the edge server sends the trained network parameters back to the vehicle terminal, so that the updated network parameters can be used for inference when the in-vehicle application task is unloaded.

[0090] like Figure 6 As shown, a second aspect of this application also provides a vehicle-side collaborative task unloading system, which includes: a memory 61 and a processor 62. The memory 61 includes a program for a vehicle-side collaborative task unloading method. When the program for the vehicle-side collaborative task unloading method is executed by the processor 62, it performs the following steps:

[0091] S101 performs task abstraction and modeling on in-vehicle applications with related sub-tasks, and breaks them down into workflow applications;

[0092] S102 calculates the priority of each subtask of the workflow application using a list scheduling method, sorts them according to priority, generates a workflow subtask scheduling sequence, and calculates the task embedding vector.

[0093] S103 utilizes a sequence-to-sequence deep reinforcement learning task scheduling method to perform binary offloading decisions on each workflow subtask in sequence, thereby obtaining a task offloading method; the task offloading method includes: offloading the task to be executed locally or offloading the task to be executed on an edge server.

[0094] The S104 vehicle terminal uploads the reinforcement learning policy network parameters to the edge server for training, and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network.

[0095] It should be noted that this application first abstracts and models the in-vehicle application with related subtasks, breaking them down into workflow applications. Then, using a list scheduling method, the priority of each subtask in the workflow application is calculated and sorted according to priority to generate a workflow subtask scheduling sequence and calculate task embedding vectors. Next, using a sequence-to-sequence deep reinforcement learning task scheduling method, a binary offloading decision is performed on each workflow subtask in sequence to obtain the task offloading method. The task offloading method includes: offloading the task to the local machine for execution or offloading the task to an edge server for execution. Finally, the in-vehicle terminal uploads the reinforcement learning policy network parameters to the edge server for training and transmits the trained network parameters back to the local machine to update the reinforcement learning policy network. This application achieves joint optimization of task offloading processing latency and vehicle energy consumption, enabling fast and efficient task offloading and effectively reducing latency and cost. The task offloading method is not only highly scalable and adaptable to complex vehicle networking environments, but also has high execution efficiency.

[0096] Please refer to Figure 2This application provides a method for abstracting and modeling in-vehicle applications with related sub-tasks, and breaking them down into workflow application flowcharts.

[0097] According to an embodiment of the present invention, specifically:

[0098] The S201 identifies sub-module tasks with independent functions in automotive applications and estimates the amount of input data for each sub-module task. Output data volume Total CPU cycles required for execution X i ;

[0099] S202 models each of the sub-module tasks as a directed acyclic graph, represented as follows: in Given the set of vertices in a directed acyclic graph, each vertex... Corresponding to logically divisible sub-module tasks in the application, This represents the edge set in the WDG, where each bidirectional edge... For two subtasks t i and t j Data and control dependencies between them, edge weights This represents the amount of data passed from the execution result of the predecessor task to the successor task. τ is the maximum allowable latency for this workflow application, in milliseconds.

[0100] It should be noted that, in a specific embodiment, it is assumed that there are two relatively independent but dependent subtasks A and B, where the input data size of A is 500KB and the output result is 128B, and the clock cycles required to execute the subtask are 5×10⁻⁶. 9 Then A can be represented as A(500K, 128, 5×10). 9 ).

[0101] In a specific embodiment, it is assumed that there are two relatively independent subtasks A(100K, 128, 5×10) that are dependent on each other. 9 → B(20K, 16, 10×10) 9 In a directed acyclic graph G, where → represents a dependency, then... Assume τ = 10,000 ms.

[0102] According to an embodiment of the present invention, the step of calculating the priority of each subtask of the workflow application using a list scheduling method, sorting them according to priority, generating a workflow subtask scheduling sequence, and calculating a task embedding vector specifically includes:

[0103] S301 For the exit node φ in the directed acyclic graph iDefine the priority of the subtask as rank(φ) i ) = T i,e ,in This indicates the processing time in seconds for uploading the subtask to the edge server for execution and sending the result back to the vehicle terminal.

[0104] It should be noted that, in a specific embodiment, the exit node is B(20K, 16, 10×10). 9 The task transmits results wirelessly to and from the edge server at a rate of 4 Mbps, and the edge server's CPU frequency is 10 GHz. Therefore, rank(B) = T. B,e =0.04+1+0.000032≈1.04.

[0105] From the exit node of the self-directed acyclic graph S302, the priority of other subtasks is recursively calculated as follows: In the formula, succ(φ) i ) represents the subtask φ i The subsequent tasks. After calculating the priority of each task node, sort them from high to low to obtain a scheduling sequence that meets the workflow dependency constraints;

[0106] It should be noted that, in a specific instance, the only predecessor node of B is A(100K, 128, 5×10). 9 ), calculate its priority rank(A) = rank(B) + (0.2 + 0.5 + 0.000256) ≈ 1.74. Next, the node set The nodes in the array are sorted from highest to lowest priority.<A,B> This sequence corresponds to the subsequent task scheduling sequence. S303 encodes the scheduling sequence by encoding the data size and estimated cost characteristics of each task node into the embedding.

[0107] It should be noted that in this scheme, task node φ i Embedded ψ i The features include: the subtask's ID i; and the vector of its direct predecessor nodes. Direct successor node vector Estimated latency for task execution locally and energy consumption The estimated latency for offloading the edge server includes: upload latency. Execution delay and latency of returning results Execution energy consumption

[0108] According to an embodiment of the present invention, a task scheduling method using sequence-to-sequence deep reinforcement learning is employed to perform binary offloading decisions on each workflow subtask sequentially, resulting in a task offloading method, specifically including:

[0109] S401 uses a seq2seq network as the core of the policy and value networks in a deep reinforcement learning model. The embedding sequences of each task serve as the input to the seq2seq network encoder, which computes the hidden state h of the current step. i =f e (ψ i ,h i-1 ), where f e For the encoder network, the hidden states at all time steps are eventually transformed into context vectors c;

[0110] It should be noted that, as Figure 5 The image shows a seq2seq model structure in this embodiment, where φ1, φ2, ..., φ n The task scheduling sequence obtained in the previous step, sorted by priority from high to low, is transformed into a sequence of embedding vectors for related tasks after being represented by the embedding layer. The task embedding vector ψ i It contains the task's feature information: the task's ID i, and the vector of its direct predecessor nodes. Direct successor node vector Estimated latency for task execution locally and energy consumption The estimated latency for offloading the edge server includes: upload latency. Execution delay and latency of returning results Execution energy consumption

[0111] Furthermore, to enhance the seq2seq network's ability to capture dependency information related to tasks, a bidirectional LSTM unit is used in the encoder, and the generated hidden states are weighted and summed using an attention mechanism to obtain the context vector, resulting in:

[0112] The decoder network parameters of the S402seq2seq network are f d The hidden layer transform of the computation decoder is h′ j =f d (a j-1 ,c j ,h′ i-1 ), where a j-1 The output of the decoder at the previous time step is denoted as subtask φ. j-1 The binary scheduling action;

[0113] After obtaining the hidden state of the decoder, S403 uses the output layer and softmax to calculate the output scheduling action a at time step j. j The conditional probability distribution P(a) j |a1,a2,…,a j-1 c), the unloading decision can be obtained using conditional probability.

[0114] It should be noted that in the seq2seq model structure of this embodiment, after processing by the fully connected layer and the Softmax layer, it is assumed that for a certain related task scheduling sequence...<A,B> If the uninstallation decision is <0,1>, then according to the binary uninstallation decision representation method, uninstalling to the local machine is represented as 0, and uninstalling to the edge server is represented as 1. Therefore, these two tasks will be uninstalled to the local machine and the edge server for execution, respectively.

[0115] According to an embodiment of the present invention, the vehicle-mounted terminal uploads the reinforcement learning policy network parameters to an edge server for training, and transmits the trained network parameters back to the local machine to update the reinforcement learning policy network, specifically including:

[0116] During off-peak hours for computational tasks, such as at night, vehicle terminals upload the network parameters of sequence-to-sequence models to edge servers with sufficient computing resources for training via road units.

[0117] The edge server uses the collected vehicle application workflow scheduling trajectory to calculate the reinforcement learning advantage, reward value and other values, and uses the near-end policy optimization method to update the network parameters, thereby optimizing the workflow application execution time and vehicle energy consumption.

[0118] It should be noted that, in order to improve training efficiency, speed, and stability, the seq2seq algorithm is trained based on the Proximal Policy Optimization (PPO) algorithm. The optimization objective of PPO is... Where L CLIP The PPO algorithm adds targets through pruning to avoid significant policy changes, and has... In the formula It is an estimate of the dominance function, and ∈ is a hyperparameter that controls the clipping range. It is the ratio of the current strategy to the old strategy. And... The mean squared error is the expected value of the state and the expected return.

[0119] The training process requires collecting old policy networks π. old The scheduling trajectory generated during execution, including the unloading plan. By executing an unloading plan during training, a sequence of reward values ​​[r1, r2, ..., r] from environmental feedback can be obtained. nTo calculate the time difference error (td-error) δ t It is also necessary to collect the output state value sequence of the value network from the trajectory [v] π (s1),v π (s2),…,v π (s n )], and has δ t =r t +γv π (s t+1 )-v π (s t Then, the dominant factor is obtained through calculation. Next, utilizing the state value, reward value, payout value, and advantage estimate of the mobile phone trajectory, a mini-batch sampling method is employed in each iteration, and the Adam algorithm is used to optimize the objective function.

[0120] Finally, the edge server sends the trained network parameters back to the vehicle terminal, so that the updated network parameters can be used for inference when the in-vehicle application task is unloaded.

[0121] Thirdly, this application also provides a computer-readable storage medium including a vehicle-side collaborative task unloading method program, which, when executed by a processor, implements the steps of the vehicle-side collaborative task unloading method.

[0122] This application provides a vehicle-side collaborative task unloading method, system, and medium that uses list scheduling and deep reinforcement learning to make decisions on the task unloading method, thereby quickly and effectively executing task unloading, effectively reducing latency and cost, and improving execution efficiency.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0124] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0125] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0126] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for unloading a vehicle-side collaborative task, characterized in that, The method includes: Task abstraction and modeling are performed on in-vehicle applications with related sub-tasks, and they are broken down into workflow applications; Using a list-based scheduling method, the priority of each subtask in the workflow application is calculated, and they are sorted according to priority to generate a workflow subtask scheduling sequence and calculate the task embedding vector. Specifically, this includes: For the exit subtask in a directed acyclic graph Define the priority of the exit subtask as ,in This indicates the processing time for uploading the subtask to the edge server for execution and sending the result back to the vehicle terminal. From the exit subtask of the directed acyclic graph, the priority of other subtasks is recursively calculated upwards. In the formula Subtasks The subsequent tasks are then sorted from highest to lowest priority to obtain a scheduling sequence that conforms to workflow dependency constraints. The scheduling sequence is encoded by encoding the data size and estimated overhead feature information of each subtask into an embedding vector; Subtask Embedded vector The features include: the subtask's own ID number. Direct predecessor node vector Direct successor node vector Estimated latency for task execution locally and energy consumption The estimated latency for offloading the edge server includes: upload latency. Execution delay and latency of returning results Energy consumption during execution ; A task scheduling method using sequence-to-sequence deep reinforcement learning is employed to perform binary offloading decisions on each workflow subtask in sequence, resulting in a task offloading method. The task offloading method includes: offloading the task to local execution or offloading the task to an edge server for execution. The vehicle-mounted terminal uploads the reinforcement learning policy network parameters to the edge server for training, and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network. Specifically, this includes: During off-peak hours for computational tasks, the vehicle-mounted terminal uploads the network parameters of the sequence-to-sequence model to an edge server with sufficient computing resources for training via road units; The edge server uses the collected vehicle application workflow scheduling trajectory to calculate the advantage and reward value of reinforcement learning, and uses the near-end policy optimization method to update network parameters, optimize the execution time of workflow applications and vehicle energy consumption. The edge server sends the trained network parameters back to the vehicle terminal, so that the updated network parameters can be used for inference when the vehicle application task is unloaded.

2. The vehicle-side collaborative task unloading method according to claim 1, characterized in that, For in-vehicle applications with related subtasks, task abstraction and modeling are performed, and they are broken down into workflow applications, specifically: Identify subtasks with independent functions in automotive applications and estimate the amount of input data for each subtask. Output data volume Total number of CPU cycles required for execution ; Each of the aforementioned subtasks is modeled as a directed acyclic graph, represented as follows: ,in Given the set of vertices in a directed acyclic graph, each vertex... Corresponding to logically divisible subtasks in the application, This represents the set of edges in the WDG, where each edge... Two sub-tasks and Data and control dependencies between them, edge weights This indicates the amount of data passed from the execution result of the predecessor task to the successor task. The maximum allowable latency for this workflow application is in milliseconds.

3. The vehicle-side collaborative task unloading method according to claim 1, characterized in that, Using a sequence-to-sequence deep reinforcement learning-based task scheduling method, a binary offloading decision is made for each workflow subtask in sequence to obtain the task offloading method, specifically including: A seq2seq network is used as the main body of the policy network and value network in the deep reinforcement learning model. The embedding sequences of each task are used as the input of the seq2seq network encoder, which calculates the hidden state of the current step. ,in For the encoder network, the hidden states at all time steps are eventually transformed into context vectors. ; The decoder network parameters of the seq2seq network are: The hidden layer transformation of the computation decoder is ,in The output of the decoder at the previous time step is represented as a subtask. The binary scheduling action; After obtaining the hidden state of the decoder, the output layer and softmax are used to compute the time step. Time output scheduling action conditional probability distribution The unloading decision can be obtained using conditional probability. .

4. A vehicle-side collaborative task unloading system, characterized in that, The system includes a memory and a processor. The memory includes a program for a vehicle-side collaborative task unloading method. When the program for the vehicle-side collaborative task unloading method is executed by the processor, it performs the following steps: Task abstraction and modeling are performed on in-vehicle applications with related sub-tasks, and they are broken down into workflow applications; Using a list-based scheduling method, the priority of each subtask in the workflow application is calculated, and they are sorted according to priority to generate a workflow subtask scheduling sequence and calculate the task embedding vector. Specifically, this includes: For the exit subtask in a directed acyclic graph Define the priority of the exit subtask as ,in This indicates the processing time for uploading the subtask to the edge server for execution and sending the result back to the vehicle terminal. From the exit subtask of the directed acyclic graph, recursively calculate the priority of other subtasks upwards. In the formula Subtasks The subsequent tasks are then sorted from highest to lowest priority to obtain a scheduling sequence that conforms to workflow dependency constraints. The scheduling sequence is encoded by encoding the data size and estimated overhead feature information of each subtask into an embedding vector; Subtasks Embedded vector The features include: the subtask's own ID number. Direct predecessor node vector Direct successor node vector Estimated latency for task execution locally and energy consumption The estimated latency for offloading the edge server includes: upload latency. Execution delay and latency of returning results Energy consumption during execution ; A task scheduling method using sequence-to-sequence deep reinforcement learning is employed to perform binary offloading decisions on each workflow subtask in sequence, resulting in a task offloading method. The task offloading method includes: offloading the task to local execution or offloading the task to an edge server for execution. The vehicle-mounted terminal uploads the reinforcement learning policy network parameters to the edge server for training, and then transmits the trained network parameters back to the local machine to update the reinforcement learning policy network. Specifically, this includes: During off-peak hours for computational tasks, the vehicle-mounted terminal uploads the network parameters of the sequence-to-sequence model to an edge server with sufficient computing resources for training via road units; The edge server uses the collected vehicle application workflow scheduling trajectory to calculate the advantage and reward value of reinforcement learning, and uses the near-end policy optimization method to update network parameters, optimize the execution time of workflow applications and vehicle energy consumption. The edge server sends the trained network parameters back to the vehicle terminal, so that the updated network parameters can be used for inference when the vehicle application task is unloaded.

5. A vehicle-side collaborative task unloading system according to claim 4, characterized in that, For in-vehicle applications with related subtasks, task abstraction and modeling are performed, and they are broken down into workflow applications, specifically: Identify subtasks with independent functions in automotive applications and estimate the amount of input data for each subtask. Output data volume Total number of CPU cycles required for execution ; Each of the aforementioned subtasks is modeled as a directed acyclic graph, represented as follows: ,in Given the set of vertices in a directed acyclic graph, each vertex... Corresponding to logically divisible subtasks in the application, This represents the set of edges in the WDG, where each edge... Two sub-tasks and Data and control dependencies between them, edge weights This indicates the amount of data passed from the execution result of the predecessor task to the successor task. The maximum allowable latency for this workflow application is in milliseconds.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a vehicle-side collaborative task unloading method program, which, when executed by a processor, implements the steps of the vehicle-side collaborative task unloading method as described in any one of claims 1 to 3.

Images