Task unloading decision-making method based on dynamic priority in space-air-ground fusion vehicle-mounted network scene

Abstract

According to the task unloading and calculation scheduling method in the space-air-ground fusion vehicle-mounted network scene, the vehicle dynamic state, the task requirement, the network state and the calculation resources in the space-air-ground fusion vehicle-mounted network scene are comprehensively monitored and optimally distributed, and the overall task processing efficiency and the resource utilization rate of the system are improved. According to the method, a dynamic priority calculation model, a mathematical model of a space-air-ground fusion car networking task unloading scene, a task unloading strategy solving method and an optimized target function are constructed in sequence, and an optimal task unloading strategy is solved by using an improved deep reinforcement learning method.

Description

Technical Field

[0001] The present invention designs a task offloading decision method based on dynamic priority in the air-space-ground-integrated vehicle network scenario, which improves the overall task processing efficiency and resource utilization of the system by comprehensively monitoring and optimizing the vehicle dynamics, task requirements, network status and computing resources in the air-space-ground-integrated vehicle network scenario. Background Art

[0002] The space-air-ground integrated network (SAGIN) is an important direction for the development of the next generation network. With the rapid development of Internet of Vehicles (IoV) technology, SAGIN provides a wider communication coverage and more powerful computing power support for IoV. By integrating low-orbit satellites, drones and ground communication equipment, SAGIN can achieve cross-regional and cross-level data transmission and processing, thereby meeting the diverse service needs in the IoV. The space-air-ground integrated vehicular network (SAGVN) is a network architecture designed for special application scenarios of the IoV. It is further optimized on the basis of SAGIN to meet the needs of on-board equipment for low latency, high reliability and dynamic resource scheduling. By strengthening the coordination of three types of communication nodes in the air, space and ground, SAGVN can achieve seamless coverage and efficient communication in high-speed mobile environments, providing solid network support for future intelligent transportation systems.

[0003] With the support of SAGIN, a series of collaborative computing and resource management solutions have emerged in the field of Internet of Vehicles to solve the efficiency problems of data processing and task scheduling. Among them, task offloading, as a key technology in the Internet of Vehicles, aims to reduce the computing burden of vehicle terminals and improve the response speed of the system. However, existing research has mostly focused on the optimization of a single network level, ignoring the resource sharing and dynamic scheduling opportunities brought about by the integration of multi-level networks in the air, space, and ground. In addition, traditional optimization methods usually find it difficult to effectively reduce system energy consumption and computing costs while maintaining the offloading success rate.

[0004] Although the current task offloading strategies and resource allocation methods can improve system performance to a certain extent, they still face challenges in SAGIN. The main problem is how to reasonably allocate computing and communication resources between multiple nodes in the air, ground and space to optimize the success rate of task offloading and the overall performance of the system. However, the existing resource allocation strategies lack sufficient consideration of task priorities. This paper designs a priority calculation method that integrates multiple objectives and a preemptive scheduling method based on dynamic priorities to improve the system's resource allocation capabilities and task offloading efficiency in a dynamic environment.

[0005] In recent years, domestic and foreign scholars have conducted systematic research on the task offloading decision-making and resource scheduling issues in the air-space-ground-vehicle network scenario, and proposed solutions such as: multi-agent deep reinforcement learning offloading solution based on Markov game, drone joint deployment solution, and collection task optimization strategy solution. The above solutions mostly start from a single dimension such as shortening the system response delay or reducing the energy consumption of mobile terminals, and design their own task offloading strategies.

[0006] However, there is a trade-off between latency and energy consumption. Reducing latency usually requires more computing resources and communication spectrum, which will lead to increased energy consumption; while reducing energy consumption may sacrifice the efficiency of computing resources, thereby extending latency. The present invention proposes a collaborative offloading and resource optimization scheme suitable for SAGVN. Under the premise of ensuring the success rate of task offloading, this paper performs a secondary optimization of the system's energy consumption and computing cost, focusing on solving the problem of dynamic coordination of resource utilization in complex scenarios.

[0007] The shortcomings of existing technologies are: 1. Most of the current research focuses on the optimization of a single network level, ignoring the resource sharing and dynamic scheduling opportunities brought about by the integration of multi-level networks in the air, space and ground; 2. Traditional optimization methods often focus on the optimization of a single goal such as latency or energy consumption, and lack comprehensive coordination of multiple goals. In the actual application of air-space-ground-vehicle networks, goals such as latency, energy consumption, task offloading success rate and resource allocation usually need to be considered at the same time, and there are certain contradictions and trade-offs between these goals; 3. Existing resource scheduling strategies usually lack the ability to effectively adapt to dynamic environments, especially when the task urgency and network status change, and cannot make timely adjustments. Summary of the invention

[0008] Purpose of the invention: A task offloading and computing scheduling method is designed for the SAGVN scenario, aiming to optimize resource allocation and task scheduling, and improve the overall task processing efficiency and resource utilization of the system. This method improves system performance from the following two aspects:

[0009] 1. Task offloading decision: This paper proposes a task offloading solution based on SAGVN, which realizes task offloading decision by improving the deep reinforcement learning algorithm. Specifically, by taking advantage of reinforcement learning, the system can optimize the effect of task offloading through intelligent decision-making in a variety of dynamic environments.

[0010] 2. Dynamic priority and preemptive scheduling strategy: In view of the problem that existing resource scheduling strategies are not adaptable enough to dynamic environments, the present invention proposes a preemptive scheduling strategy based on task dynamic priority. In this strategy, the priority of a task is determined by the urgency of the task, resource requirements, and the current network status, and the priority will be dynamically adjusted as the task and network status change. The preemptive scheduling mechanism can dynamically adjust the order of task execution according to the task priority when system resources are tight, ensuring that high-priority tasks can obtain the necessary resources in a timely manner and be processed first.

[0011] The task offloading and computing scheduling method in the air-ground-integrated vehicle network scenario includes the following steps:

[0012] (1) Using a unified method for describing and expressing the characteristics of heterogeneous network resources, the communication resources of various wireless communication networks can be aggregated into a vector communication resource library. Cognitive radio technology can be used to perceive the communication resource usage status of various wireless communication networks in real time and describe and express the perceived information.

[0013] (2) Based on the task urgency, task importance, and communication quality, a model is constructed to dynamically prioritize tasks. By constructing a task offloading resource optimization model suitable for SAGVN by maximizing the task offloading success rate and minimizing the computing delay, energy consumption, and cost as the optimization objective function, a task offloading resource optimization model suitable for SAGVN is established through the above modeling;

[0014] (3) Use the improved TD3 algorithm based on the multi-head attention mechanism to solve the task offloading resource optimization model and obtain the optimal offloading decision plan;

[0015] (4) In a simulation environment, verify and evaluate the obtained task offloading resource optimization solution, obtain evaluation data, modify relevant parameters such as learning rate and experience buffer pool size, and obtain a dynamic task offloading resource optimization model;

[0016] (5) Solve the task offloading resource optimization model to obtain the final offloading decision and priority sorting scheme. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawing is a diagram of the task offloading solution process of the present invention; DETAILED DESCRIPTION

[0018] Assume that there are M satellites, N drones, K edge servers, and n vehicles in this scenario. The task created by vehicle i can be expressed as where d i represents the storage resources required by the task, c i represents the computational amount of the task, Indicates the maximum tolerable time for the task to complete, imp iIndicates the importance of the task, imp i ∈{0.1, 0.3, 0.5, 0.7, 0.9}, the larger the value, the higher the importance of the task. Vehicle i is represented by in Indicates the maximum storage capacity of the vehicle. represents the maximum computing resource owned by the vehicle; edge server i is represented by in It is expressed as the maximum storage capacity of the edge server. represents the maximum computing resource of the edge server; the drone node is represented by in Indicates the maximum storage capacity of the drone node, represents the maximum computing resource of the drone node; the satellite node is represented by in Indicates the maximum storage capacity of a spatial node, Indicates the maximum computing resources of the satellite node.

[0019] In the step (1), the unified heterogeneous network resource feature description and expression method specifically includes the following steps:

[0020] (1a) The data transmission rate R of the wireless communication link between the low-orbit satellite, the UAV, the edge server, and the user terminal is calculated by Shannon's Theorem, which is expressed as:

[0021]

[0022] Where R s Represents the transmission rate of the space-ground link; R a Represents the transmission rate of the air-ground link; R m Indicates the transmission rate of the ground link; B s , B m , B m Represent the bandwidth of the space-ground link, air-ground link, and ground link respectively; p s 、p a 、p m They represent the signal transmission power of the space-ground link, air-ground link, and ground link respectively; h represents the channel gain from the vehicle terminal to the satellite; PL a and PL m Respectively represent the path loss of air-ground link and ground link; P N Represents the background noise power.

[0023] (1b) The transmission rate of the space-ground link, air-ground link, and ground link can be calculated based on the transmission delay. The specific steps are as follows:

[0024]

[0025] where d i Represents the data size of the task, K∈{m,a,s}.

[0026] (1c) According to the computational size of the offloaded task and the computing power of each server, the computational delay of the task can be calculated as follows:

[0027]

[0028] Where α=0 means the task is calculated locally; α=1 means the task is calculated on the edge server; α=2 means the task is calculated on the drone; α=3 means the task is calculated on the satellite; β i Indicates the proportion of computing resources allocated to task i.

[0029] (1d) The unloading delay of the task is as follows:

[0030]

[0031] in Indicates the computational waiting delay of the task.

[0032] In step (2), a model is constructed based on task urgency, task importance, and communication quality to dynamically prioritize tasks. The present invention proposes a priority preemptive resource scheduling and calculation rule based on a fuzzy logic system. The method calculates the priority of a task through three indicators: task urgency, task importance, and communication quality. The urgency of the task is determined by the maximum delay limit. to calculate (unit: ms); the communication quality is calculated by the transmission delay Calculate (unit: ms), the specific steps are as follows:

[0033] (2a) Construct a membership function for the task urgency, where the urgency is divided into three levels: high, medium, and low. The membership function is as follows:

[0034]

[0035]

[0036] (2b) Construct a membership function for communication quality, where communication quality is divided into three levels: high, medium, and low. The membership function is as follows:

[0037]

[0038] (2c) Construct a membership function for the task urgency, where the communication quality is divided into three levels: high, medium, and low. The membership function is as follows:

[0039]

[0040]

[0041] (2d) Based on the three indicators of task urgency, task importance and communication quality, the present invention designs a set of fuzzy reasoning rules to accurately evaluate the priority level of the task. The specific fuzzy rules are as follows:

[0042]

[0043] (2e) Assume that μ High , η High The membership of task i is high in urgency, medium in importance, and high in communication quality. Then the membership of this fuzzy rule can be calculated as:

[0044]

[0045] (2f) In a fuzzy control system, there are usually multiple rules acting at the same time, and each rule gives a different output fuzzy set according to different input conditions. The process of combining rule output is to merge (or combine) the output results of all rules to obtain an overall fuzzy output. The present invention adopts the maximum value method for combination, that is, for different fuzzy rules with the same output results, we select the value with the largest membership as the membership of the task priority at this level. For example, r High =max(r High，1 , r High，2 )

[0046] (2g) Defuzzification is the process of converting fuzzy output values ​​into precise output values. The ultimate goal of a fuzzy control system is to obtain a specific numerical output, so it is necessary to convert the fuzzified result into an actual control value. The present invention uses the centroid method to defuzzify the rules and uses the defuzzified output result as the priority of the task, as shown below:

[0047]

[0048] (2h) In the same computing server, tasks are sorted from large to small according to their priorities, and tasks with high priorities are calculated and processed first.

[0049] (2i) In step (2), by constructing the objective function of minimizing the offloading delay, maximizing the offloading success rate of the task, minimizing the energy consumption, and minimizing the cost, a task offloading resource optimization model suitable for SAGVN is established through the above modeling. The specific steps are as follows:

[0050]

[0051] The objective function of the joint optimization is: minimize the unloading delay, maximize the unloading success rate of the task, minimize the energy consumption, and minimize the cost. This model deals with the unloading strategy when the time slot is t, so the constraint condition C2 means taking the maximum unloading delay of the task generated at the time slot t as the system delay of this optimization. The constraint C3 means calculating the unloading success rate of n tasks. H(x) is a unit step function, which is expressed as Constraint C4 means the solution method for system energy consumption, SP i represents the computing power of the i-th computing server; constraint C5 means that the computing power of the computing server is between 0 and 200, in W; constraint C6 means that the limited delay of the task is between 2 and 10, in ms; constraint 7 means the proportion of computing resources allocated to the task, in 0 to 1; constraint C8 means the calculation method of the offloading cost; constraint C9 means that the costs of local computing, offloading to edge servers, offloading to drones, and offloading to satellites are 0, 0.5, 0.7, and 0.8, respectively.

[0052] In step (3), the TD3 algorithm improved based on the multi-head attention mechanism is used to solve the task offloading resource optimization model to obtain the optimal offloading decision plan. The specific steps are as follows:

[0053] (3a) The state space of the TD3 algorithm used in the present invention is:

[0054] state={task i , servers}

[0055]

[0056] servers={server1, server2, server M+N+K+1}

[0057] server i ={C i ,distence,load}

[0058] Where distance represents the straight-line distance between the vehicle and server i; load represents the current load of the server, which is calculated as the sum of the computing resources of all tasks unloaded to the server, as shown below:

[0059]

[0060] o iIndicates whether to offload to the server, the value is 0 to 1;

[0061] (3b) The action space of the TD3 algorithm used in the present invention is the probability of offloading to each server, which is as follows:

[0062] action={θ1, θ2,..., θ M+N+K+1}

[0063] where θ i Indicates offloading to the i-th server.

[0064] (3c) The reward function of the TD3 algorithm used in the present invention is:

[0065]

[0066] (3d) The network structure of the Actor network of the TD3 algorithm used in the present invention is:

[0067] For task i and servers correspond to a fully connected layer respectively, and use the Relu activation function, the hidden layer size is 512, and the task i The output result is used as the Q parameter of the multi-head attention mechanism, and the servers output result is used as the K and V parameters of the multi-head attention mechanism. After passing through a multi-head attention layer with 4 heads, the result is input into a fully connected layer to obtain the probability distribution of unloading.

[0068] (3e) The network structure of the Critic network of the TD3 algorithm used in the present invention is a five-layer fully connected neural network, each layer uses a Relu activation function, and the hidden layer size is 512.

[0069] Beneficial effects: The task offloading and computing resource scheduling method in the air-space-ground integrated vehicle network scenario provided by the present invention takes into account the priority issue of server computing from the perspective of reducing the offloading delay and offloading cost, and provides strong support for the development of green communications; the deep reinforcement learning method based on the multi-head attention mechanism of the present invention solves the offloading decision plan, which can more effectively prompt the task offloading success rate and reduce the offloading delay and offloading cost.

Claims

1. A task offloading decision method based on dynamic priority in an air-ground-integrated vehicle network scenario, comprising the following steps: (1) Using a unified method for describing and expressing the characteristics of heterogeneous network resources, the communication resources of various wireless communication networks can be aggregated into a vector communication resource library. Cognitive radio technology can be used to perceive the communication resource usage status of various wireless communication networks in real time and describe and express the perceived information. (2) Based on the task urgency, task importance, and communication quality, a model is constructed to dynamically prioritize tasks. By constructing a task offloading resource optimization model suitable for SAGVN by maximizing the task offloading success rate and minimizing the computing delay, energy consumption, and cost as the optimization objective function, a task offloading resource optimization model suitable for SAGVN is established through the above modeling; (3) Use the improved TD3 algorithm based on the multi-head attention mechanism to solve the task offloading resource optimization model and obtain the optimal offloading decision plan; (4) In a simulation environment, verify and evaluate the obtained task offloading resource optimization solution, obtain evaluation data, modify relevant parameters such as learning rate and experience buffer pool size, and obtain a dynamic task offloading resource optimization model; (5) Solve the task offloading resource optimization model to obtain the final offloading decision and priority sorting scheme.

2. According to the task offloading decision method based on dynamic priority in the air-ground-integrated vehicle network scenario of claim 1, in step (1), the unified heterogeneous network resource feature description and expression method specifically includes the following steps: (1a) The data transmission rate R of the wireless communication link between the low-orbit satellite, the UAV, the edge server, and the user terminal is calculated by Shannon's Theorem, which is expressed as: Where R s Represents the transmission rate of the space-ground link; R a Represents the transmission rate of the air-ground link; R m Indicates the transmission rate of the ground link; B s , B m , B m Respectively represent the bandwidth of the space-ground link, air-ground link, and ground link; p s 、p a 、p m They represent the signal transmission power of the space-ground link, air-ground link, and ground link respectively; h represents the channel gain from the vehicle terminal to the satellite; PL a and PL m Respectively represent the path loss of air-ground link and ground link; P N Represents the background noise power. (1b) The transmission rate of the space-ground link, air-ground link, and ground link can be calculated based on the transmission delay. The specific steps are as follows: where d i Represents the data size of the task, K∈{m,a,s}. (1c) According to the computational size of the offloaded task and the computing power of each server, the computational delay of the task can be calculated as follows: Where α=0 means the task is calculated locally; α=1 means the task is calculated on the edge server; α=2 means the task is calculated on the drone; α=3 means the task is calculated on the satellite; β i Indicates the proportion of computing resources allocated to task i. (1d) The unloading delay of the task is as follows: Where T i wait Indicates the computational waiting delay of the task.

3. According to the dynamic priority-based task offloading decision method in the air-space-ground integrated vehicle network scenario described in claim 1, in the step (2), a model is constructed based on the task urgency, task importance, and communication quality to dynamically prioritize the tasks. The present invention proposes a priority preemptive resource scheduling and calculation rule based on a fuzzy logic system. The method calculates the priority of the task through three indicators: task urgency, task importance, and communication quality. The urgency of the task is measured by the maximum limit delay. to calculate (unit: ms); the communication quality is calculated by the transmission delay T i tran Calculate (unit: ms), the specific steps are as follows: (2a) Construct a membership function for the task urgency, where the urgency is divided into three levels: high, medium, and low. The membership function is as follows: (2b) Construct a membership function for communication quality, where communication quality is divided into three levels: high, medium, and low. The membership function is as follows: (2c) Construct a membership function for the task urgency, where the communication quality is divided into three levels: high, medium, and low. The membership function is as follows: (2d) Based on the three indicators of task urgency, task importance and communication quality, the present invention designs a set of fuzzy reasoning rules to accurately evaluate the priority level of the task. The specific fuzzy rules are as follows: (2e) Assume that μ High , η High The membership of task i is high in urgency, medium in importance, and high in communication quality. Then the membership of this fuzzy rule can be calculated as: (2f) In a fuzzy control system, there are usually multiple rules acting at the same time, and each rule gives a different output fuzzy set according to different input conditions. The process of combining rule output is to merge (or combine) the output results of all rules to obtain an overall fuzzy output. The present invention adopts the maximum value method for combination, that is, for different fuzzy rules with the same output results, we select the value with the largest membership as the membership of the task priority at this level. For example, r High =max(r High，1 , r High,2 ). (2g) Defuzzification is the process of converting fuzzy output values ​​into precise output values. The ultimate goal of a fuzzy control system is to obtain a specific numerical output, so it is necessary to convert the fuzzified result into an actual control value. The present invention uses the centroid method to defuzzify the rules and uses the defuzzified output result as the priority of the task, as shown below: (2h) In the same computing server, tasks are sorted from large to small according to their priorities, and tasks with high priorities are calculated and processed first. (2i) In step (2), by constructing the objective function of minimizing the offloading delay, maximizing the offloading success rate of the task, minimizing the energy consumption, and minimizing the cost, a task offloading resource optimization model suitable for SAGVN is established through the above modeling. The specific steps are as follows: OP: min T; max A; min E; min V st C1: i∈{1, 2, ..., n} C2:T=max(T i ) C5:0≤SP i ≤200 C7：0≤β i ≤1 C9:V=v i ∈{0,0.5,0.7,0.8} The objective function of the joint optimization is: minimize the unloading delay, maximize the unloading success rate of the task, minimize the energy consumption, and minimize the cost. This model deals with the unloading strategy when the time slot is t, so the constraint condition C2 means taking the maximum unloading delay of the task generated at the time slot t as the system delay of this optimization. The constraint C3 means calculating the unloading success rate of n tasks. H(x) is a unit step function, which is expressed as Constraint C4 means the solution method for system energy consumption, SP i represents the computing power of the i-th computing server; constraint C5 means that the computing power of the computing server is between 0 and 200, in W; constraint C6 means that the limited delay of the task is between 2 and 10, in ms; constraint 7 means the proportion of computing resources allocated to the task, in 0 to 1; constraint C8 means the calculation method of the offloading cost; constraint C9 means that the costs of local computing, offloading to edge servers, offloading to drones, and offloading to satellites are 0, 0.5, 0.7, and 0.8, respectively.

4. According to the task offloading decision method based on dynamic priority in the air-ground-integrated vehicle network scenario of claim 1, in the step (3), the TD3 algorithm improved based on the multi-head attention mechanism is used to solve the task offloading resource optimization model to obtain the optimal offloading decision plan, and the specific steps are as follows: (3a) The state space of the TD3 algorithm used in the present invention is: state={task i ,servers} servers={server1,server2,server M+N+K+1 } server i ={C i ,distence,load} Where distance represents the straight-line distance between the vehicle and server i; load represents the current load of the server, which is calculated as the sum of the computing resources of all tasks unloaded to the server, as shown below: oi indicates whether to offload to the server, and the value is 0 to 1; (3b) The action space of the TD3 algorithm used in the present invention is the probability of offloading to each server, which is as follows: action={θ1,θ2,...,θ M+N+K+1 } where θ i Indicates offloading to the i-th server. (3c) The reward function of the TD3 algorithm used in the present invention is: (3d) The network structure of the Actor network of the TD3 algorithm used in the present invention is: For task i and servers correspond to a fully connected layer respectively, and use the Relu activation function, the hidden layer size is 512, and the task i The output result is used as the Q parameter of the multi-head attention mechanism, and the servers output result is used as the K and V parameters of the multi-head attention mechanism. After a multi-head attention layer with 4 heads, the result is input into a fully connected layer to obtain the probability distribution of unloading. (3e) The network structure of the Critic network of the TD3 algorithm used in the present invention is a five-layer fully connected neural network, each layer uses a Relu activation function, and the hidden layer size is 512.

Images