Optimization method for edge computing task unloading and resource scheduling of Internet of Vehicles

By introducing deep reinforcement learning based on Markov decision processes and a dual-priority scheduling mechanism into the Internet of Vehicles (IoV), the task paths and resource allocation are dynamically adjusted, solving the problems of task latency and low resource utilization in IoV edge computing and realizing system-level intelligent collaborative optimization of resources.

CN120916201AActive Publication Date: 2025-11-07NANJING UNIV OF POSTS & TELECOMM

Patent Information

Application Number
CN202511450203.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-10-11
Publication Date
2025-11-07
Estimated Expiration
2045-10-11

AI Technical Summary

Technical Problem

Existing edge computing task offloading methods for connected vehicles lack real-time adjustment capabilities in dynamic environments, resulting in task processing delays and low resource utilization. Furthermore, existing deep reinforcement learning methods are insufficient in terms of interpretability and adaptability.

Method used

A deep reinforcement learning algorithm based on Markov decision process is adopted, combined with a dual-priority scheduling mechanism, to realize dynamic task switching and real-time resource preemption. The unloading strategy is optimized through the interaction between the agent and the environment, and the task path and resource allocation are dynamically adjusted.

Benefits of technology

Significantly reduces task processing latency, improves resource utilization and task completion rate, adapts to complex and ever-changing vehicle networking environment, and achieves intelligent collaborative optimization of system-level resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120916201A_ABST
    Figure CN120916201A_ABST
Patent Text Reader

Abstract

The invention relates to an internet of vehicles edge computing task unloading and resource scheduling optimization method, which comprises the following steps: constructing an internet of vehicles edge computing system model fusing a task jump mechanism and dual-priority scheduling, the system model comprising a road test unit deployment model, a communication model, a computing model and a task jump model; establishing a joint optimization function aiming at minimizing the total time delay and the total energy consumption of the system; decomposing a joint optimization problem into three sub-problems of unloading decision, computing resource allocation and communication strategy selection; a deep reinforcement learning algorithm is adopted to intelligently sense a road environment state, an optimal communication priority strategy combination is dynamically selected, and a task unloading decision and computing resource allocation are collaboratively optimized; and in each scheduling time slot, the edge server allocates communication bandwidth and computing resources to the vehicle tasks according to the selected strategy, and triggers a task preemption and cross-server jump mechanism, thereby realizing maximization of system throughput and minimization of task processing time delay.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of Internet of Vehicles, and particularly relates to an optimization method for edge computing task offloading and resource scheduling of Internet of Vehicles. BACKGROUND

[0002] The present application considers a very complete simulation environment, and the bandwidth allocation of edge transmission is determined by the task priority. In addition to the offloading scheme, the resource allocation of the edge server is also an optimization target. At the same time, this paper introduces a task jump mechanism with priority, all vehicles can choose to offload tasks to the nearest server or migrate tasks to the edge server in the adjacent domain by the server for calculation, so as to further shorten the total time delay, and a part of the resource allocation decision will also be determined by the task priority. The work to be studied is how to coordinate multiple parameters to optimize the final goal and develop a suitable offloading strategy.

[0003] The classical algorithm optimization method can explain each step in the solving process, but the result may be affected by factors such as initial value and constraint, and attention should be paid to parameter setting, constraint condition and parameter adjustment, and it may not converge or converge slowly, but it usually has the characteristics of good flexibility, high interpretability and the like. Deep reinforcement learning combines the perception prediction ability of deep learning and the decision-making ability of reinforcement learning, uses neural network to fit value and strategy, so that the agent can efficiently process scheduling information in a complex environment, but it has poor interpretability and needs time for modeling and optimization. Therefore, this paper plans to combine traditional scheduling methods and deep reinforcement learning based on Markov decision process, improve prediction ability, accelerate model building and learning, and maximize edge computing scheduling efficiency.

[0004] According to the search, the Chinese invention patent with the publication number CN119996446A discloses an RSU-assisted multi-hop task offloading method based on asynchronous deep reinforcement learning, relates to the technical field of edge offloading and deep reinforcement, accurately evaluates the connection stability between nodes by establishing a vehicle movement model and a communication model; models the task offloading problem as a high-dimensional MDP, and solves the problem by using A3C algorithm; the A3C algorithm trains multiple agents asynchronously, effectively handles the optimization problem in a large-scale action space, and thus realizes a more optimal task offloading strategy in a complex and variable vehicle networking environment; compared with existing task offloading strategies, the method has significant performance improvement in reducing task processing delay and improving computing resource utilization, especially in multi-RSU and high dynamicity scenarios, significantly improves the task completion rate and resource utilization efficiency of the system; in addition, the application also has good scalability and can adapt to more complex application requirements in future vehicle networking.

[0005] The technical comparison between the present application and the "RSU-assisted multi-hop task offloading method based on asynchronous deep reinforcement learning" is as follows: The present application and the "RSU-assisted multi-hop task offloading method based on asynchronous deep reinforcement learning" are both in the field of vehicle networking edge computing, both involve task offloading and resource scheduling, but there are fundamental differences in core architecture and scheduling mechanism, mainly in the following two aspects: 1. The underlying task scheduling mechanism; 2. The final optimization decision.

[0006] In terms of task scheduling mechanism, an architectural upgrade from "multi-hop transmission" to "coordinated preemption and jumping" is achieved. The "RSU-assisted multi-hop task offloading based on asynchronous deep reinforcement learning" adopts "RSU-assisted multi-hop task offloading", which is essentially a static and passive transmission path optimization. Once the task is offloaded, its execution path is relatively fixed, lacking the ability to dynamically adjust according to the real-time state of the system, while the present application proposes a new mechanism of "task preemption + task dynamic jumping", achieving a fundamental breakthrough in architecture. The system can dynamically preempt the computing resources occupied by low-priority tasks according to the urgency of newly arrived high-priority tasks, achieving instantaneous reallocation of resources and greatly improving the system's response capability to emergency events. At the same time, the task can dynamically migrate to the optimal server according to the load state of the neighboring server during execution, breaking through the single-server resource bottleneck and upgrading from "path optimization" to "global resource pool utilization". This mechanism far surpasses the single "multi-hop transmission" of the "RSU-assisted multi-hop task offloading method based on asynchronous deep reinforcement learning", achieving true cross-domain coordination and resource elastic scheduling.

[0007] In terms of optimization decision, a paradigm shift from "single decision" to "dual-priority invisible cooperation" is achieved. The "RSU-assisted multi-hop task offloading method based on asynchronous deep reinforcement learning" uses asynchronous deep reinforcement learning (A3C) to optimize offloading decisions, with the core optimization goal focusing on the selection of transmission paths to minimize processing delay. The present application introduces a "dual-priority invisible cooperation" decision framework, achieving multi-dimensional and adaptive integrated optimization: it innovatively designs a dual-independent priority mechanism that decouples transmission and computation. Transmission priority integrates vehicle speed, task data volume, and urgency to dynamically allocate bandwidth, while computation priority introduces remaining time awareness to dynamically allocate computing power. The cooperation of the two ensures that high-urgency tasks are given priority in both transmission and computation stages. Through deep reinforcement learning, the road scene (such as emergency braking and regular cruising) is intelligently perceived, and priority strategy parameters (lambda coefficient) are dynamically generated and adaptively adjusted, enabling the resource allocation strategy to form "invisible cooperation" with the real-time vehicle networking environment, thereby optimizing delay, energy consumption, and task success rate at the system level. The optimization dimension and intelligence level far exceed the single delay optimization goal of the application.

[0008] The RSU-assisted multi-hop task offloading method based on asynchronous deep reinforcement learning provides a multi-hop transmission scheme based on fixed rules, and the present application proposes an intelligent scheduling system with dynamic preemption, cross-domain jumping, double-priority cooperation and scene adaptation capability, realizing the leap from "transmission optimization" to "system-level resource intelligent cooperation", and the technical distinction is significant.

[0009] Through retrieval, the US invention patent with publication number US12399756B2 provides a method for optimizing task offloading in vehicle networking using deep reinforcement learning, including the following steps: S1: constructing a vehicle-mounted system network; S2: constructing a task model of the application; S3: constructing a task load model: calculating the time delay, energy consumption and incentive compensation of three offloading modes of local calculation, offloading to nearby vehicles and offloading to nearby RSUs according to the vehicle-mounted network system and the task model; S4: determining the priority of the task: first, determine the priority of each sub-task according to the allocation of the predecessor node of the sub-task, and then schedule according to the multi-queue algorithm; S5: finding the optimal offloading strategy using deep reinforcement learning.

[0010] The technical comparison between the present application and the method for optimizing task offloading in vehicle networking using deep reinforcement learning is as follows: Although both the present application and the method for offloading dependent tasks in vehicle networking using deep reinforcement learning use the deep reinforcement learning framework and focus on task priority design, there are essential differences in system completeness and scheduling mechanism, mainly in the following two key aspects: one is the completeness of the system model, and the other is the optimization target and cooperation dimension.

[0011] In terms of the completeness of the system model: the mechanism completeness upgrade from "static offloading" to "dynamic jumping and preemption" is realized. The optimization core of the method for offloading dependent tasks in vehicle networking using deep reinforcement learning is to select the optimal execution location for dependent sub-tasks, and its model is essentially a static and one-time offloading decision. Once the task is offloaded to a specific location, its execution process is fixed at that node and completed, lacking the ability to dynamically adjust according to the system state changes during execution. The present application constructs a complete system model that supports "task dynamic jumping" and "resource real-time preemption", realizing dynamic scheduling at the full life cycle level; the task can dynamically migrate to a server with better resources for continuous execution according to the real-time load state of the neighboring server during execution. At the same time, a real-time preemption mechanism based on double priority is introduced. When a high-priority new task arrives, it can preempt the resources occupied by a low-priority task, ensuring the immediate processing of urgent tasks (such as safety alarms). Compared with the static scheduling in the application that relies on "initial priority sorting", this can dynamically guarantee the service quality of high-priority tasks, which is a substantial enhancement of the priority model.

[0012] In the optimization target and the coordination dimension: the optimization paradigm expansion from "cost weighted sum" to "multi-objective implicit coordination" is realized. The optimization target of the "vehicle networking task offloading method based on deep reinforcement learning" is "the weighted sum of energy consumption and incentive compensation under the time delay constraint", which is essentially a linear weighted optimization of multi-objective into a single objective, and the weight needs to be set in advance, which is difficult to adapt to the complex and variable vehicle networking environment. The present application proposes a "time delay-energy consumption" multi-objective implicit coordination optimization framework. By designing the calculation priority of the remaining time and the multi-dimensional dynamic weight communication priority, the time delay constraint, energy consumption sensitivity, task urgency and other optimization objectives are implicitly embedded into the priority calculation and resource allocation strategy, and the intelligent agent automatically learns the optimal balance point, avoiding the difficulty of subjective weight coefficient setting.

[0013] Compared with the "vehicle networking task offloading method based on deep reinforcement learning", the present application provides an initial offloading strategy for dependent tasks, while the present application proposes a global optimization system integrating dynamic jumping, real-time preemption and multi-objective implicit coordination, realizing the leap from "static offloading decision" to "dynamic resource coordination", and significantly improving the model completeness and optimization dimension. SUMMARY

[0014] The present application aims to overcome the shortcomings and deficiencies of the prior art, and provides an optimization method for vehicle networking edge computing task offloading and resource scheduling, which optimally allocates the additive noise of the base station to the vehicle user and optimizes the position of the movable antenna, realizes the optimization of the base station compensation cost in vehicle networking, and improves the transmission rate of the vehicle user and reduces the communication energy consumption.

[0015] To achieve the above purpose, the technical scheme adopted by the present application is: The optimization method for vehicle networking edge computing task offloading and resource scheduling comprises the following steps: S1: a vehicle networking task offloading and resource scheduling optimization model based on double priority and task dynamic jumping is constructed, and a target function of minimizing the offloading scheduling time delay and total energy consumption is established according to the total cost of mobile vehicle task offloading and scheduling; S2: the joint optimization problem is modeled as a Markov decision process; a deep reinforcement learning algorithm is adopted, the optimization strategy is continuously learned through the interaction between the intelligent agent and the environment, and the optimal joint strategy of each scheduling time slot is directly output, so as to realize the minimization of the total cost of the system; S3: In each scheduling time slot, the edge server allocates communication bandwidth and computing resources for vehicle users according to the optimal communication strategy, task offloading decision and computing resource allocation scheme obtained by solving, triggers task preemption and cross-server hopping mechanism, calculates the total system delay and total energy consumption of the current time slot, and outputs the optimized system performance index.

[0016] As a preferred technical solution of the present application: in step S1, the objective function of minimizing the weighted sum of offloading scheduling delay and total energy consumption includes: The objective function includes two parts, namely delay cost and energy consumption cost, and the optimization problem objective function is as follows: (1); ; Wherein, denotes the sum of the delay generated by offloading and scheduling of vehicle task i at time slot t on edge server m, denotes the sum of the energy consumption generated by offloading and scheduling of vehicle task i at time slot t on edge server m; denotes the weighted sum of delay and energy consumption; M denotes the total set of edge servers, and m denotes a specific edge server in the set M, denotes all vehicle tasks under edge server m, i denotes the task generated by the ith vehicle, denotes the maximum processing delay that the task i can tolerate, denotes the percentage of computing resources allocated by edge server m to vehicle task i, N denotes the total number of vehicle sets within the range of edge server m, denotes the offloading decision of task i at time slot t, denotes that the task is processed locally by the vehicle, denotes that the task is offloaded to the nearest MEC server; denote the weight coefficients of speed / data volume / urgency respectively, used to control the influence degree of different factors on priority, denotes a neighboring service domain of the edge server, used to judge the feasibility of task hopping, and the neighboring server set thereof is defined as wherein is the distance between two edge servers, is the maximum single-hop distance of the main edge server, and the preferred value is 2 times the RSU coverage radius , that is, Because this model only considers a single main server, the subscript of is omitted.

[0017] As a preferred technical solution of the present invention: In step S1, based on the vehicle network task offloading and resource scheduling optimization model with dual priorities and dynamic task jump, the local offloading model of the vehicle user is represented as follows: (2); (3); This indicates the latency required for task i to be processed locally. This indicates the energy consumption required for task i to perform calculations locally. This represents the computing resources required for task i. This represents the local computing power of the mobile vehicle corresponding to task i. This represents the energy consumption of the moving vehicle corresponding to task i for one CPU cycle of computation. Specifically, a system is constructed that includes multiple base stations, each deploying an edge server and a movable antenna, with N vehicle users. Task transmission priority is represented as: (4); This represents the overall communication priority of task i, used to indicate the weight of communication resource allocation; It is divided into weighted coefficients representing speed, data volume, and urgency, used to control the degree of influence of different factors on the priority of task communication; This represents the instantaneous velocity of vehicle i in time slot t. This indicates the maximum speed allowed on the current road. This represents the amount of raw data for vehicle i in the current task within time slot t. This indicates all vehicle tasks within the same edge server service range under the current time slot t. The average data volume, i.e. This is used to eliminate deviations in the absolute value of data. This is a task urgency indicator, used to ensure the absolute priority of urgent tasks; Task calculation priority is represented as: (5); This indicates the overall computational priority of task i. Let $\mathbf{i}$ be the maximum latency that task $i$ can tolerate. This represents the time that task i has been computed on the edge server. For a time smoothing term, This indicates the urgency of a task, with α being the urgency coefficient, ensuring the absolute priority of urgent tasks.

[0018] As a preferred technical solution of the present invention: In step S1, based on the vehicle network task offloading and resource scheduling optimization model with dual priority and dynamic task jump, the task preemption mechanism is as follows: (6); Where C is the candidate task set, representing the set of tasks that the current task can preempt. This represents the set of tasks currently running on this edge server. This indicates the maximum computing resources that the main edge server in the current time slot can provide. This represents task j, which is assumed to be preempted from computing resources. This represents the minimum computing resources required for the current task i. This represents the dynamic release ratio, used to ensure that the released resources exactly meet the needs of the new task. It is calculated as follows: If C is not an empty set, then select the candidate set C for priority calculation. The lowest priority task j is selected, and it is determined whether the current priority of task i is higher than that of task j. If it is higher, partial preemption of the computing resources of task j is performed, and the computing power occupied by task i is released proportionally and allocated to the new task. If C is an empty set, or the computing priority of task j that needs to be preempted is higher than that of task i, the preemption phase is skipped, and task i directly enters the migration process. Specifically, the task migration mechanism is as follows: (7); This indicates the cluster of adjacent edge servers to the current region's main server. , This indicates the server with the lowest load rate in the cluster. Indicates server The original computing resources, Indicates server The total amount of computing resources already used.

[0019] As a preferred technical solution of the present invention: In step S1, based on the vehicle network task offloading and resource scheduling optimization model with dual priorities and dynamic task switching, the communication resource allocation strategy in the same time slot is expressed as follows: (8); (9); This represents the proportion of spectrum resources currently allocated by the main edge server to vehicle task i, and N represents the set of all tasks within the current main edge server's coverage area. Here, the impact of mutual interference between different edge servers and different channels is ignored, and resource allocation within the coverage area of ​​a single server is optimized only. denotes the communication priority weight of task i, denotes the sum of communication priority weights of all tasks in the edge server scope; denotes the actual uplink rate of vehicle i unloading task to the main server, W denotes the channel bandwidth of the total available spectrum of the edge server, denotes the transmission power of vehicle i uploading task to the RSU, denotes the noise power spectral density, denotes the channel gain of wireless propagation between vehicle i and the edge server.

[0020] As a preferred technical solution of the present application: in step S1, based on the dual-priority and task dynamic jump vehicle networking task unloading and resource scheduling optimization model, the consumed delay and energy of vehicle task unloading and scheduling are respectively represented as: (10); (11); (12); (13); denotes the transmission delay of task i, denotes the uplink rate of task i uploaded to the edge server, denotes the data size of task i; denotes the transmission energy consumption of task i, denotes the upload power of vehicle i uploading task; denotes the computing delay of edge server m processing task i, denotes the computing resource required by task i, denotes the total computing resource of edge server m, denotes the computing resource ratio of edge server m allocated to task i; denotes the computing energy consumption of edge server m processing task i, denotes the computing power of edge server m.

[0021] As a preferred technical solution of the present application: in step S1, based on the dual-priority and task dynamic jump vehicle networking task unloading and resource scheduling optimization model, in the case of task jump, the consumed delay and energy are represented as: (14); (15); denotes the jump delay of task i migration, is the exclusive channel bandwidth between edge servers, denotes the data size of task i, denotes the signal transmission power of the edge server, denotes the channel gain between the two servers, denotes the noise power spectral density; denotes the jump energy consumption of task i when migration occurs, denotes the signal transmission power of the edge server when task migration occurs.

[0022] As a preferred technical solution of the present application: in step S1, based on the double-priority and task dynamic jump vehicle networking task offloading and resource scheduling optimization model, the consumed delay and energy are represented as: (16); (17); denotes the total edge computing delay of task i, denotes the transmission delay of task i, denotes the sum of the computing delays of the edge server k processing task i, k is the edge server where the task finally locates, is the time slice, which describes the dynamic situation that the task is interrupted and allocated with different computing resources multiple times during execution, denotes the jump delay of task i when migration occurs; denotes the total edge computing energy consumption of task i, denotes the transmission energy consumption of task i, denotes the sum of the computing energy consumptions of the edge server k processing task i, denotes the jump energy consumption of task i when migration occurs, m denotes the main edge server selected by the current model.

[0023] As a preferred technical solution of the present application: in step S1, based on the double-priority and task dynamic jump vehicle networking task offloading and resource scheduling optimization model, the consumed delay and energy are represented as: (18); (19); denotes the comprehensive processing delay of task i, denotes the local processing delay of task i, denotes the edge offloading processing delay of task i; denotes the comprehensive processing energy consumption of task i, denotes the local processing energy consumption of task i, denotes the edge offloading processing energy consumption of task i; denotes the offloading decision of task i.

[0024] As a preferred technical solution of the present application: in step S2, the Deep Q-learning algorithm in reinforcement learning is used for solving, specifically: using the Deep Q-learning algorithm in reinforcement learning to solve the optimal joint strategy of each scheduling time slot, the specific process is as follows: S21, the target joint optimization problem is represented as: (1); ; wherein, denotes the sum of the delay generated by the offloading and scheduling of vehicle task i at time slot t on edge server m, denotes the sum of the energy consumption generated by the offloading and scheduling of vehicle task i at time slot t on edge server m; denotes the weighted sum of delay and energy consumption; M denotes the total set of edge servers, and m denotes a specific edge server in the set M, denotes all vehicle tasks under edge server m, and i denotes the task generated by the ith vehicle in the set, denotes the maximum processing delay that the task i can tolerate, denotes the percentage of computing resources allocated by edge server m to vehicle task i, and N denotes the total number of vehicle sets within the range of edge server m, denotes the offloading decision of task i at time slot t, denotes that the task is processed locally by the vehicle, denotes that the task is offloaded to the nearest MEC server; respectively denote the weight coefficients of speed / data volume / urgency, used to control the influence degree of different factors on priority, denotes a neighboring service domain of the edge server, used to judge the feasibility of task jumping, and the neighboring server set thereof is defined as wherein is the distance between two edge servers, is the maximum single-hop distance of the main edge server, and the preferred value is 2 times the RSU coverage radius , i.e. , because this model only considers a single main server, the subscript of is omitted; S22, the Deep Q-learning algorithm in reinforcement learning is used to solve the problem, and the specific steps are as follows: S221, a Markov decision process is constructed and represented as: (20); where denotes the state space, denotes the action space, denotes the immediate decision reward, denotes the state transition probability, where denotes is: (21); where denotes the set of neighboring edge servers, denotes the set of vehicle task information, denotes the set of vehicle dynamic information, including the speed vector and position vector of the vehicle, denotes is: (22); where denote the weight coefficients of speed / data volume / urgency, respectively, used to control the degree of influence of different factors on priority, denotes the offloading decision of task i at time slot t, denotes the percentage of computing resources allocated by edge server m to vehicle task i; S222, construct the target network: In the deep Q-learning algorithm, the main purpose of introducing the target network is to improve the stability of Q-value training and alleviate the problems caused by overestimating Q-value. The network updates the target Q-value according to the Bellman equation, and its mathematical expression is as follows: (23); where denotes the target Q-value, denotes the reward obtained by the agent at the i-th step, denotes the discount factor, denotes the action selected at the i+1-th step, is the Q-value output by the target network, represents the state at the i+1-th step, is the parameter set in the target network, S223, construct the experience replay pool: The experience replay mechanism is used to store historical state transition data generated during the interaction between the agent and the environment, and to extract batch samples from it in a random sampling manner during model training. This mechanism represents each step of interaction as a multi-tuple as follows: (24); where represents the experience replay buffer, denotes the state at the i-th step, It is the action chosen by the agent at step i. Indicates the execution of an action The instant reward obtained afterward Indicates the successor state. This is a termination identifier used to indicate whether the state is a termination state. In the training process of deep reinforcement learning algorithms, the loss function is designed by combining the mean squared error criterion and the Bellman optimality principle, and is constructed as follows: (25); in, Indicates The training loss is for the parameters. For mathematical expectation operators, For the transfer samples sampled from the experience cache, This represents the target Q-value calculated through the target network. This represents the current Q-network's estimated output for the state and action; S224, Construction Greedy strategy: In the action selection mechanism of the deep Q-learning algorithm, the following is adopted: A greedy strategy aims to achieve an effective balance between exploration and exploitation, and this strategy uses probability... Randomly selecting actions enhances the exploration of potential environmental behaviors; simultaneously, using probability... Choosing the action with the maximum Q value in the current state to achieve optimal utilization of existing knowledge can be expressed in the following form: (26); (27); in This represents the action selected in step i. This represents a uniform distribution in action space A. This represents the exploration probability hyperparameter. This represents the current Q-network's estimated output regarding the state and action. This represents the action that produces the maximum Q value. To select the function for this action, In the exploration-exploitation balancing strategy of reinforcement learning, a linear decay mechanism is used to dynamically adjust the exploration rate parameter to achieve the goal of sufficient exploration in the early stage of training and stable exploitation in the later stage. This mechanism can be expressed in the following mathematical form: (28); in This represents the exploration rate value during the i-th training iteration. denotes the maximum value, denotes the minimum value, is the current training step number, denotes the preset total training step number.

[0025] Compared with the prior art, the beneficial effects of the present application are: The present application builds a vehicle networking edge computing system model by constructing a fusion task jump mechanism and a double-priority scheduling, establishes a joint optimization function with the goal of minimizing the total system delay and total energy consumption. Through the deep reinforcement learning algorithm, the road environment state is dynamically sensed, the optimal communication priority strategy is intelligently selected, and the task offloading decision and computing resource allocation are cooperatively optimized, the system global performance optimization is realized, and the task processing delay is significantly reduced and the service quality is guaranteed. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a vehicle networking edge computing task offloading and resource scheduling optimization method flowchart; Figure 2 is a vehicle networking task offloading and resource scheduling optimization system schematic diagram; Figure 3 is a comparison chart of the double-priority dynamic jump algorithm and the greedy allocation algorithm, the uniform allocation algorithm and the random allocation algorithm in the system total cost aspect proposed by the present application; Figure 4 is a communication energy consumption comparison chart of the algorithm proposed by the present application and the fixed resource scheduling algorithm and the random scheduling algorithm. DETAILED DESCRIPTION

[0027] The present application will be further described in detail below in combination with the drawings and specific embodiments: As Figure 1 shown, the present embodiment provides a vehicle networking edge computing task offloading and resource scheduling optimization method, including the following steps: S1: build a vehicle networking task offloading and resource scheduling optimization model based on double-priority and task dynamic jump, and establish a target function of minimizing offloading scheduling delay and total energy consumption according to the total cost of mobile vehicle task offloading and scheduling; S2: model the joint optimization problem as a Markov decision process; adopt the DeepQ-learning algorithm in deep reinforcement learning to continuously learn and optimize the strategy through the interaction between the agent and the environment, directly output the optimal joint strategy of each scheduling time slot, so as to realize the minimization of the system total cost; S3: In each scheduling time slot, the edge server allocates communication bandwidth and computing resources to vehicle users according to the optimal communication strategy, task offloading decision and computing resource allocation scheme obtained by solving, triggers task preemption and cross-server hopping mechanism, calculates the total system delay and total energy consumption of the current time slot, and outputs the optimized system performance index.

[0028] The method of the embodiment realizes the minimization of the total cost of the vehicle networking edge computing system through the joint optimization of vehicle user priority scheduling and task dynamic hopping, significantly improves the task completion rate, and reduces the system communication energy consumption.

[0029] In step S1, the target function of minimizing the weighted sum of offloading scheduling delay and total energy consumption includes: The target function includes two parts, namely delay cost and energy consumption cost, and the optimization problem target function is as follows: (1); ; Wherein, denotes the sum of the delay generated by offloading and scheduling of vehicle task i at time slot t on edge server m, denotes the sum of the energy consumption generated by offloading and scheduling of vehicle task i at time slot t on edge server m; denotes the weighted sum of delay and energy consumption; M denotes the total set of edge servers, and m denotes a specific edge server in the set M, denotes all vehicle tasks under edge server m, i denotes the task generated by the ith vehicle, denotes the maximum processing delay that the task i can tolerate, denotes the percentage of computing resources allocated by edge server m to vehicle task i, N denotes the total number of vehicle sets within the range of edge server m, denotes the offloading decision of task i at time slot t, denotes that the task is processed by the vehicle locally, denotes that the task is offloaded to the nearest MEC server; denote the weight coefficients of speed / data volume / urgency, respectively, for controlling the influence degree of different factors on priority, denotes a neighboring service domain of the edge server, which is used to judge the feasibility of task hopping, and the neighboring server set thereof is defined as wherein is the distance between two edge servers, is the maximum single-hop distance of the main edge server, and the preferred value is 2 times the RSU coverage radius , that is, Since this model only considers a single master server, it is omitted. subscript; In step S1, based on the vehicle-to-everything (V2X) task offloading and resource scheduling optimization model with dual priorities and dynamic task switching, the local offloading model for vehicle users can be represented as: (2); (3); This indicates the latency required for task i to be processed locally. This indicates the energy consumption required for task i to perform calculations locally. This represents the computing resources required for task i. This represents the local computing power of the mobile vehicle corresponding to task i. This represents the energy consumption of the moving vehicle corresponding to task i for one CPU cycle of computation. Specifically, a system is constructed that includes multiple base stations, each deploying an edge server and a movable antenna, with N vehicle users. Task transmission priority is represented as: (4); This represents the overall communication priority of task i, used to indicate the weight of communication resource allocation; It is divided into weighted coefficients representing speed, data volume, and urgency, used to control the degree of influence of different factors on the priority of task communication; This represents the instantaneous velocity of vehicle i in time slot t. This indicates the maximum speed allowed on the current road. This represents the amount of raw data for vehicle i in the current task within time slot t. This indicates all vehicle tasks within the same edge server service range under the current time slot t. The average data volume, i.e. This is used to eliminate deviations in the absolute value of data. This is a task urgency indicator, used to ensure the absolute priority of urgent tasks (such as collision warnings).

[0030] Based on the vehicle-to-everything (V2X) task offloading and resource scheduling optimization model with dual priorities and dynamic task switching, the task computation priority can be expressed as: (5);

[0031] This indicates the overall computational priority of task i. Let $\mathbf{i}$ be the maximum latency that task $i$ can tolerate. This represents the time that task i has been computed on the edge server. For a time smoothing term, This indicates the urgency of a task, with α being the urgency coefficient, ensuring the absolute priority of urgent tasks.

[0032] Current research often employs a single priority approach to coordinate bandwidth and computing resources. However, in practice, the constraint characteristics of these two types of resources differ. Therefore, this application proposes... and The dual-priority mechanism decouples transmission and computation, better aligning with physical layer characteristics and demonstrating technical rationality. Meanwhile, traditional solutions only consider static computational load. The design of this application introduces dynamic remaining time perception, which is more suitable for latency-sensitive scenarios in vehicle networking; by isolating time windows, conflicts are avoided, and high transmission priority tasks can naturally enter the computing queue earlier, forming implicit collaboration.

[0033] In step S1, based on the vehicle-to-everything (V2X) task offloading and resource scheduling optimization model with dual priority and dynamic task switching, the task preemption mechanism is as follows: (6); Where C is the candidate task set, representing the set of tasks that the current task can preempt. This represents the set of tasks currently running on this edge server. This indicates the maximum computing resources that the main edge server in the current time slot can provide. This represents task j, which is assumed to be preempted from computing resources. This represents the minimum computing resources required for the current task i. This represents the dynamic release ratio, used to ensure that the released resources exactly meet the needs of the new task. It is calculated as follows: If C is not an empty set, then select the candidate set C for priority calculation. The lowest priority task j is selected, and it is determined whether the current priority of task i is higher than that of task j. If it is higher, partial preemption of the computing resources of task j is performed, and the computing power occupied by task i is released proportionally and allocated to the new task. If C is an empty set, or the computing priority of task j that needs to be preempted is higher than that of task i, the preemption phase is skipped, and task i directly enters the migration process. Specifically, the task migration mechanism is as follows: (7); This indicates the cluster of adjacent edge servers to the current region's main server. , This indicates the server with the lowest load rate in the cluster. Indicates server The original computing resources, Indicates server The total sum of the computing resources that have been used. The first formula selects the neighboring server with the lowest load rate instead of the server with the absolute lowest load, which better reflects the relative load pressure between servers and achieves load balancing. Note that at this time the vehicle still unloads the local task to its corresponding main server, and whether the task migrates is determined and implemented by the server. The communication bandwidth between edge servers is fixed by default, i.e., the communication speed between servers is fixed.

[0034] In step S1, the communication resource allocation strategy in the same time slot can be represented based on the double-priority and task dynamic jump vehicle networking task offloading and resource scheduling optimization model as follows: (8); (9); denotes the spectrum resource proportion allocated by the current main edge server to the vehicle task i, N denotes all task sets within the range of the current main edge server, and the influence of mutual interference between different edge servers and different channels is ignored here, and only the resource allocation within the coverage range of a single server is optimized; denotes the communication priority weight of task i, denotes the sum of the communication priority weights of all tasks within the range of the edge server; denotes the actual uplink rate of the vehicle i unloading the task to the main server, W denotes the channel bandwidth of the total available spectrum of the edge server, denotes the transmission power of the vehicle i uploading the task to the RSU, denotes the noise power spectral density, denotes the channel gain of wireless propagation between the vehicle i and the edge server.

[0035] In step S1, based on the double-priority and task dynamic jump vehicle networking task offloading and resource scheduling optimization model, the delay and energy consumed by vehicle task offloading and scheduling can be represented as follows: (10); (11); (12); (13); denotes the transmission delay of task i, denotes the uplink rate of task i uploaded to the edge server, denotes the data size of task i; denotes the transmission energy consumption of task i, denotes the upload power of the vehicle i uploading the task; denotes the computation latency of edge server m handling task i, denotes the computation resource required by task i, denotes the total computation resource of edge server m, denotes the computation resource ratio of edge server m allocated to task i; denotes the computation energy consumption of edge server m handling task i, denotes the computation power of edge server m.

[0036] In step S1, based on the vehicle networking task offloading and resource scheduling optimization model of double priority and task dynamic jump, in the case of task jump, the consumed latency and energy can be expressed as: (14); (15); denotes the jump latency of task i migration, is the channel bandwidth exclusive to edge servers, denotes the data size of task i, denotes the signal transmission power of edge server, denotes the channel gain between two servers, denotes the noise power spectral density; denotes the jump energy consumption of task i migration, denotes the signal transmission power of edge server when performing task migration.

[0037] In step S1, based on the vehicle networking task offloading and resource scheduling optimization model of double priority and task dynamic jump, in the case of non-local computing, the consumed latency and energy can be expressed as: (16); (17); denotes the total edge computing latency of task i, denotes the transmission latency of task i, denotes the sum of computation latency of edge server k handling task i, k is the edge server where the task finally locates, is the time slice, describing the dynamic situation that the task is interrupted and allocated with different computation resources multiple times during execution, denotes the jump latency of task i migration; denotes the total edge computing energy consumption of task i, denotes the transmission energy consumption of task i, denotes the sum of computation energy consumption of edge server k handling task i, The jump energy consumption represents the task i migration, and m represents the main edge server selected by the current model.

[0038] In step S1, based on the double priority and task dynamic jump, the task offloading and resource scheduling optimization model of Internet of Vehicles, the time delay and energy consumed by the model for task offloading and resource scheduling can be represented as: (18); (19); The comprehensive processing time delay of task i is represented as The local processing time delay of task i is represented as The edge offloading processing time delay of task i is represented as The comprehensive processing energy consumption of task i is represented as The local processing energy consumption of task i is represented as The edge offloading processing energy consumption of task i is represented as The offloading decision of task i is represented as

[0039] In step S2, the Deep Q-learning algorithm in reinforcement learning is used to solve it, specifically: using the Deep Q-learning algorithm in reinforcement learning to solve the optimal joint strategy of each scheduling time slot, the specific process is as follows: S21, the target joint optimization problem is represented as: (1); ; Wherein, The sum of the time delay generated by the offloading and scheduling of vehicle task i on edge server m at time slot t is represented as The sum of the energy consumption generated by the offloading and scheduling of vehicle task i on edge server m at time slot t is represented as The weighted sum of time delay and energy consumption is represented as M represents the total set of edge servers, and m represents a specific edge server in set M, All vehicle tasks under edge server m are represented as i represents The task generated by the i-th vehicle in the set, The maximum processing time delay that the task i can tolerate is represented as The percentage of computing resources allocated by edge server m to vehicle task i is represented as N represents the total number of vehicle sets within the range of edge server m, The offloading decision of task i at time slot t is represented as The task is processed by the vehicle locally, The task is offloaded to the nearest MEC server; weight coefficients of speed / data volume / urgency, used to control the degree of influence of different factors on priority, denotes a proximity service domain of the edge server, used to determine the feasibility of task hopping, and defines its proximity server set as wherein is the distance between two edge servers, is the maximum single-hop distance of the main edge server, and the preferred value is 2 times the RSU coverage radius , i.e. Since the model only considers a single main server, the subscript of is omitted; S22, the Deep Q-learning algorithm in reinforcement learning is used to solve the problem, and the specific steps are as follows: S221, a Markov decision process is constructed and represented as: (20); wherein denotes the state space, denotes the action space, denotes the immediate decision reward, denotes the state transition probability, wherein denotes (21); wherein denotes the proximity edge server set, denotes the vehicle task information set, denotes the vehicle dynamic information set, including the speed vector and position vector of the vehicle, denotes (22); wherein weight coefficients of speed / data volume / urgency, used to control the degree of influence of different factors on priority, denotes the offloading decision of task i at time slot t, denotes the percentage of computing resources allocated by edge server m to vehicle task i; S222, the target network is constructed: In the deep Q-learning algorithm, the main purpose of introducing the target network is to improve the stability of Q value training and alleviate the problems caused by overestimating Q values. The network updates the target Q value according to the Bellman equation, and its mathematical expression is as follows: (23); wherein denotes the target Q value, represents the reward obtained by the agent at the i-th step, represents the discount factor, represents the action selected at the i+1-th step, is the Q value output by the target network, represents the state at the i+1-th step, is the parameter set in the target network, S223, constructing an experience replay pool: The experience replay mechanism is a core design in deep Q learning algorithms, mainly used to store historical state transition data generated during the interaction between the agent and the environment, and to extract batch samples from it in a random sampling manner during model training. This mechanism represents each interaction record as a multi-tuple as follows: (24); wherein represents the experience replay buffer, represents the state at the i-th step, is the action selected by the agent at the i-th step, represents the immediate reward obtained after executing the action , represents the successor state, is a termination identifier, used to identify whether the state is a terminal state, In the training process of deep reinforcement learning algorithms, the design of the loss function has a key impact on the performance of the model. This method combines the mean square error criterion with the Bellman optimality principle to construct a loss function as follows: (25); wherein, represents the training loss with as the parameter, is the mathematical expectation operator, is the transition sample sampled from the experience buffer, represents the target Q value calculated by the target network, represents the estimation output of the current Q network for the state and action; S224, constructing greedy policy: In the action selection mechanism of deep Q learning algorithms, a greedy policy is used to achieve an effective balance between exploration and exploitation. This policy randomly selects an action with a probability , thereby enhancing the exploration of potential behaviors of the environment; at the same time, it selects the action with the maximum Q value in the current state with a probability , achieving optimal utilization of existing knowledge, and its decision-making process can be expressed as follows: (26); (27); wherein denotes the action selected at the i-th step, denotes a uniform distribution over the action space A, denotes the exploration probability hyperparameter, denotes the current Q-network's estimated output for a state-action pair, denotes the action that yields the maximum Q-value, is a function that selects this action, In the exploration-exploitation balance strategy of reinforcement learning, a linear decay mechanism is usually used to dynamically adjust the exploration rate parameter to achieve the goal of sufficient exploration in the early stage of training and stable exploitation in the later stage. This mechanism can be expressed in mathematical form as follows (28); wherein denotes the exploration rate value at the i-th training iteration, denotes the maximum value, denotes the minimum value, is the current training step number, denotes the preset total training step number.

[0040] Figure 3 is a comparison chart of the total system cost of the double-priority dynamic jump algorithm and the greedy allocation algorithm, the uniform allocation algorithm, and the random allocation algorithm proposed in the present application. Figure 4 is a communication energy consumption comparison chart of the algorithm proposed in the present application and the fixed resource scheduling algorithm and the random scheduling algorithm. From Figure 3 it can be seen that the algorithm proposed in the present application significantly reduces the total system running cost through intelligent priority scheduling and cross-server resource coordination, and its performance is obviously better than that of the traditional greedy strategy, uniform allocation, and random unloading method. From Figure 4 it can be seen that the algorithm proposed in the present application effectively optimizes the communication energy consumption efficiency through the dynamic resource allocation and task jump mechanism, and exhibits a significant energy consumption advantage compared with the fixed scheduling and random scheduling scheme. The embodiment constructs a task offloading and resource scheduling optimization model based on double priority and task dynamic jump of Internet of Vehicles, establishes a joint optimization objective function of minimizing total time delay and total energy consumption of the system according to a computing model, a communication model, a quality of service requirement and a system energy consumption of a vehicle task, intelligently perceives an environment state through a deep reinforcement learning algorithm, dynamically outputs an optimal communication priority strategy, a task offloading decision and a computing resource allocation scheme, and triggers a task preemption and server jump mechanism, and finally realizes minimization of a global cost of an edge computing system of Internet of Vehicles, significantly improves a completion rate of a high-priority task, and reduces system communication energy consumption.

[0041] As shown in Figure 2 The embodiment also provides a task offloading and resource scheduling optimization system of Internet of Vehicles, which comprises: A construction module is configured to construct a task offloading and resource scheduling model based on double priority and task dynamic jump of Internet of Vehicles, and establish a target function of minimizing offloading and scheduling time delay and total energy consumption according to a total cost of mobile vehicle task offloading and scheduling; A joint optimization module is configured to obtain an optimal joint strategy, model the joint optimization problem as a Markov decision process, and directly output an optimal joint strategy of each scheduling time slot through interaction between an agent and an environment by using a Deep Q-learning algorithm in deep reinforcement learning, so as to realize minimization of a total cost of the system; A task preemption migration module is configured to optimize a system performance index, allocate communication bandwidth and computing resources for a vehicle user according to the optimal communication strategy, task offloading decision and computing resource allocation scheme obtained by solving, and trigger a task preemption and cross-server jump mechanism in each scheduling time slot, calculate a total time delay and total energy consumption of the system in the current time slot, and output an optimized system performance index.

[0042] Preferably, the construction module establishes a target function of minimizing a weighted sum of offloading and scheduling time delay and total energy consumption, and the target function comprises: The target function comprises two parts, namely a time delay cost and an energy consumption cost, and the optimization problem target function is as follows: (1); ; Wherein, denotes a sum of time delays of vehicle task i offloaded and scheduled on edge server m in time slot t, denotes a sum of energy consumptions of vehicle task i offloaded and scheduled on edge server m in time slot t; denotes a weighted sum of time delay and energy consumption; M denotes a total set of edge servers, and m denotes a specific edge server in the set M, denotes all vehicle tasks under edge server m, i denotes the i-th vehicle generated task, denotes the maximum processing delay that task i can tolerate, denotes the percentage of computing resources that edge server m allocates to vehicle task i, N denotes the total number of vehicle set within edge server m, denotes the offloading decision of task i at time slot t, denotes that the task is processed locally by the vehicle, denotes that the task is offloaded to the nearest MEC server; respectively denote the weight coefficients of speed / data volume / urgency, which are used to control the influence degree of different factors on priority, denotes a neighboring service domain of the edge server, which is used to judge the feasibility of task jump, and defines its neighboring server set as wherein is the distance between two edge servers, is the maximum single-hop distance of the main edge server, and the preferred value is 2 times the RSU coverage radius , that is, because the present model only considers a single main server, the subscript of is omitted; the Deep Q-learning algorithm in reinforcement learning is used to solve the objective function of the sub-problem, and the optimized offloading decision and communication and computing resource allocation decision are obtained.

[0043] The specific implementation process of the functions realized by each module in the vehicle networking task offloading and resource scheduling optimization system in the present embodiment is the same as the implementation process in the vehicle networking task offloading and resource scheduling optimization method, and will not be repeated here.

[0044] The above only describes the preferred embodiments of the present application, and does not limit the protection scope of the present application, and any equivalent structural transformation made under the concept of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. An optimization method for task offloading and resource scheduling of edge computing in Internet of Vehicles, characterized in that, The method comprises the following steps: S1: constructing a task offloading and resource scheduling optimization model based on double priority and task dynamic jump, and establishing a target function for minimizing offloading scheduling delay and total energy consumption according to the total cost of mobile vehicle task offloading and scheduling; S2: modeling the joint optimization problem as a Markov decision process; using a deep reinforcement learning algorithm, the agent continuously learns and optimizes the strategy through interaction with the environment, directly outputs the optimal joint strategy for each scheduling time slot, and thus minimizes the total cost of the system; S3: in each scheduling time slot, the edge server allocates communication bandwidth and computing resources to the vehicle user according to the optimal communication strategy, task offloading decision and computing resource allocation scheme obtained by solving, and triggers the task preemption and cross-server jump mechanism, calculates the total delay and total energy consumption of the system in the current time slot, and outputs the optimized system performance indicators. 2.The optimization method for task offloading and resource scheduling of edge computing in V2X, according to claim 1, wherein, In step S1, the target function for minimizing the weighted sum of offloading scheduling delay and total energy consumption comprises: The target function includes two parts, namely delay cost and energy cost, and the optimization problem target function is as follows: (1); ; wherein, denotes the sum of the latency caused by offloading and scheduling of vehicle task i at edge server m in time slot t, denotes the sum of the energy consumption caused by offloading and scheduling of vehicle task i at edge server m in time slot t; denotes the weighted sum of latency and energy consumption; M denotes the total set of edge servers, m denotes a specific edge server in set M, denotes all vehicle tasks at edge server m, i denotes the task generated by the i-th vehicle, denotes the maximum processing latency that task i can tolerate, denotes the percentage of computing resources allocated by edge server m to vehicle task i, N denotes the total number of vehicles within the range of edge server m, denotes the offloading decision of task i in time slot t, denotes that the task is processed locally by the vehicle, denotes that the task is offloaded to the nearest MEC server; denote the weight coefficients of speed / data volume / urgency, respectively, for controlling the degree of influence of different factors on the priority, denotes a neighboring service domain of the edge server, for judging the feasibility of task hopping, and defines its neighboring server set as wherein is the distance between two edge servers, is the maximum single-hop distance of the primary edge server, and the preferred value is 2 times the RSU coverage radius i.e. because this model only considers a single primary server, the subscript of is omitted. 3.The optimization method for task offloading and resource scheduling of edge computing in V2X, according to claim 1, wherein, In step S1, the task offloading and resource scheduling optimization model based on double priority and task dynamic jump is represented as: (2); (3); denotes the latency needed for task i to be processed locally, denotes the energy consumption needed for task i to be computed locally, denotes the computing resources needed for task i, denotes the local computing capability of the mobile vehicle corresponding to task i, denotes the energy consumption of the mobile vehicle corresponding to task i for one CPU cycle computation, Specifically, a system comprising multiple base stations, each of which deploys an edge server and a movable antenna, exists N vehicle users, The task transmission priority is represented as: (4); This represents the overall communication priority of task i, used to indicate the weight of communication resource allocation; It is divided into weighted coefficients representing speed, data volume, and urgency, used to control the degree of influence of different factors on the priority of task communication; This represents the instantaneous speed of vehicle i in time slot t. This indicates the maximum speed allowed on the current road. This represents the amount of raw data for vehicle i in the current task within time slot t. This indicates all vehicle tasks within the same edge server service range under the current time slot t. The average data volume, i.e. This is used to eliminate deviations in the absolute value of data. This is a task urgency indicator, used to ensure the absolute priority of urgent tasks; The task computation priority is represented as: (5); a combined computation priority of task i, a maximum latency tolerable for task i, a time that task i has been computed at the edge server, a time smoothing term, an identification of task urgency, and a is an urgency coefficient to ensure absolute priority of urgent tasks.

4. The optimization method for task offloading and resource scheduling of edge computing in Internet of Vehicles according to claim 1, characterized in that, In step S1, the task preemption mechanism of the task offloading and resource scheduling optimization model based on double priority and task dynamic jump is as follows: (6); Where C is the candidate task set, representing the set of tasks that the current task can preempt. This represents the set of tasks currently running on this edge server. This indicates the maximum computing resources that the main edge server in the current time slot can provide. This represents task j, which is assumed to be preempted from computing resources. This represents the minimum computing resources required for the current task i. This represents the dynamic release ratio, used to ensure that the released resources exactly meet the needs of the new task. It is calculated as follows: If C is not an empty set, then select the candidate set C to calculate the priority. The lowest priority task j is selected, and it is determined whether the current priority of task i is higher than that of task j. If it is higher, partial preemption of the computing resources of task j is performed, and the computing power occupied by task i is released proportionally and allocated to the new task. If C is an empty set, or the computing priority of task j that needs to be preempted is higher than that of task i, the preemption phase is skipped, and task i directly enters the migration process. Specifically, the task migration mechanism is as follows: (7); a cluster of proximate edge servers representing the current regional master server , a server in the cluster with the lowest load rate, a server original computing resources of the server, a server total computing resources already used by the server.

5. The optimization method for task offloading and resource scheduling of edge computing in Internet of Vehicles according to claim 1, characterized in that, In step S1, the communication resource allocation strategy in the same time slot of the task offloading and resource scheduling optimization model based on double priority and task dynamic jump is represented as: (8); (9); denotes the spectrum resource proportion allocated to vehicle task i by the current primary edge server, N denotes the set of all tasks within the current primary edge server range, the influence of mutual interference between different edge servers and different channels is ignored here, and only the resource allocation within the coverage range of a single server is optimized; denotes the communication priority weight of task i, denotes the sum of the communication priority weights of all tasks within the edge server range; denotes the actual uplink rate of vehicle i unloading tasks to the primary server, W denotes the channel bandwidth of the total available spectrum of the edge server, denotes the transmission power of vehicle i when uploading tasks to the RSU, denotes the noise power spectral density, denotes the channel gain of wireless propagation between vehicle i and the edge server.

6. The optimization method for task offloading and resource scheduling of edge computing in Internet of Vehicles according to claim 1, characterized in that, In step S1, the delay and energy consumed by the vehicle task offloading and scheduling of the task offloading and resource scheduling optimization model based on double priority and task dynamic jump are represented as: (10); (11); (12); (13); denotes the transmission latency of task i, denotes the uplink rate of task i uploaded to the edge server, denotes the data size of task i; denotes the transmission energy consumption of task i, denotes the upload power of vehicle i when uploading tasks; denotes the computation latency of edge server m processing task i, denotes the computation resource required by task i, denotes the total computation resource of edge server m, denotes the computation resource ratio of edge server m allocated to task i; denotes the computation energy consumption of edge server m processing task i, denotes the computation power of edge server m.

7. The optimization method for task offloading and resource scheduling of edge computing in Internet of Vehicles according to claim 1, characterized in that, In step S1, the delay and energy consumed by the vehicle task offloading and scheduling of the task offloading and resource scheduling optimization model based on double priority and task dynamic jump are represented as: (14); (15); a jump latency representing a migration of task i, a channel bandwidth exclusive to the edge servers, a data size of task i, a signal transmission power of the edge server, a channel gain between the two servers, a noise power spectral density; a jump energy consumption representing a migration of task i, a signal transmission power of the edge server when performing task migration. 8.The optimization method for task offloading and resource scheduling of edge computing in Internet of Vehicles according to claim 1, wherein, In step S1, the delay and energy consumed by the vehicle task offloading and scheduling of the task offloading and resource scheduling optimization model based on double priority and task dynamic jump are represented as: (16); (17); denotes the total edge computing latency of task i, denotes the transmission latency of task i, denotes the sum of the computation latency of task i at edge server k, k is the edge server where the task finally locates, is the time slice, which describes the dynamic situation that the task is interrupted and allocated with different computing resources for multiple times during the execution, denotes the jump latency of task i when migration occurs; denotes the total edge computing energy consumption of task i, denotes the transmission energy consumption of task i, denotes the sum of the computation energy consumption of task i at edge server k, denotes the jump energy consumption of task i when migration occurs, m denotes the main edge server selected by the current model. 9.The optimization method of task offloading and resource scheduling for edge computing in V2X, according to claim 1, wherein, In step S1, the delay and energy consumed by the vehicle task offloading and scheduling of the task offloading and resource scheduling optimization model based on double priority and task dynamic jump are represented as: (18); (19); a composite processing latency of task i, a local processing latency of task i, an edge offloading processing latency of task i; a composite processing energy consumption of task i, a local processing energy consumption of task i, an edge offloading processing energy consumption of task i; an offloading decision of task i.

10. The optimization method for task offloading and resource scheduling of edge computing in Internet of Vehicles according to claim 1, characterized in that, In step S1, the delay and energy consumed by the vehicle task offloading and scheduling of the task offloading and resource scheduling optimization model based on double priority and task dynamic jump are represented as: In step S2, the Deep Q-learning algorithm in reinforcement learning is used for solving, specifically: the Deep Q-learning algorithm in reinforcement learning is used to solve the optimal joint strategy for each scheduling time slot, and the specific process is as follows: (1); ; wherein, denotes the sum of the latency caused by offloading and scheduling of vehicle task i at edge server m in time slot t, denotes the sum of the energy consumption caused by offloading and scheduling of vehicle task i at edge server m in time slot t; denotes the weighted sum of latency and energy consumption; M denotes the total set of edge servers, m denotes a specific edge server in set M, denotes all vehicle tasks at edge server m, i denotes the task generated by the i-th vehicle, denotes the maximum processing latency that task i can tolerate, denotes the percentage of computing resources allocated by edge server m to vehicle task i, N denotes the total number of vehicles within the range of edge server m, denotes the offloading decision of task i in time slot t, denotes that the task is processed locally by the vehicle, denotes that the task is offloaded to the nearest MEC server; denote the weight coefficients of speed / data volume / urgency, respectively, for controlling the degree of influence of different factors on the priority, denotes a neighboring service domain of the edge server, for judging the feasibility of task hopping, and defines its neighboring server set as wherein, is the distance between two edge servers, is the maximum single-hop distance of the primary edge server, and the preferred value is 2 times the RSU coverage radius i.e. because this model only considers a single primary server, the subscript of is omitted; S21, the target joint optimization problem is represented as: S22, the Deep Q-learning algorithm in reinforcement learning is used to solve the problem, and the specific steps are as follows: S221, Constructing Markov Decision Process representation as: (20); wherein denotes the state space, denotes the action space, denotes the immediate decision reward, denotes the state transition probability, wherein denotes: (21); wherein denotes a set of neighboring edge servers, denotes a set of vehicle task information, denotes a set of vehicle dynamic information, containing a velocity vector and a position vector of the vehicle, is represented by: (22); wherein respectively represent the weight coefficients of speed / data volume / urgency, used to control the degree of influence of different factors on the priority, denotes the offloading decision of task i at time slot t, denotes the percentage of computing resources allocated by edge server m to vehicle task i; S222, Constructing target network: In the deep Q-learning algorithm, the main purpose of introducing the target network is to improve the stability of Q-value training and alleviate the problem caused by overestimating Q-value. The network updates the target Q-value according to the Bellman equation, and its mathematical expression is as follows: (23); wherein represents the target Q value, represents the reward obtained by the agent at the i-th step, represents the discount factor, represents the action selected at the i+1-th step, is the Q value output by the target network, represents the state at the i+1-th step, is the set of parameters in the target network, S223, Constructing experience replay pool: The experience replay mechanism is used to store historical state transition data generated during the interaction between the agent and the environment, and to extract batch samples from it in a random sampling manner during model training. This mechanism records each interaction as a multi-tuple as follows: (24); wherein represents an experience replay buffer, represents a state at the i-th step, is an action selected by the agent at the i-th step, represents an immediate reward obtained after performing the action represents a successor state, represents a successor state, is a terminal identifier for identifying whether the state is a terminal state, In the training process of the deep reinforcement learning algorithm, the design of the loss function combines the mean square error criterion with the Bellman optimality principle to construct a loss function of the following form: (25); wherein, denotes the training loss parameterized by is the mathematical expectation operator, is a transition sample sampled from the experience buffer, represents the target Q value calculated by the target network, denotes the estimation output of the current Q network for the state and action;​ S224, constructing - Greedy strategy: In the action selection mechanism of deep Q-learning algorithm, the greedy strategy is adopted to achieve an effective balance between exploration and utilization, which selects an action with a probability and a random action with a probability of 1- to enhance the exploration of potential behaviors of the environment; meanwhile, an action with the maximum Q value in the current state is selected with a probability of 1- to achieve the optimal utilization of existing knowledge, and the decision-making process can be expressed in the following form: (26); (27); where denotes the action selected at step i, represents a uniform distribution over the action space A, denotes the exploration probability hyperparameter, denotes the current Q-network's estimate of the value of a state-action pair, denotes the action that yields the maximum Q-value, is a function that picks this action, In the exploration-exploitation balance strategy of reinforcement learning, a linear decay mechanism is used to dynamically adjust the exploration rate parameter to achieve the goal of sufficient exploration in the early stage and stable exploitation in the later stage. This mechanism can be expressed in the following mathematical form: (28); wherein denotes the value of the exploration rate at the i-th training iteration, denotes the maximum value, denotes the minimum value, is the current training step number, denotes the preset total training step number.

Citation Information

Patent Citations

  • RSU auxiliary multi-hop task unloading method based on asynchronous deep reinforcement learning

    CN119996446A

  • Method of optimizing dependent task offloading in internet of vehicles using deep reinforcement learning

    US12399756B2

  • Multi-target joint optimization task unloading strategy based on deep reinforcement learning in Internet of Vehicles

    CN116321298A

  • Task scheduling and resource allocation method based on federal reinforcement learning in Internet of Vehicles

    CN116709378A

  • Task unloading method based on deep reinforcement learning in vehicle-mounted edge environment

    CN117221951A

Cited By

  • Value maximization calculation unloading method and system based on dynamic priority scheduling and reinforcement learning

    CN121560419A

  • Value maximization computing offloading method and system based on dynamic priority scheduling and reinforcement learning

    CN121560419B

  • Intelligent driving task scheduling method and device for high energy efficiency and low time delay

    CN121900842A