A td3-smt task offloading method for minimizing delay in internet of vehicles

Abstract

The application discloses a kind of TD3-SMT task unloading methods of minimizing delay in Internet of Vehicles, comprising the following steps: step 1, construct system model framework;Step 2, user side and server side task processing are modeled;Step 3, initialize computing offloading system state;Step 4, train TD3-SMT network, update critic current network parameters;Step 5, use Softmax to estimate value function;Step 6, computing offloading system state is input into the computing offloading TD3-SMT network pre-trained;Step 7, according to the action decision of θ n Value obtained from policy function, get computing offloading system action.The application can make user vehicle optimize the unreasonable problem of offloading proportion and offloading time existing in Internet of Vehicles computing offloading while guaranteeing communication demand, meet the high reliable low delay demand of numerous applications.

Description

Technical Field

[0001] This invention relates to the Internet of Vehicles (IoV), and more specifically to a method for minimizing latency in TD3-SMT task offloading within the IoV. Background Technology

[0002] With the development of 5G wireless communication networks and vehicle-to-everything (V2X) networks, new-generation smart terminal devices generate massive amounts of data, placing higher demands on computing and communication security and quality of service. Traditional cloud computing can no longer meet the low latency and high reliability requirements of computationally intensive users. In the current in-vehicle communication environment, the amount of computational data generated by in-vehicle tasks within strict processing deadlines is growing exponentially. Limited communication and computing resources mean that reducing computational offloading decision latency remains a bottleneck. The rapidly increasing volume of data computation and the higher latency requirements, such as in autonomous driving, real-time navigation, and infotainment services, pose significant challenges. These applications are typically highly latency-sensitive. Therefore, these computationally intensive demands present a huge challenge to vehicles with limited computing resources. Mobile edge computing (MEC) pushes storage, network control, and mobile computing to the edge, resulting in lower latency, higher bandwidth, and more workloads. Currently, edge computing is the most promising technology for solving this problem. Vehicles are becoming increasingly intelligent, with continuous development in sensors, cameras, advanced driver-assistance systems (ADAS), and computing power. More and more vehicles have powerful onboard computing capabilities, which can act as servers to assist in processing tasks for other vehicles and even reduce the computing load on other devices on the network.

[0003] In new vehicular networks, edge devices can be deployed at base stations (BS), roadside units (RSU), and some vehicles. Edge devices, with their superior computing power, can significantly reduce task latency. However, if tasks are not properly scheduled, a large influx of users onto the more powerful base stations can occur. Data is transmitted via the radio access network, and users compete for spectrum resources, resulting in significant transmission latency. Furthermore, when tasks are offloaded to edge devices, users compete for computing resources, leading to substantial queuing latency. Summary of the Invention

[0004] To address the problems in existing technologies, this invention discloses a TD3-SMT task offloading method that minimizes latency in vehicle-to-everything (V2X) networks. This method solves the problems of unreasonable offloading ratios and offloading times in existing V2X computing offloading methods and reduces offloading latency.

[0005] A method for minimizing latency in TD3-SMT task offloading in vehicle-to-everything (V2X) communication includes the following steps:

[0006] Step 1: Constructing a System Model Framework. Establish a communication scenario consisting of vehicles, base stations, roadside units, and their associated MEC server equipment. Each vehicle terminal device has a computationally intensive task to complete, and each vehicle terminal device can offload the task to an edge server or execute it locally. Multiple vehicle terminal devices that simultaneously offload tasks in the system will equally share the wireless bandwidth for uploading data. The upload data rate of the nth vehicle terminal device (EUn) is given by the following formula.

[0007] ,

[0008] Where W is the bandwidth of the wireless channel, and K is the number of EUs to be offloaded. The transmission power for uploading data to EUN. This refers to the wireless channel gain between EUn and the base station. The variance of the complex Gaussian white noise;

[0009] Step 2: Construct the local computing model and the offloading model to obtain the total latency of local computing, the total latency of partially offloading computing, and the total time cost of fully offloading computing;

[0010] When executing tasks locally, the local computing model is as follows:

[0011] ,

[0012] in This represents the computational power of EUN. To indicate the task The latency of local computation, for each EU, They can be different;

[0013] Will ∈[0,1] represents the computational unloading decision of the i-th vehicle terminal device, and the computational unloading strategy A of the EU set N is defined by the unloading decision vector, which consists of all EUs, and is defined by A=[ , ,..., ],when When = 0, this means the i-th task It is calculated locally, when When =1, this means Completely unload to the MEC server when 0 < When <1, this means Tasks that are offloaded to the MEC server Part, another part of the task (1- Part of it is calculated locally;

[0014] When an EU requests computational unloading during task execution on an edge server, the task... of Some tasks were offloaded to edge servers. of The total latency includes data upload latency, MEC computation latency, and data download latency for the task. of (1- The part is computed locally, task (1- The latency of this part is similar to the local computation latency described in the local computation model. Based on the above description, the following is defined: To indicate the task of The first step in addressing data upload delays during partial offloading to the MEC server. The calculation method is given by the following formula:

[0015] = ,

[0016] Will Define the task of The partial unloading to the MEC server indicates a delay in the second-step MEC computation. The calculation method is given by the following formula:

[0017] = ,

[0018] in This indicates that the task is assigned by MEC. The computing resources are defined. To indicate when the task is of The third step of partial unloading to the MEC server resulted in data download delays. The calculation method is given by the following formula:

[0019] = ,

[0020] in This indicates the size of the data returned by the edge server in the execution result. This represents the download rate at which EUn downloads the execution results from the edge server. of (1- The computational delay of the part is defined. To indicate the task of (1- The computational delay of this part The calculation method is given by the following formula: = ,

[0021] Based on the above formula, define Tasks for executing partial computational offloading strategies Total delay, The calculation method is given by the following formula:

[0022] =max{( + ), },

[0023] Due to the mission Due to the parallelism, the total time delay of the EUN execution part's computational offloading strategy should be the task's... of Partial delays and tasks of (1- The larger of the delays in the ) part, max is the function that takes the maximum value;

[0024] Step 3: The optimization objective is to calculate the total latency of vehicle-to-everything (V2X) unloading; therefore, the state is defined as follows:

[0025] S= ,

[0026] That is, the latency of the entire vehicle-to-everything (V2X) MEC system;

[0027] Step 4: During each training session, M samples are drawn from the experience pool to train the TD3-SMT network and the current network parameters of the critic are updated. After the current network parameters of the critic are updated a specified number of times, the current network parameters of the actor are updated through loss gradient and the target network parameters of the actor are updated using a soft update method.

[0028] Step 5: Add Softmax normalization to the gradient of the double-delay deep deterministic policy to reduce the number of times local optima are obtained and reduce the sensitivity of the algorithm to different initialization parameters;

[0029] Step 6: If the current network parameter update of the critic is greater than the total number of training iterations, then the training round ends and the round update count is incremented by 1; if the round update count has not reached the specified total number of round updates, then continue to step 3 and proceed to the next round of training; otherwise, proceed to step 7.

[0030] Step 7: Input the calculated unloading system status into the actor's current network in the TD3-SMT network. The actor's current network is obtained according to the policy function. The value makes an action decision, obtains the calculated unloading system action, and the task. of (1- The part is local computation, task Partially delayed uninstallation, among which ∈[0,1].

[0031] In this invention, firstly, the vehicle preprocesses its own generated tasks. It calculates the attributes of the unloading task, including the amount of unloaded data, required CPU computing resources, and latency constraints, forming task unloading information. When a vehicle generates a task, it uploads the task's unloading information to the SDN controller via RSU (Remote Scheduler), which can be used to make unloading decisions. After receiving the task's unloading information, the SDN controller processes the information, and the processed result becomes the input to the computational unloading algorithm proposed in this invention. The output of this algorithm is the unloading decision result for the unloading task. Then, the SDN controller sends the scheduling scheme to the server and sends the unloading decision result to the corresponding vehicle via RSU. Finally, based on the unloading strategy, the vehicle decides whether to unload the task. If task unloading is selected, the task is sent to the server via RSU. Then, the server allocates computing resources to the task and returns the computation result to the corresponding vehicle.

[0032] To improve the quality of user service, this invention considers using vehicles with low computing requirements but high computing power as edge devices to provide services to user vehicles. This makes full use of idle computing resources and greatly reduces the burden and cost of data communication and computing for base stations and roadside units.

[0033] This invention addresses application tasks in vehicular network (V2V) MEC systems. To minimize latency during the offloading process, it formulates the problem as a Markov decision process by considering the serving vehicle set, base station, and RSU states. A latency-minimizing TD3-SMT task offloading method is then employed to obtain the optimal offloading scheme, providing offloading services with minimal latency. Simulation results demonstrate that the proposed scheme reduces offloading latency costs; the TD3-SMT scheme reduces latency by 10% compared to the traditional Q-learning scheme. The most practical aspect of this method is its effective utilization of all application computation cycles, starting from the actual computation cycle of the V2V MEC system, thus saving computational costs and significantly improving the application efficiency of V2V MEC systems. Attached Figure Description

[0034] Figure 1 This is a model framework diagram of a vehicle-to-everything (V2X) system;

[0035] Figure 2 Task uninstallation method flowchart;

[0036] Figure 3 TD3-SMT algorithm framework diagram;

[0037] Figure 4 Comparison of experimental results. Detailed Implementation

[0038] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, the vehicle-to-everything (V2X) system of this invention consists of vehicles, base stations, roadside units, and their associated MEC server equipment. In the V2X environment, each vehicle has computationally intensive tasks that need to be computed. Due to the limitations of vehicle resources, it may not be able to meet the computational demands of these computationally intensive tasks on its own. Vehicles can migrate tasks to MEC servers to reduce computational latency. Compared with traditional cloud computing, MEC servers are closer to the user, which can reduce communication costs between the EU (Enterprise Controller) and the MEC server. However, MEC servers have relatively low computing and storage capabilities, and may not be able to meet all tasks simultaneously.

[0040] First, the vehicle preprocesses its own generated tasks. It calculates the attributes of the unloading task, including the amount of unloaded data, required CPU computing resources, and latency constraints, forming the task unloading information. When a vehicle generates a task, it uploads the task's unloading information to the SDN controller via RSU (Remote Submission Unit), which can be used to make unloading decisions. After receiving the task's unloading information, the SDN controller processes the information, and the processed result is the input to the computational unloading algorithm proposed in this invention. The output of this algorithm is the unloading decision result for the unloading task. Then, the SDN controller sends the scheduling scheme to the server and sends the unloading decision result to the corresponding vehicle via RSU. Finally, the vehicle decides whether to unload the task based on the unloading policy. If task unloading is selected, the task is sent to the server via RSU. Then, the server allocates computing resources to the task and returns the computation result to the corresponding vehicle.

[0041] The following is combined Figure 2 The task unloading method of this invention is described in detail. First, it is necessary to model the task processing on the user side and the server side, and initialize the calculation unloading system state.

[0042] Each vehicle terminal device (EU) has a computationally intensive task to complete, and each EU can offload this task to an edge server or execute it locally. Assume that each EU has the same task priority. When tasks are offloaded simultaneously, wireless bandwidth resources are evenly distributed across each EU to upload their tasks. The upload data rate of the nth EU (EUn) is given by the following formula.

[0043] ,

[0044] Where W is the bandwidth of the wireless channel, K is the number of EUs to be offloaded, pn is the transmission power of the data uploaded by the EU, and hn is the wireless channel gain between the EU and the base station. The variance is the variance of the complex Gaussian white noise.

[0045] Construct a local computing model and an offloading model to obtain the total latency of local computing, the total latency of partially offloading computing, and the total time cost of fully offloading computing.

[0046] Will ( , , This is defined as a task that EUN needs to execute. This task can be computed locally or offloaded to the MEC server. This indicates the amount of data required to complete the task, including some data that needs to be processed, such as images from in-vehicle equipment, sensors, cameras, videos from Advanced Driver-Assistance Systems (ADAS), and the code and models that process this data. Indicates completion of task The number of CPU cycles required. Typically, and The size is positively correlated with the calculation of unloading. Its size will not change. It is a task The maximum tolerable delay, Indicates completion of task The time should not exceed The three parameters may differ for each EU task and can be estimated using the relevant configuration file in the application.

[0047] The EU's workload can be offloaded at any rate. This is what distinguishes the model of this invention from other binary computation offloading schemes. Therefore, partial computation offloading also places higher demands on the corresponding algorithms. ∈[0,1] as The computational offloading strategy means that EUN will Part of the task is offloaded to the MEC server, and (1- The task is computed locally. The computation offloading strategy A of EU set N consists of all EUs, defined by A=[ , ,..., ].when When = 0, this means the i-th task It is calculated locally. When When =1, this means Completely offload to the MEC server. When 0 < When <1, this means Tasks that are offloaded to the MEC server Part, another part of the task (1- The part is calculated locally.

[0048] When the EU executes a task locally, the task completion process is independent of the MEC server, utilizing local computing resources entirely. (Definition) To indicate the task Local computation latency. The calculation method is given by the following formula.

[0049] =

[0050] in This represents the computing power of the EU (EU in CPU cycles per second). For each EU, Possibly different. When the EU requests computational offloading, the task... of Some were unloaded to the MEC server. (Task) of The total latency includes data upload latency, MEC computation latency, and data download latency. (Task) of (1- The part is computed locally, task (1- The latency of this part is similar to the local computation latency described in the local computation model. Based on the above description, the following is defined: To indicate the task of The first step in mitigating data upload delays when partially offloading to the MEC server. The calculation method is given by the following formula:

[0051] = ,

[0052] Will Define the task of The partial unloading to the MEC server indicates a delay in the second-step MEC computation. The calculation method is given by the following formula:

[0053] = ,

[0054] in, This indicates that the task is assigned by MEC. Computing resources.

[0055] Let F be defined as the computing power of the MEC server (CPU cycles per second). The computing resources allocated to all offloaded tasks by the MEC server should not exceed the total computing resources, which means that the following condition should be met. ≤F.

[0056] definition To indicate when the task is of The third step in reducing data download delays when partially unloading to the MEC server. The calculation method is given by the following formula:

[0057] = ,

[0058] in This indicates the size of the data returned by the MEC server in the execution result. This represents the download rate at which EUn downloads the execution results from the MEC server. The intelligent latency hardware-based task offloading and download rates are much higher than the upload rates. Therefore, data download latency is not considered.

[0059] Due to the mission of (1- The part is computed locally, so the process does not require consideration of the MEC server. Therefore, it can be obtained that... of (1- The computational delay of this part is similar to that of the local computation model. (Definition) To indicate the task of (1- The computational delay of this part. The calculation method is given by the following formula:

[0060] = ,

[0061] Based on the above formula, define Tasks for executing partial computational offloading strategies Total delay, The calculation method is given by the following formula:

[0062] =max{( + ), },

[0063] Due to the mission Due to the parallelism, the total time delay of the EUN execution part's computational offloading strategy should be the task's... of Partial delays and tasks of (1- The larger of the delays in the part is max, which is the function that takes the maximum value.

[0064] The optimization objective is to determine the total latency of vehicle-to-everything (V2X) computational unloading; therefore, the state is defined as follows:

[0065] S= ,

[0066] That is, the latency of the entire vehicle-to-everything (V2X) MEC system; the action space of EUn is Therefore, calculate the unloading vector A=[ , ,..., Set N satisfies Each EU's action space is a continuous domain, and tasks can be offloaded at any rate. This differs from the binary offloading schemes of other traditional DRL algorithms. This algorithm expands the action space of traditional DRL and, to some extent, improves the possibility of optimizing the total latency of the vehicular MEC system. To evaluate the advantages and disadvantages of the computational offloading strategy, the reward function should be inversely proportional to the latency of all EUs in the vehicular MEC system. Therefore, a reward function is defined.

[0067] R(S,A)=- ,

[0068] This means that the better the offloading strategy, the lower the time latency of the MEC system, and therefore the greater the reward. The computational offloading system state is initialized, the critic's current network parameter update count is set to 0, and the computational offloading system state S is input into the TD3-SMT network to obtain the computational offloading system action decision, thus constructing training samples.

[0069] Based on the above description, the latency problem of user terminals can be described as problem P1:

[0070] (1)

[0071] N(2)

[0072] 0 F N(3)

[0073] F(4)

[0074] N(5)

[0075] Condition (2) represents each task It can be computed locally, completely offloaded to the MEC server, or offloaded in any proportion.

[0076] Condition (3) means that for each task unloaded to the MEC server, the computing resources allocated to the MEC server are positive and should not exceed the total computing resources of the MEC server.

[0077] Condition (4) means that the total computing resources allocated by the MEC server to all unloading tasks should not exceed the total computing resources of the MEC server.

[0078] Condition (5) represents the completion time of each task. It should not exceed Otherwise, it will affect the quality of the user experience.

[0079] In condition (1), if The value can only be 0 or 1. Analysis shows that as the number of users increases, the time complexity of finding the optimal solution increases exponentially.

[0080] State Space: To apply the DRL solution to the system model and problem framework, a state space, action space, and reward function need to be defined. Theoretically, the entire MEC system needs to be observed to determine the state space, including the number of users, task status, and MEC server computing resource usage. However, in practice, this would incur additional system overhead. This overhead increases with the number of EUs around the MEC server. The optimization objective is the total latency of the vehicular MEC system; therefore, the state is defined as S = ... This refers to the latency of the entire vehicle-to-everything (V2X) MEC system.

[0081] Action Space: To describe the computation offloading strategy for each EU, the action space is defined as the sum of the action spaces of all EUs in the MEC system. The action space of EUn is... Therefore, calculate the unloading vector A=[ , ,..., Set N satisfies It is worth noting that the action space of each EU is a contiguous domain, and tasks can be offloaded at any rate. This differs from the binary offloading schemes of other traditional DRL algorithms. This algorithm expands the action space of traditional DRL and, to some extent, improves the possibility of total latency optimization in vehicular MEC systems.

[0082] Reward Function: The reward function is represented by R(S,A), which means that the agent receives a reward when executing computation offloading strategy A in state S. This invention considers the latency of all EUs in the vehicular MEC system. To evaluate the advantages and disadvantages of the computation offloading strategy, the reward function should be inversely proportional to the latency of all EUs in the vehicular MEC system. Therefore, the reward function R(S,A) = - This means that the better the offloading strategy, the lower the time latency of the MEC system, and therefore the greater the reward. Here, we construct the three key elements required for the state space, action space, and reward function.

[0083] like Figure 3 As shown, the latency-minimizing task offloading method based on TD3-SMT includes one pair of actor networks and two pairs of critic networks. The actor network pair includes one current actor network and one target actor network. The critic network pair includes one current critic network and one target critic network. The target actor network and the current actor network have the same structure, as do the target critic network and the current critic network. First, the total number of training iterations and the number of round updates are set to zero. Then, the computational offloading system state is initialized, the number of updates for the current critic network is set to zero, and training samples are constructed. The system state is input into the current actor network, which obtains the current policy through the environment, adds noise, and obtains the current action. This action is applied to the interactive environment to obtain the computational offloading system state and reward value for the next time slot. The current time slot action, system state, reward value, and the next time slot state are packaged into a sample and stored in the experience pool. If the experience pool is not full, the interaction with the environment continues to generate samples and store them in the experience pool. If the experience pool is full, the TD3 network is trained until the specified total number of training iterations is reached, at which point the number of round updates is incremented by one. If the number of rounds for updating has not reached the specified total number of rounds for updating, the system state is reinitialized and the above steps are repeated for the next round of updating. If the number of rounds for updating has reached the specified total number of rounds for updating, the training process is stopped, and the trained actor network is saved. The target Q value is calculated using Softmax. During network training, the critic's current network uses a mean squared error function (MSE) and updates its parameters through backpropagation of the neural network's gradient. The MSE of the critic's current network is calculated based on the actual gain of the critic's target network and the squared error of the state-action value of the critic's current network in the current time slot. The actual gain of the critic's target network consists of the reward Rt in the current time slot and the target value R(S,A) of the state-action pair output by the critic's target network for the next time slot. The critic's current network is then updated multiple times before updating the actor's current network. The actor's current network updates its parameters using the loss gradient formula of this invention. Finally, both the actor's target network and the critic's target network perform soft updates to themselves.

[0084] Specifically, each training iteration draws M samples from the experience pool to train the TD3-SMT network, updating the critic's current network parameters; after a specified number of updates to the critic's current network parameters, the actor's current network parameters are adjusted using the loss gradient.

[0085]

[0086] Update, and use soft update to update the actor target network parameters.

[0087] Softmax normalization is added to the traditional double-delay deep deterministic policy gradient algorithm to reduce the number of times local optima are obtained and to decrease the algorithm's sensitivity to different initialization parameters. Importance sampling can be used to obtain... The specific formula is as follows:

[0088]

[0089] in: Let Q be the minimum value obtained from the two Critic networks. Let be the probability density function of a Gaussian distribution, and β be the parameter in the Softmax operation. This is used to solve for the objective Q value. Join at time :

[0090]

[0091] If the current network parameter update by the critic is greater than the total number of training iterations, then the training round ends and the round update count is incremented by 1; if the round update count does not reach the specified total number of round updates, then the next round of training continues.

[0092] If the number of rounds to be updated reaches the specified number of rounds, the calculated unload system state will be input into the actor's current network in the TD3-SMT network. The actor's current network will then be determined based on the policy function. The value makes an action decision, obtains the calculated unloading system action, and the task. of (1- The part is local computation, task Partially delayed uninstallation, among which ∈[0,1].

[0093] This invention addresses application tasks in vehicle-to-everything (V2X) MEC systems. To minimize latency during the offloading process, it plans the problem as a Markov decision process by considering the serving vehicle set and the states of base stations and RSUs. It then utilizes a latency-minimizing TD3-SMT task offloading method in V2X to obtain the optimal offloading scheme, thereby providing offloading services with minimal latency. Figure 4 As shown in the simulation results, the proposed solution can reduce offload latency costs. The TD3-SMT solution reduces latency by 10% compared to the traditional Q-learning solution. The most practical aspect of this method is that it starts from the actual computing cycle of the vehicle-to-everything (V2X) MEC system and effectively utilizes all the computing cycles of the application, thus saving computing costs and playing an important role in improving the application efficiency of V2X MEC systems.

Claims

1. A method for minimizing latency in TD3-SMT task offloading in vehicle-to-everything (V2X) communication, characterized in that, Includes the following steps: Step 1: Constructing a System Model Framework. Establish a communication scenario consisting of vehicles, base stations, roadside units, and their associated MEC server equipment. Each vehicle terminal device has a computationally intensive task to complete, and each vehicle terminal device can offload the task to an edge server or execute it locally. Multiple vehicle terminal devices that simultaneously offload tasks in the system will equally share the wireless bandwidth for uploading data. The upload data rate of the nth vehicle terminal device (EUn) is given by the following formula. , Where W is the bandwidth of the wireless channel, and K is the number of EUs to be offloaded. The transmission power for uploading data to EUN. This refers to the wireless channel gain between EUn and the base station. The variance of the complex Gaussian white noise; Step 2: Construct the local computing model and the offloading model to obtain the total latency of local computing, the total latency of partially offloading computing, and the total time cost of fully offloading computing; When executing tasks locally, the local computing model is as follows: ， in This represents the computational power of EUN. To indicate the task The latency of local computation, for each EU, They can be different; Will ∈[0,1] represents the computational unloading decision of the i-th vehicle terminal device, and the computational unloading strategy A of the EU set N is defined by the unloading decision vector, consisting of all EUs, and is defined by A=[ , ,..., ],when When = 0, this means the i-th task It is calculated locally, when When =1, this means Completely unload to the MEC server when 0 < When <1, this means Tasks that are offloaded to the MEC server Part, another part of the task (1- Part of it is calculated locally; When an EU requests computational unloading during task execution on an edge server, the task... of Some tasks were offloaded to edge servers. of The total latency includes data upload latency, MEC computation latency, and data download latency for the task. of (1- The part is computed locally, task (1- The latency of the ) part is similar to the local computation latency described in the local computation model; based on the above description, the following is defined: To indicate the task of The first step in addressing data upload delays during partial offloading to the MEC server. The calculation method is given by the following formula: = ， Will Define the task of Partial offloading to the MEC server indicates a delay in the second-step MEC computation. The calculation method is given by the following formula: = ， in This indicates that the task is assigned by MEC. Computing resources; definition To indicate the task of (1- The computational delay of this part The calculation method is given by the following formula: = , Based on the above formula, define Tasks for executing partial computational offloading strategies Total delay, The calculation method is given by the following formula: =max{( + ), }, Due to the mission Due to the parallelism of the EUn execution part, the total time delay of the computation offloading strategy is the task's... of Partial delays and tasks of (1- The larger of the delays in the ) part, max is the function that takes the maximum value; Step 3: The optimization objective is to calculate the total latency of vehicle-to-everything (V2X) unloading; therefore, the state is defined as follows: S= ， That is, the latency of the entire vehicle-to-everything (V2X) MEC system; EUn's action space is Calculate the unloading vector A=[ , ,..., Set N satisfies The action space of each EU is a continuous domain, and tasks can be unloaded at any rate. The reward function is inversely proportional to the latency of all EUs in the vehicle-to-everything (V2X) MEC system, and is defined as follows: R(S,A)=- , Initialize the computational unloading system state, set the critic's current network parameter update count to 0, input the computational unloading system state S into the TD3-SMT network to obtain the computational unloading system action decision, and construct training samples; In step 2, the latency of the user terminal is described as problem P1: min (1) N(2) 0 F N(3) F(4) N(5) Wherein, condition (2) represents each task It can be computed locally, completely offloaded to the MEC server, or offloaded in any proportion. Condition (3) means that for each task unloaded to the MEC server, the computing resources allocated to the MEC server are positive and should not exceed the total computing resources of the MEC server; Condition (4) means that the total computing resources allocated by the MEC server to all unloading tasks should not exceed the total computing resources of the MEC server. Condition (5) represents the completion time of each task. It should not exceed Otherwise, it will affect the quality of the user experience; Step 4: Each training session draws M samples from the experience pool to train the TD3-SMT network and updates the critic's current network parameters; after the critic's current network parameters are updated a specified number of times, the actor's current network parameters are updated through loss gradient and the actor's target network parameters are updated using a soft update method. Step 5: Add Softmax normalization to the gradient of the double-delay deep deterministic policy to reduce the number of times local optima are obtained and reduce the sensitivity of the algorithm to different initialization parameters; Step 6: If the current network parameter update of the critic is greater than the total number of training iterations, then the training round ends and the round update count is incremented by 1; if the round update count has not reached the specified total number of round updates, then continue to step 3 and proceed to the next round of training; otherwise, proceed to step 7. Step 7: Input the calculated unloading system status into the actor's current network in the TD3-SMT network. The actor's current network is obtained according to the policy function. The value makes an action decision, obtains the calculated unloading system action, and the task. of (1- The part is local computation, task Partially delayed uninstallation, among which ∈[0,1].

2. The task unloading method according to claim 1, characterized in that, In step 1, ( , , ) is defined as the task that EUN needs to execute; this task can be computed locally, or it can be computed on the MEC server by offloading the computation. This indicates the amount of data required to complete this task, including images from in-vehicle devices, sensors, cameras, video from advanced driver assistance systems, and the code and models that process this data; Indicates completion of task Required number of CPU cycles; It is a task The maximum tolerable delay, Indicates completion of task The time should not exceed The three parameters may differ for each EU task and can be estimated using the relevant configuration file in the application.

3. The task unloading method according to claim 1, characterized in that, In step 4, the loss gradient is: 。 4. The task unloading method according to claim 1, characterized in that, In step 5, importance sampling is used to obtain... The specific formula is as follows: ， in: Let Q be the minimum value obtained from the two Critic networks. Let be the probability density function of a Gaussian distribution, and β be the parameter in the Softmax operation.

5. The task unloading method according to claim 4, characterized in that, In step 5, the target Q value is calculated. Join at time : ）。 6. The task unloading method according to claim 1, characterized in that, In step 6, the latency minimization task offloading method based on TD3-SMT includes one pair of actor networks and two pairs of critic networks; one pair of actor networks includes one actor current network and one actor target network; one pair of critic networks includes one critic current network and one critic target network; the actor target network and the actor current network have the same structure, and the critic target network and the critic current network have the same structure.

7. The task unloading method according to claim 6, characterized in that, First, set the total number of training iterations to zero and the number of rounds to zero; then initialize the computational unloading system state, set the current network update count for the critic to zero, and construct training samples; The system state is input into the actor's current network. The actor's current network obtains the current policy through the environment, adds noise to obtain the current action, and applies this action to the interactive environment to obtain the computational unloading system state and reward value for the next time slot. The current time slot action, system state, reward value, and next time slot state are packaged into a sample and stored in the experience pool. If the experience pool is not full, the interaction with the environment continues to generate samples and store them in the experience pool. If the experience pool is full, the TD3 network is trained until the specified total number of training iterations is reached, and the round update count is incremented by one. If the round update count has not reached the specified total number of round updates, the system state is reinitialized and the above steps are repeated for the next round update. If the round update count reaches the specified total number of round updates, the training process stops, and the trained actor's current network is saved.

8. The task unloading method according to claim 7, characterized in that, Calculate the target Q value using Softmax. During network training, the critic's current network uses a mean squared error function, updating its parameters through backpropagation of the neural network's gradient. The mean squared error function of the critic's current network is calculated based on the actual gain of the critic's target network and the squared error of the state-action value of the critic's current network in the current time slot. The actual gain of the critic's target network consists of the reward Rt of the current time slot and the target value R(S,A) of the state-action pair output by the critic's target network for the next time slot. After that, the critic's current network updates multiple times before updating the actor's current network. The actor's current network updates its parameters using the loss gradient formula. Finally, the actor's target network and the critic's target network perform soft updates on themselves.

Images