Deep learning method for vehicle calculation unloading

Abstract

The invention discloses a deep learning method for vehicle edge calculation unloading, and the method employs an improved Transform feature extraction model, and introduces an Encoder module to enhance the mining of local dependence in vehicle feature data, further strengthens the expression capability and generalization capability of a network, and improves the accuracy and stability of an unloading decision. By simplifying the network, the algorithm complexity of the model is reduced, and the calculation efficiency of the model is improved. Besides, the method is also combined with a VE-Kmeans algorithm and an MADDPG network model for training, so that the calculation speed of the model is greatly improved and the memory consumption is reduced under the condition that the accuracy is ensured, and the practicability and the flexibility of vehicle calculation unloading are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle computation offloading, and in particular to a deep learning method for vehicle computation offloading. Background Art

[0002] In the field of vehicle computational offloading, most current research adopts methods based on traditional algorithms and empirical models. These methods usually rely on static rules and historical data and are difficult to adapt to dynamic changes in the vehicle environment. Specifically, rule-based offloading decisions rely on preset decision rules and features for task offloading, mainly relying on static data and empirical judgment. This makes it difficult for these systems to maintain efficient decision-making capabilities when facing complex and rapidly changing environments. For example, rule-based methods usually require a complete set of offloading strategies to be defined in advance and rely on historical data to generate decisions. The limitation of this process is that the real-time dynamics and diverse computing needs of vehicles make these methods challenging to deal with unknown situations. In addition, vehicles show high flexibility during dynamic operation, which may lead to reduced efficiency of offloading decisions and increased task loss rate. Traditional algorithms also have difficulties in dealing with high-dimensional state spaces and complex tasks, especially in the scenario of vehicle computational offloading, where traditional algorithms have difficulty understanding and adapting to changes in the environment. Rule-based systems may face high task loss rates and have difficulty dealing with the challenges brought about by environmental changes, resulting in reduced efficiency of offloading decisions. Therefore, traditional rule-based offloading methods have significant limitations in today's changing computing environment. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of existing vehicle calculation unloading methods and provide an unloading decision method based on deep learning. The unloading decision method can better adapt to the dynamics and complexity of the vehicle environment and has the advantages of high efficiency and flexibility.

[0004] The object of the present invention is achieved by the following technical solution: The deep learning method for calculating unloading of a vehicle comprises the following steps:

[0005] S1,vehicles are clustered using the VE-Kmeans algorithm in the unloading environment to form a dynamic database;

[0006] S2. Build a reinforcement learning agent, which includes a Transformer feature extraction model and a MADDPG network model;

[0007] S3, building a Transformer feature extraction model, and using this Transformer feature extraction model to extract state features from the dynamic database for processing;

[0008] S4. Based on the experience replay mechanism and random sampling of historical decisions, the Critic network in the MADDPG network model evaluates the value of the state-action pair and updates the parameters of the Actor network in the MADDPG network model through policy gradient to achieve continuous optimization of the MADDPG network model. The optimized Transformer feature extraction model processes the state features and the result is used as the input of the optimized MADDPG network model to obtain the unloading decision.

[0009] S5. The offloading decision obtained in S4 is updated in real time according to the continuously optimized MADDPG network model and deployed to the vehicle edge computing system. The vehicle edge computing system will make offloading decisions on the vehicle dynamic task data in the vehicle edge computing environment and record the relevant results to the RSU database.

[0010] Preferably, step S1 includes the following specific steps:

[0011] S11. Select RSU as the cluster center and calculate the I value between each vehicle and the cluster center:

[0012]

[0013] Among them, dis max represents the maximum communication distance, w∈(1,2) is the slack variable, Θ is a sufficiently large number, syn(x) is the sign function, and I is the complement of the Euclidean distance in Kmeans clustering;

[0014] S12, based on I and relaxation distance dis max w clusters each vehicle:

[0015] When the vehicle is ST i Drive to the base station RSU k When , I>0, it means that vehicles with farther distance can be clustered;

[0016] When the vehicle is ST i Departure RSU k When , I < 0, it means that vehicles with closer distances can be clustered;

[0017] When the vehicle is ST i Heading to RSU k But when the maximum communication distance is exceeded, a relaxation distance dis is set. max w, at this time the vehicle ST i It can still be used as a clusterable object of RSU;

[0018] When the vehicle is ST i When outside the slack distance, the vehicle will be considered an RSU k Non-clusterable objects;

[0019] S13. Collect data of clustered vehicles to form a dynamic data set.

[0020] Preferably, the Transformerr feature extraction model constructed in step S3 includes an embedding layer, an Encoder module containing a vehicle feature extraction self-attention layer, and a Linear Layer linear layer connected in sequence.

[0021] Embedding layer, used to convert the input state features into an embedding vector;

[0022] Vehicle features are extracted from the attention layer to capture long-range dependencies in vehicle features;

[0023] The Encoder module extracts key features from vehicle status and mission information, enhances the global feature extraction capability of sequence data, and maintains the stability of the training process and stronger generalization capability through layer normalization and residual connection to map high-dimensional features to the target dimension;

[0024] The Linear Layer maps the high-dimensional features output by the Encoder to the target dimension for calculating specific task offloading decisions.

[0025] Preferably, the process of processing the input state features using the constructed Transformer feature extraction model in step S3 includes the following steps:

[0026] 3a. Let the key vectors in the Transformer feature extraction model be Query, Key, and Value. Query first undergoes a linear transformation of the self-attention layer in the attention mechanism and performs a convolution operation to obtain:

[0027] Query = w Q Emb,Key=w k Emb,Value=w v Emb; where Emb is the embedding vector output by the embedding layer of the neural network, and Query, Key, and Value are all key vectors of the model;

[0028] S3b, multiply the convolution result by Value and divide it by its square sum to normalize and get the attention weight:

[0029]

[0030] Where d is the dimension of the query, which is used to adjust the scale of the result; T represents vector transposition;

[0031] S3c, combined with the weight parameter and bias ∈ (0, 1), calculate the attention score:

[0032]

[0033] S3d, finally the attention score is processed by the linear layer and then normalized to obtain the output result of the vehicle feature vector:

[0034] Among them, LayerNorm means normalization, MultiHead means that multiple Attentions are used to form a multi-head self-attention mechanism to calculate the attention score, and FeedForward means a feedforward neural network.

[0035] Preferably, the optimization of the MADDPG optimization process in step S4 includes the following steps:

[0036] S41, initializing the MADDPG network model according to the dimension parameters of the MADDPG network model;

[0037] S42, MADDPG network model follows the ∈-greedy strategy:

[0038]

[0039] Where ρ conforms to the random distribution, random represents the generation of random unloading decisions, ∈∈(0, 1), the initial ∈ is set to a number close to 1, and each iteration ∈ thereafter obeys exponential decay:

[0040] ∈ e+1 =∈ e δ, 0<δ<1, e>0,

[0041] When ∈<ρ<1, the state It is input into the Actor network of the MADDP network model and outputs the action vector

[0042] S43, the vehicle calculation unloading Transformer feature extraction model and the MADDPG network model train and learn the input vehicle observation state vector, wherein the loss function of the Transformer feature extraction model is formed by the loss function of the MADDPG network model through back propagation;

[0043] S44. The loss function in the MADDPG network model consists of two parts: the loss value between the output of the Actor network and the actual unloading decision, and the mean square error loss between the Q value of the current state and action and the target Q value of the Critic network;

[0044] S45. Execute action And get rewards After that, the system will Actions taken Rewards and the next state Stored as transition in the experience replay pool; the experience replay pool is used to randomly extract historical transitions, break time correlation, and reduce variance in training;

[0045] S46. The algorithm in the MADDPG network model uses a target network to stabilize the training process: the parameters of the TargetActor and Target Critic networks are updated by soft updating, namely:

[0046] (θ|target)=τθ+(1-τ)(θ|target)

[0047] Where (θ|target) is the parameter of the target network, θ is the parameter of the current network, and τ is the update coefficient, which usually takes a value between 0 and 1.

[0048] Preferably, when initializing the MADDPG network model in step S41, the initial parameters are set, the training batch size is set to 16, the number of training times is set to 800, the Actor and Critic learning rates are 0.000001, the optimizer uses the Adam optimizer, and the loss function uses the mean square error loss function.

[0049] Preferably, before building the Transformer feature extraction model, the data set in the dynamic database is preprocessed to divide the data set into a training set, a validation set, and a test set, thereby obtaining training samples.

[0050] The present invention has the following advantages over the prior art:

[0051] 1. The deep learning method for vehicle computing offloading of the present invention adopts a model based on the combination of the MADDPG algorithm and Transformer, and combines the VE-Kmeans clustering method to optimize the state capture of the vehicle agent. This model can effectively extract implicit features in complex environments, optimize through deep reinforcement learning, and flexibly adjust to adapt to different vehicle computing scenarios. It has strong portability, does not need to rely on historical data guidance of specific scenarios, and more efficiently identifies the best solution in computing offloading decisions.

[0052] 2. The deep learning model in the present invention is optimized through joint training, which can significantly improve the unloading efficiency while ensuring the accuracy of decision-making. The application of VE-Kmeans enables the vehicle agent to identify task clusters more quickly, thereby improving the overall response speed. In addition, this method better adapts to the rapidly changing vehicle computing environment and improves the overall reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is an overall flow chart of the deep learning method for vehicle edge computing offloading of the present invention.

[0054] Figure 2 It is a structural diagram of a neural network model of a deep learning method for vehicle edge computing offloading of the present invention.

[0055] Figure 3 It is a structural diagram of the vehicle feature extraction from the attention layer of the present invention.

[0056] Figure 4 It is a schematic diagram of the process of VE-Kmeans clustering of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0058] like Figure 1 As shown, the deep learning method for calculating unloading of a vehicle is characterized by comprising the following steps:

[0059] S1, the vehicles are clustered in the unloading environment using the VE-Kmeans algorithm to form a dynamic database; Figure 4 As shown, step S1 includes the following specific steps:

[0060] S11. Select RSU as the cluster center and calculate the I value between each vehicle and the cluster center:

[0061]

[0062] Among them, dis max represents the maximum communication distance, w∈(1,2) is the slack variable, Θ is a sufficiently large number, syn(x) is the sign function, and I is the complement of the Euclidean distance in Kmeans clustering;

[0063] S12, based on I and relaxation distance dis max w clusters each vehicle:

[0064] When the vehicle is ST i Drive to the base station RSU k When I>0, it means that vehicles with farther distances can be clustered more easily;

[0065] When the vehicle is ST i Departure RSU k When , I < 0, it means that vehicles with closer distances can be clustered;

[0066] When the vehicle is ST i Heading to RSUk But when the maximum communication distance is exceeded, a relaxation distance dis is set. max w, at this time the vehicle ST i It can still be used as a clusterable object of RSU;

[0067] When the vehicle is ST i When outside the slack distance, the vehicle will be considered an RSU k Non-clusterable objects;

[0068] S13. Collect the data of clustered vehicles to form a dynamic data set. Specifically, the dynamic data set contains samples from multiple vehicles. In this embodiment, there are about 10,000 samples in total. The vehicle data includes speed, position, task attributes, vehicle computing power, etc. The dynamic data set also contains label types, including "unloading success", "unloading failure", etc., and the labels are vectorized for subsequent model training. The data of the clustered vehicles uses sensors and communication protocols to transmit vehicle status information and network condition data to the central server in real time. In the central server, Python program is used in conjunction with numpy and pandas libraries for data processing and storage.

[0069] S2. Build a reinforcement learning agent, which includes a Transformer feature extraction model and a MADDPG network model;

[0070] When building a reinforcement learning agent, the data set in the dynamic database is first preprocessed to divide the data set into a training set, a validation set, and a test set to obtain training samples. Specifically, the discrete features in the data set are one-hot encoded, and the continuous features are standardized. That is, the Pytorch library tool is used to vectorize the data, and finally each sample data is converted into a vector of size (batch_size, seq_length, input_size). In the present invention, batch_size is 16, seq_length is 1, and input_size is the corresponding vehicle feature dimension, so the data vector size of a batch is (16, 1, feature dimension).

[0071] S3, building a Transformer feature extraction model, and using this Transformer feature extraction model to extract state features from the dynamic database for processing;

[0072] Specifically, such as Figure 2 and Figure 3As shown, a Transformer feature extraction model is constructed. This neural network model includes an embedding layer connected in sequence, an Encoder module containing vehicle feature extraction from an attention layer, and a LinearLayer linear layer;

[0073] The embedding layer is used to convert the input state features into embedded vectors; that is, the embedding layer is used to convert the input state features, such as the discrete features and numerical features of the vehicle's state information, into embedded vectors so that the subsequent network can better capture the relationship and semantics between the features. It is equivalent to finding a suitable multi-dimensional space representation for each feature, improving the feature expression ability of the model.

[0074] The Encoder module includes a vehicle feature extraction self-attention layer, a first residual connection & a first normalization layer, a feedforward network layer, and a second residual connection & a second normalization layer connected in sequence; at the same time, the embedding layer inputs the Query value and the Value value into the vehicle feature extraction self-attention layer for feature extraction processing, and the embedding layer is also connected to the first residual connection & the first normalization layer; the first residual connection & the first normalization layer transmits the calculation results to the feedforward network layer and the second residual connection & the second normalization layer respectively, and the second residual connection & the second normalization layer is connected to the Linear Layer linear layer. This Encoder module extracts key features from vehicle status and task information, enhances the global feature extraction capability of sequence data, and maintains the stability of the training process and stronger generalization capability through layer normalization and residual connection, so as to obtain high-dimensional feature mapping to the target dimension.

[0075] The vehicle feature extraction self-attention layer includes a linear layer, a one-dimensional convolution layer, a transposition layer, a linear layer, a first multiplication module, an addition module, a linear layer, and a second multiplication module; the linear layer inputs the query values ​​into the fully connected neural network layer, and then inputs them into the first multiplication module for normalization operation, and then combines them with the addition module. The addition module inputs the operation result into the convolution layer to obtain a score, and the score is calculated through the Value value input by the linear layer through the second multiplication module to obtain the final vehicle feature result. The result is passed through the two fully connected layers of the Actor network to obtain the final vehicle unloading decision result. That is, the vehicle feature extraction self-attention layer calculates the mutual influence between feature vectors, namely the attention weight, to help the model focus on important feature information while ignoring irrelevant information to capture long-distance dependencies in vehicle features.

[0076] The Linear Layer maps the high-dimensional features output by the Encoder to the target dimension for calculating specific task offloading decisions. This Linear Layer can adjust the dimension of the features or be used as an output layer to generate the final offloading decision. The linear layer plays a key role in feature combination and output generation by performing linear transformation on the input features.

[0077] Specifically, the Transformer feature extraction model constructed by the present invention is optimized based on the native Transformer model architecture, and its structure is similar to that of the native Transformer. When the position encoding layer in the native Transformer model is removed, adjustments are made in the Encoder module. After the input data is calculated by the embedding layer, it is passed to the Encoder module for processing. The module contains an efficient vehicle feature extraction self-attention layer and a feedforward network layer. After the calculation of each self-attention layer and feedforward network layer is completed, they are all subjected to residual connection and normalization processing, aiming to alleviate the problem of gradient disappearance, so that the model can more effectively learn the mapping relationship between input and output, and enhance the ability of long-distance dependency and large-scale data training. After being processed by the Encoder module, the data will be converted to a size suitable for the output label to obtain a vehicle feature vector. In addition, the decoder module of the traditional Transformer model is replaced with two fully connected layers to directly generate the vehicle unloading decision results, further reducing the network complexity and improving the computational efficiency.

[0078] The process of processing the input state features using the constructed Transformer feature extraction model in step S3 includes the following steps:

[0079] 3a. Let the key vectors in the Transformer feature extraction model be Query, Key, and Value. Query first undergoes a linear transformation of the self-attention layer in the attention mechanism and performs a convolution operation to obtain:

[0080] Query = w Q Emb,Key=w k Emb,Value=w v Emb; where Emb is the embedding vector output by the embedding layer of the neural network, and Query, Key, and Value are all key vectors of the model;

[0081] S3b, multiply the convolution result by Value and divide it by its square sum to normalize and get the attention weight:

[0082]

[0083] Where d is the dimension of the query, which is used to adjust the scale of the result; T represents vector transposition;

[0084] S3c, combined with the weight parameter and bias ∈ (0, 1), calculate the attention score:

[0085]

[0086] S3d, finally the attention score is processed by the linear layer and then normalized to obtain the output result of the vehicle feature vector:

[0087] Among them, LayerNorm means normalization, MultiHead means that multiple Attentions are used to form a multi-head self-attention mechanism to calculate the attention score, and FeedForward means a feedforward neural network.

[0088] S4. Based on the experience replay mechanism and random sampling of historical decisions, the Critic network in the MADDPG network model evaluates the value of the state-action pair and updates the parameters of the Actor network in the MADDPG network model through policy gradient to achieve continuous optimization of the MADDPG network model. The Transformer feature extraction model processes the state features and the results are used as the input of the optimized MADDPG network model to obtain the unloading decision.

[0089] First, the data set in the dynamic database is preprocessed to divide the data set into a training set, a validation set and a test set to obtain training samples.

[0090] The optimization of the MADDPG optimization process in step S4 includes the following steps:

[0091] S41. Initialize the MADDPG network model according to the dimensional parameters of the MADDPG network model; when initializing the MADDPG network model in step S41, set the initial parameters, the training batch size is set to 16, the number of training times is set to 800, the Actor and Critic learning rates are 0.000001, the optimizer uses the Adam optimizer, and the loss function uses the mean square error loss function.

[0092] S42, MADDPG network model follows the ∈-greedy strategy:

[0093]

[0094] Where ρ conforms to the random distribution, random represents the generation of random unloading decisions, ∈∈(0, 1), the initial ∈ is set to a number close to 1, and each iteration ∈ thereafter obeys exponential decay:

[0095] ∈ e+1 =∈ e δ, 0<δ<1, e>0,

[0096] When ∈<ρ<1, the state It is input into the Actor network of the MADDP network model and outputs the action vector

[0097] S43, the vehicle calculation unloading Transformer feature extraction model and the MADDPG network model train and learn the input vehicle observation state vector, wherein the loss function of the Transformer feature extraction model is formed by the loss function of the MADDPG network model through back propagation;

[0098] S44. The loss function in the MADDPG network model consists of two parts: the loss value between the output of the Actor network and the actual unloading decision, and the mean square error loss between the Q value of the current state and action and the target Q value of the Critic network;

[0099] S45. Execute action And get rewards After that, the system will Actions taken Rewards and the next state Stored as transition in the experience replay pool; the experience replay pool is used to randomly extract historical transitions, break time correlation, and reduce variance in training;

[0100] S46. The algorithm in the MADDPG network model uses a target network to stabilize the training process: the parameters of the Target Actor and Target Critic networks are updated by soft updates, namely:

[0101] (θ|target)=τθ+(1-τ)(θ|target)

[0102] Where (θ|target) is the parameter of the target network, θ is the parameter of the current network, and τ is the update coefficient, which usually takes a value between 0 and 1. This update method ensures that the parameters of the target network change smoothly, helps to reduce the instability in the training process, makes the learning process more stable, and improves model performance.

[0103] Specifically, the Actor network outputs an unloading strategy based on the current state, and the Critic network evaluates the Actor's actions and calculates the corresponding Q value. In each training round, the Target Actor and Target Critic copy the parameters of the Actor and Critic respectively to maintain the stability of the target network. In each round of training, the output of the Actor network is processed by the fully connected layer to generate an unloading decision, and the Critic network receives the current state and the Actor's output as input, and calculates the Q value through a series of layers; these values ​​are compared with the target Q value to optimize the model performance.

[0104] The present invention adopts the methods of VE-Kmeans, Transformer and MADDPG to perform training and reasoning of vehicle computational unloading. The method clusters vehicles through the VE-Kmeans algorithm and assigns tasks to vehicles in the cluster; then, the Transformer feature extraction model processes the vehicle features in the cluster to extract effective information. Finally, MADDPG makes decisions through the Actor and Critic network to optimize the vehicle unloading strategy. Through this combined method, the accuracy and speed of unloading decisions can be effectively improved, while reducing the system's computing resource consumption. The following is the algorithm flow pseudo code of the present invention:

[0105]

[0106]

[0107]

[0108]

[0109] S5. The unloading decision obtained in S4 is updated and deployed to the vehicle edge computing system in real time according to the continuously optimized MADDPG network model. The vehicle edge computing system will make unloading decisions on the vehicle dynamic task data in the vehicle edge computing environment and record the relevant results to the RSU database. Based on the decision results, the system can dynamically notify or control the vehicles and RSUs in the network to adopt corresponding unloading strategies. The unloading decision results will be classified into different categories, including binary unloading, partial unloading, and local execution.

[0110] The method of the present invention significantly enhances the model's ability to extract and analyze vehicle features by introducing the VE-Kmeans algorithm and Transformer structure into the model, further improving the accuracy and efficiency of unloading decisions. At the same time, by optimizing the deep learning framework, the computational complexity of the model is reduced and the operating efficiency is improved. The system of the present invention can effectively adapt to different network environments, realize flexible vehicle unloading decisions, and provide an efficient and practical solution for edge computing.

[0111] The above specific implementation modes are preferred embodiments of the present invention and cannot be used to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solution of the present invention are included in the protection scope of the present invention.

Claims

1. A deep learning method for vehicle computational offloading, characterized in that: The following steps are involved: S1,vehicles are clustered using the VE-Kmeans algorithm in the unloading environment to form a dynamic database; S2. Build a reinforcement learning agent, which includes a Transformer feature extraction model and a MADDPG network model; S3, building a Transformer feature extraction model, and using this Transformer feature extraction model to extract state features from the dynamic database for processing; S4. Based on the experience replay mechanism and random sampling of historical decisions, the Critic network in the MADDPG network model evaluates the value of the state-action pair and updates the parameters of the Actor network in the MADDPG network model through policy gradient to achieve continuous optimization of the MADDPG network model. The Transformer feature extraction model processes the state features and the results are used as the input of the optimized MADDPG network model to obtain the unloading decision. S5. The offloading decision obtained in S4 is updated in real time according to the continuously optimized MADDPG network model and deployed to the vehicle edge computing system. The vehicle edge computing system will make offloading decisions on the vehicle dynamic task data in the vehicle edge computing environment and record the relevant results to the RSU database.

2. The deep learning method for vehicle computation offloading according to claim 1, characterized in that: Step S1 includes the following specific steps: S11. Select RSU as the cluster center and calculate the I value between each vehicle and the cluster center: Among them, dis max represents the maximum communication distance, w∈(1,2) is the slack variable, Θ is a sufficiently large number, syn(x) is the sign function, and I is the complement of the Euclidean distance in Kmeans clustering; S12, based on I and relaxation distance dis max w clusters each vehicle: When the vehicle is ST i Drive to the base station RSU k When , I>0, it means that vehicles with farther distance can be clustered; When the vehicle is ST i Departure RSU k When , I < 0, it means that vehicles with closer distances can be clustered; When the vehicle is ST i Heading to RSU k But when the maximum communication distance is exceeded, a relaxation distance dis is set. max w, at this time the vehicle ST i It can still be used as a clusterable object of RSU; When the vehicle is ST i When outside the slack distance, the vehicle will be considered an RSU k Non-clusterable objects; S13. Collect data of clustered vehicles to form a dynamic data set.

3. The deep learning method for vehicle computation offloading according to claim 1, characterized in that: The Transformerr feature extraction model constructed in step S3 includes an embedding layer, an Encoder module containing vehicle feature extraction from the attention layer, and a Linear Layer. Embedding layer, used to convert the input state features into an embedding vector; Vehicle features are extracted from the attention layer to capture long-range dependencies in vehicle features; The Encoder module extracts key features from vehicle status and mission information, enhances the global feature extraction capability of sequence data, and maintains the stability of the training process and stronger generalization capability through layer normalization and residual connection to map high-dimensional features to the target dimension; The Linear Layer maps the high-dimensional features output by the Encoder to the target dimension for calculating specific task offloading decisions.

4. The deep learning method for vehicle computation offloading according to claim 3, characterized in that: The process of processing the input state features using the constructed Transformer feature extraction model in step S3 includes the following steps: 3a. Let the key vectors in the Transformer feature extraction model be Query, Key, and Value. Query first undergoes a linear transformation of the self-attention layer in the attention mechanism and performs a convolution operation to obtain: Query = w Q Emb,Key=w k Emb,Value=w v Emb; where Emb is the embedding vector output by the embedding layer of the neural network, and Query, Key, and Value are all key vectors of the model; S3b, multiply the convolution result by Value and divide it by its square sum to normalize and get the attention weight: Where d is the dimension of the query, which is used to adjust the scale of the result; T represents vector transposition; S3c, combined with the weight parameter and bias ∈ (0, 1), calculate the attention score: S3d, finally the attention score is processed by the linear layer and then normalized to obtain the output result of the vehicle feature vector: Among them, LayerNorm means normalization, MultiHead means that multiple Attentions are used to form a multi-head self-attention mechanism to calculate the attention score, and FeedForward means a feedforward neural network.

5. The deep learning method for vehicle computation offloading according to claim 1, characterized in that: The optimization of the MADDPG optimization process in step S4 includes the following steps: S41, initializing the MADDPG network model according to the dimensional parameters of the MADDPG network model; S42, MADDPG network model follows the ∈-greedy strategy: Where ρ conforms to the random distribution, random represents the generation of random unloading decisions, ∈∈(0, 1), the initial ∈ is set to a number close to 1, and each iteration ∈ thereafter obeys exponential decay: ∈ e+1 =∈ e δ, 0<δ<1, e>0, When ∈<ρ<1, the state It is input into the Actor network of the MADDP network model and outputs the action vector S43, the vehicle calculation unloading Transformer feature extraction model and the MADDPG network model train and learn the input vehicle observation state vector, wherein the loss function of the Transformer feature extraction model is formed by the loss function of the MADDPG network model through back propagation; S44. The loss function in the MADDPG network model consists of two parts: the loss value between the output of the Actor network and the actual unloading decision, and the mean square error loss between the Q value of the current state and action and the target Q value of the Critic network; S45. Execute action and get rewarded After that, the system will Actions taken Rewards and the next state Stored as transition in the experience replay pool; the experience replay pool is used to randomly extract historical transitions, break time correlation, and reduce variance in training; S46. The algorithm in the MADDPG network model uses a target network to stabilize the training process: the parameters of the Target Actor and Target Critic networks are updated by soft updates, namely: (θ|target)=τθ+(1-τ)(θ|target) Where (θ|target) is the parameter of the target network, θ is the parameter of the current network, and τ is the update coefficient, which usually takes a value between 0 and 1.

6. The deep learning method for vehicle computation offloading according to claim 5, characterized in that: When initializing the MADDPG network model in step S41, the initial parameters are set, the training batch size is set to 16, the number of training times is set to 800, the Actor and Critic learning rates are 0.000001, the optimizer uses the Adam optimizer, and the loss function uses the mean square error loss function.

7. The deep learning method for vehicle computation offloading according to claim 1, characterized in that: Before building the Transformer feature extraction model, the data set in the dynamic database is preprocessed to divide the data set into a training set, a validation set, and a test set to obtain training samples.

Images