Traffic Optimal Path Planning Method Based on Sparse Completion Collaborative Q Deep Learning

Abstract

The present invention provides a traffic optimal path planning method based on sparse completion and collaborative Q deep learning, which relates to the field of traffic path planning, including: a key area adaptive selection method based on weighted error control to sample and select sparse areas; a traffic map feature privacy protection method based on transformer to efficiently extract and protect traffic map features; a sparse traffic map feature matrix completion method based on KNN weighted averaging to complete missing parts; a Q deep learning adaptive gradient protection method for sparse traffic map pre-training to protect model parameters; a sparse traffic flow optimal path planning method based on Kalman filter to select the optimal path; the protection method adopted by the present invention obtains #imgabs0# local differential privacy, and realizes high-availability optimal path planning while ensuring the privacy protection of sparse traffic data.

Description

Technical Field

[0001] The present application relates to the technical field of traffic path planning, and in particular to a traffic optimal path planning method based on sparse completion collaborative Q deep learning. Background Art

[0002] As the global urban development process continues to accelerate, the challenges faced by transportation networks are becoming increasingly complex, especially in urban route planning. Route planning is a key issue in traffic management. In smart cities and intelligent transportation systems, how to achieve efficient and accurate route planning has become a focus of researchers and engineers. Traditional route planning methods such as Algorithms such as the Dijkstra algorithm often rely on graph search strategies, which have high computational complexity, especially when the search space is large or the map is very complex. As the number of nodes increases and traffic flow continues to change, the time overhead of path planning will increase significantly, which makes it difficult to meet actual needs in real-time or large-scale application scenarios.

[0003] The emergence of cloud-edge-device collaboration offers a new solution to the path planning problem. In this architecture, end devices monitor traffic conditions in real time through sensors and data acquisition equipment and upload the data to edge nodes. The edge nodes then use the collected data to train models and collaborate with cloud servers to optimize the path planning model. This cloud-edge-device collaborative path planning architecture offers significant advantages in improving traffic efficiency, reducing congestion, and saving time.

[0004] However, cloud-edge-device collaborative path planning technology still has limitations in practical applications. First, when end devices monitor road conditions through sensors and data acquisition devices, the vast amount of road data makes them unable to identify the importance of routes and insensitive to abnormal road sections. Uploading all road condition data significantly increases path planning overhead. Uploading only sparse road condition data can affect model accuracy. Second, when training path selection models at edge nodes, multi-vehicle path selection models often need to consider both the local optimality of individual vehicles and the collaborative efficiency of the global traffic system. Traditional path planning models often face a trade-off between individual and global optimization, failing to effectively address the potential negative impact of individual path selection on overall traffic efficiency. Furthermore, cloud-edge-device collaboration faces security issues. When end devices upload road condition data to edge nodes, and when edge nodes collaborate with cloud servers for model training, untrusted edge nodes and cloud servers may infer private information of traffic users, posing serious privacy and security risks. Summary of the Invention

[0005] The present invention provides a traffic optimal path planning method based on sparse completion and collaborative Q deep learning, which relates to the field of traffic path planning, including: a key area adaptive selection method based on weighted error control to select sparse areas; a traffic map feature privacy protection method based on transformer to efficiently extract and protect traffic map features; a sparse traffic map feature matrix completion method based on KNN weighted average to complete the missing parts; a Q deep learning adaptive gradient protection method for sparse traffic map pre-training to protect model parameters; a sparse traffic flow optimal path planning method based on Kalman filter to select the optimal path; the protection method adopted by the present invention obtains Local differential privacy achieves high-availability optimal path planning while ensuring the privacy protection of sparse traffic data.

[0006] Specifically, the first aspect of the present application provides a traffic optimal path planning method based on sparse completion collaborative Q deep learning, including the following steps:

[0007] Step 1: Based on the adaptive selection method of weighted error control, sparse sampling is performed on the key areas. The sparsity is controlled to ensure quality by calculating the change rate of adjacent sampled data.

[0008] Step 2: Based on the transformer-based traffic map feature privacy protection method, the traffic image feature matrix is obtained and differential privacy protection is performed;

[0009] Step 3: A sparse traffic map feature matrix completion method based on KNN weighted average is used to complete the feature matrix by calculating missing data through weighted average;

[0010] Step 4: Q-deep learning adaptive gradient protection method for sparse traffic map pre-training, which performs differential privacy protection on model parameters based on gradient change trends;

[0011] Step 5: Optimize the model parameters and perform optimal path planning based on the Kalman filter-based sparse traffic flow optimal path planning method.

[0012] Furthermore, the step 1 specifically includes:

[0013] Step 1.1: Based on the adaptive data collection method, dynamic weights are assigned to different paths at different times. The traffic flow and weights of the paths are calculated and normalized. The weights of the paths selected in the current cycle are accumulated and summed to obtain the overall weight. The formula is as follows:

[0014] ;

[0015] ;

[0016] ;

[0017] in, is the traffic flow of path z;

[0018] is the traffic density of path z;

[0019] is the average speed of traffic on path z;

[0020] is the weight of path z;

[0021] is the total number of paths in the selected area;

[0022] is the traffic flow of path i;

[0023] is the number of selected paths;

[0024] is the overall weight;

[0025] Step 1.2: Use the overall weight to perform weighted error control to reduce the interference of outliers on the overall optimization process. Calculate the data change rate between two adjacent cycles. When the data change rate is less than the specified threshold, stop the current round of sparse sampling. The formula is as follows:

[0026] ;

[0027] ;

[0028] Stop sampling conditions:

[0029] ;

[0030] in, is the overall weight;

[0031] is the difference between the actual path data and the predicted path data in the kth cycle;

[0032] is the actual path data of the kth cycle;

[0033] is the predicted path data of the kth cycle;

[0034] is the maximum allowed error threshold;

[0035] is the maximum tolerance of relative error;

[0036] is the actual path data of the k-1th cycle;

[0037] is the specified threshold.

[0038] Furthermore, the step 2 specifically includes:

[0039] Step 2.1: After sampling, extract the feature map of the sparsely sampled area, and then use the convolution layer of CNN to process it to obtain the preliminary feature map;

[0040] The preliminary feature map is further processed using convolution kernels with expansion rates of 1, 2, and 3 to obtain local features:

[0041] ;

[0042] ;

[0043] in, is the local feature of the i-th image after MIN-MAX normalization;

[0044] The fusion feature represented by multiplying each feature map in the i-th feature map by its weight;

[0045] The 3 feature maps that the i-th feature map is split into along the channel;

[0046] is the weight of the three feature maps divided along the channel in the i-th feature map;

[0047] is the maximum value function;

[0048] is the minimum function;

[0049] is the total number of pictures;

[0050] is the weight of the first feature map into which the Softmax result is split along the channel dimension;

[0051] The weight of the second feature map that the Softmax result is split into along the channel dimension;

[0052] The weight of the third feature map that the Softmax result is split into along the channel dimension;

[0053] Step 2.2: Feature Map It is achieved by segmenting the feature map of each image along n dimensions using 1×1 convolution, and then performing n importance score maps. Calculation and segmentation are performed to collect a set of key positions, and feature weighting is applied to the key position set to obtain the final global feature. The formula is as follows:

[0054] ;

[0055] ;

[0056] in, is the set of key locations in the image;

[0057] is the total number of pictures;

[0058] is the number of key features;

[0059] is the nth feature of the first feature map;

[0060] is the first feature of the feature of the Mth feature map;

[0061] is the nth feature of the feature of the Mth feature map;

[0062] is the global feature of the i-th picture;

[0063] is the feature map at the i-th position;

[0064] is the feature map at the nth position;

[0065] For the global maximum pooling operation, a sigmoid function is applied to compress the score to between 0 and 1 to indicate the importance of each position;

[0066] is the score of n important features in the range of 0 to 1 in the first group of positions in the i-th image;

[0067] is the score of the n important features in the range of 0 to 1 in the n-th group of positions in the i-th image;

[0068] Step 2.3: Concatenate the obtained feature maps of global features and local features, and then use the transformer's self-attention mechanism to calculate the output feature matrix for each position. The formula is as follows:

[0069] ;

[0070] in, is the feature matrix of the i-th picture;

[0071] It is the self-attention mechanism of transformer;

[0072] is the query in transformer;

[0073] is the key in transformer;

[0074] is the value in transformer;

[0075] is the local feature of the i-th image after MIN-MAX normalization;

[0076] is the global feature of the i-th picture;

[0077] Step 2.4: For the feature matrix , from the Gaussian distribution Generate a dimensional noise matrix , the feature matrix is protected by the Gaussian mechanism of differential privacy, and the noise feature matrix is obtained , and then the noise matrix set after adding noise is obtained by proving Gaussian mechanism that satisfies differential privacy. The formula is as follows:

[0078] ;

[0079] ;

[0080] in, is the number of samples;

[0081] is the characteristic number;

[0082] is the result of privacy protection for the i-th feature matrix;

[0083] is the feature matrix of the i-th picture;

[0084] is generated from a Gaussian distribution dimensional noise vector;

[0085] is the i-th noise feature matrix, with latitude ;

[0086] is the value of the mth row and jth column in the i-th feature matrix;

[0087] The noise added to the mth row and jth column.

[0088] Furthermore, the step three specifically includes:

[0089] Step 3.1: According to the KNN algorithm, for the noise feature matrix f(i)' containing missing data x, select the K data closest to it in the feature matrix and calculate the weights of the K data respectively. The formula is as follows:

[0090] ;

[0091] in: is the weight of data K in the i-th traffic map feature matrix;

[0092] is a function used to calculate the distance between data;

[0093] is a parameter used to adjust the weight ratio and prevent the distance from being 0;

[0094] is the missing data in the feature matrix;

[0095] is the adjacent data of the missing data in the feature matrix;

[0096] is the total number of pictures;

[0097] Step 3.2: Accumulate and sum the weights to obtain the K feature matrix data closest to the missing data x in the i-th traffic map feature matrix and calculate the weight sum , missing data are estimated by weighted average algorithm, the formula is as follows:

[0098] ;

[0099] ;

[0100] in: is the weight sum of the nearest feature matrix data;

[0101] is the weight of data K in the i-th traffic map feature matrix;

[0102] is the complement value of missing feature matrix data;

[0103] is the value of the adjacent feature matrix data in the i-th traffic map feature matrix;

[0104] is the total number of pictures;

[0105] Step 3.3: Construct a complete traffic matrix based on the missing feature matrix data , the set of traffic map feature matrices is , which is expressed as follows:

[0106] ;

[0107] in: is the set of completed traffic map feature matrix;

[0108] is the feature matrix after the completion of the first traffic map;

[0109] is the feature matrix after the second traffic map is completed;

[0110] is the feature matrix after the Mth traffic map is completed;

[0111] The total number of pictures.

[0112] Furthermore, the step 4 specifically includes:

[0113] Step 4.1: Based on the collection of complete traffic map feature matrix Construct individual Q networks and local Q networks. The individual Q network generates the optimal path action, and the local Q network generates the optimal joint action. After executing the action, the transferred data is stored in the experience buffer to provide training data for subsequent network updates and training samples for network updates. The Q value update formula of the individual Q network is:

[0114] ;

[0115] The Q value update formula of the local Q network is:

[0116] ;

[0117] in: is the updated Q value of the individual Q network;

[0118] Q values calculated for individual Q networks;

[0119] is the learning rate, which determines the step size of the Q value update;

[0120] is the individual reward value;

[0121] is the discount factor, The larger the value, the more emphasis the model places on long-term benefits and global collaboration, while the opposite is true for immediate rewards.

[0122] For the next observation and next path selection action of the individual Q network, select the action that maximizes the Q value;

[0123] Select an action for the path of the jth vehicle;

[0124] is the observation value of the individual Q network for the j-th vehicle;

[0125] is the updated Q value of the local Q network;

[0126] Q values calculated for the local Q network;

[0127] is the local reward value;

[0128] Select the action that maximizes the Q value for the next state and the next local joint path selection action of all vehicles in the local Q network;

[0129] Select actions for local joints;

[0130] It is a local state, i.e., the traffic environment information within the edge node area;

[0131] For all possible path choices, ;

[0132] Select a combination of routes for all vehicles,

[0133] ;

[0134] In order to achieve the balance between individual optimization and local cooperation in path planning, the network collects training data through T iterations, gradually optimizes the individual Q network and the local Q network, and each vehicle uses the individual Q network to calculate its observation value. Process and select the optimal path action:

[0135] ;

[0136] in: From the path collection Select Make maximum;

[0137] Then the edge node summarizes the optimal path selection actions of all vehicles , combining them into joint actions , the joint path selection action is evaluated through the local Q network, and the joint path solution with the greatest benefit is selected:

[0138] ;

[0139] in: From the joint action set Select an optimal joint action Make maximum;

[0140] Finally, execute the action and record the state transfer data and , stored in the experience buffer for network update.

[0141] Step 4.2: From the buffer and Randomly sample small batches of data in to calculate the loss function of the individual Q network and the local Q network loss function. The formula is as follows:

[0142] ;

[0143] ;

[0144] ;

[0145] ;

[0146] in, is the target value of the individual Q network;

[0147] is the discount factor, The larger the value, the more emphasis the model places on long-term benefits and global collaboration, while the opposite is true for immediate rewards.

[0148] is the individual reward value;

[0149] For the next observation and next path selection action of the individual Q network, select the action that maximizes the Q value;

[0150] is the local Q network target value;

[0151] is the local reward value;

[0152] Select the action that maximizes the Q value for the next state and the next local joint path selection action of all vehicles in the local Q network;

[0153] is the loss function of the individual Q network;

[0154] Buffer zone from individual experience Sampling data in the network, updating individual Q network;

[0155] An individual experience buffer that stores the historical experience and state-action pairs of individual vehicles;

[0156] The Q value calculated by the individual Q network is the estimated value of the vehicle choosing a certain action in a certain state;

[0157] is the loss function of the local Q network;

[0158] For local experience buffer Sampling data in the network, updating the local Q network;

[0159] A local experience buffer that stores historical experiences and joint state-action pairs of multiple vehicles;

[0160] Q values calculated for the local Q network;

[0161] Step 4.3: Introduce the loss function based on the cosine similarity coordination mechanism to obtain the pre-training loss function, and optimize the model parameters through the gradient descent algorithm. The formula is as follows:

[0162] ;

[0163] ;

[0164] Optimize model parameters through gradient descent algorithm:

[0165] ;

[0166] ;

[0167] in, is the loss function based on the cosine similarity coordination mechanism;

[0168] Q values calculated for individual Q networks;

[0169] Q values calculated for the local Q network;

[0170] is the number of vehicles;

[0171] It is a fully connected layer that maps the Q value to a 128-dimensional vector space to facilitate the calculation of cosine similarity.

[0172] is the pre-training loss function;

[0173] is the loss function of the individual Q network;

[0174] is the loss function of the local Q network;

[0175] is the weight of the cosine similarity coordination mechanism, The larger the value, the more emphasis is placed on the coordination mechanism's constraints on the consistency of individual and global strategies; the smaller the value, the less likely it is to interfere with the loss function excessively.

[0176] It is the gradient descent learning rate, which controls the step size of each parameter update;

[0177] are the model parameters of the optimized individual Q network;

[0178] are the model parameters of the individual Q network;

[0179] is the gradient of the individual Q network model parameters;

[0180] are the model parameters of the optimized local Q network;

[0181] are the model parameters of the local Q network;

[0182] is the gradient of the local Q network model parameters;

[0183] Step 4.4: Predict the gradient of the t+1th round of model training through the second-order exponential smoothing mechanism norm, and then according to the gradient of the t+1th round and the tth round The norm is used to obtain the gradient change trend of the t+1th round, and then the adjustment factor is calculated. , the formula for calculating the adjustment factor is as follows:

[0184] ;

[0185] in, is the adjustment factor for round t;

[0186] is the gradient of the ti-th round norm;

[0187] is the gradient of the ti-1 round norm;

[0188] is the gradient change trend weight, the smaller i is The bigger;

[0189] Step 4.5: Adjust the gradient clipping threshold according to the adjustment factor. If the gradient trend increases, the clipping threshold will be increased, adding more noise. Conversely, the clipping threshold will be lowered, adding less noise. Finally, the adaptive gradient Laplace noise is used to generate the noisy model parameter vector. The formula is as follows:

[0190] ;

[0191] ;

[0192] ;

[0193] in, For the The mean of the round adjustment factor;

[0194] is the adjustment factor of the i-th model parameter in the t-th round;

[0195] is the number of model parameters;

[0196] is the clipping threshold of the tth round;

[0197] is the pruning threshold of the t-1th round;

[0198] is the model parameter vector of round t;

[0199] is the model parameter vector after adding noise in the tth round;

[0200] is the Laplace noise vector, Each value in the vector is the Laplace noise of each parameter, , is the clipping threshold of the t-th round gradient, is the privacy budget of each parameter.

[0201] Furthermore, the step five specifically includes:

[0202] Step 5.1: The cloud receives the noise parameters from N edge nodes ,The cloud uses the federated averaging algorithm to aggregate multiple edge models and update the global model parameters. The formula is as follows:

[0203] ;

[0204] in, is the global model parameter;

[0205] is the number of edge nodes;

[0206] represents the noise parameter of the i-th edge node;

[0207] The global model parameters obtained When sending it to the edge node for optimal path planning, the presence of noise may cause errors in the optimal path planning. Kalman filter is used to Smoothing and optimization processing is performed, which includes a prediction step and an update step. The prediction step predicts the state and error covariance of the current moment by the state of the previous moment.

[0208] Step 5.2: Use the Kalman filter to smooth and optimize the obtained global model parameters, including the prediction step and the update step. The formula in the prediction step is as follows:

[0209] ;

[0210] ;

[0211] ;

[0212] in, are the global model parameters for prediction;

[0213] is the global model parameter at the previous moment;

[0214] is the predicted error covariance matrix;

[0215] is the error covariance of the previous moment;

[0216] is the process noise covariance matrix;

[0217] is a small constant to avoid the prediction error covariance Infinitely decrease to maintain Kalman gain effectiveness;

[0218] is the identity matrix.

[0219] Step 5.3: In the update step, the predicted state is combined with the noise parameter to correct the predicted value, and finally the global model parameters optimized by the Kalman filter are obtained. , the formula is as follows:

[0220] ;

[0221] ;

[0222] ;

[0223] ;

[0224] in, is the Kalman gain;

[0225] is the predicted error covariance matrix;

[0226] is the observation noise covariance matrix, which is used to represent the noise uncertainty in the actual observation value;

[0227] is the updated global model parameter;

[0228] are the global model parameters for prediction;

[0229] is the global model parameter;

[0230] is the updated error covariance;

[0231] is the identity matrix;

[0232] For the The clipping threshold of the wheel;

[0233] Budget for privacy.

[0234] Step 5.4: The global parameters optimized by Kalman filter Used for model update, defining noise loss function , to reduce the impact of noise on training, and then use the weighted loss function to comprehensively consider the pre-training loss function and noise loss function , optimize the above weighted loss function through the gradient descent algorithm:

[0235] ;

[0236] Weighted loss function: ;

[0237] Gradient Descent Algorithm:

[0238] ;

[0239] ;

[0240] in, is the noise loss function;

[0241] Q values calculated for individual Q networks;

[0242] Q values calculated for the local Q network;

[0243] is the regularization term of parameter difference, which is used to limit the optimization Deviation from the aggregation results the extent of

[0244] is the regularization coefficient, The larger it is, the stricter the limit on the deviation of the parameter from the aggregation result. The smaller it is, the more flexible the parameters are allowed to be adjusted according to local dynamics;

[0245] is the number of edge nodes;

[0246] 、 are weighted coefficients of the weighted loss function, The higher the value, the more the model focuses on improving the accuracy of path planning. The higher the value, the stronger the model's suppression of noise in the aggregated parameters;

[0247] is the pre-training loss function;

[0248] It is the gradient descent learning rate, which controls the step size of each parameter update;

[0249] are the model parameters of the optimized individual Q network;

[0250] are the model parameters of the individual Q network;

[0251] is the gradient of the individual Q network model parameters;

[0252] are the model parameters of the optimized local Q network;

[0253] are the model parameters of the local Q network;

[0254] is the gradient of the local Q network model parameters;

[0255] Step 5.5: After Kalman filtering the noise parameters and optimizing the noise loss function, use the updated model to calculate the individual optimal path for each vehicle. Then, optimize the local optimal path planning based on the local value network to obtain the overall optimal path. The formula is as follows:

[0256] ;

[0257] ;

[0258] in, is the optimal path for an individual;

[0259] Based on the updated model parameters From the path collection Choose an optimal path Maximize the individual Q value;

[0260] is the local optimal path;

[0261] Based on the updated model parameters local from the joint action set Select an optimal joint action Make maximum.

[0262] In the second aspect, the present application also provides a computing device, which has the function of implementing the method described in the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by executing the corresponding software through hardware. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the structure of the device includes an acquisition module, a training module, and optionally, a construction module. These modules can implement the function of the training node in the method example of the first aspect above. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0263] In a third aspect, the present application further provides a computing device for implementing the functions of the method described in the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The structure of the computing device includes a processor and a memory, and the memory is used to store instructions and / or data. The memory is coupled to the processor, and when the processor executes the program instructions stored in the memory, the functions of the training node in the example of the first aspect above can be implemented. The structure of the computing device also includes a communication interface for communicating with other devices.

[0264] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the method in the above-mentioned first aspect and various possible designs of the first aspect.

[0265] In a fifth aspect, the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect and various possible designs of the first aspect.

[0266] In a sixth aspect, the present application also provides a computing chip, which is connected to a memory and is used to read and execute software programs stored in the memory, and to execute the methods in the above-mentioned first aspect and various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0267] In order to more clearly illustrate the embodiments of the present drawings or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present drawings. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0268] Figure 1 is a flow chart of the steps of the present invention;

[0269] Figure 2It is a specific operation flow chart of the present invention;

[0270] Figure 3 Graph comparing the path planning accuracy of the embodiment of the present invention and the traditional sparse data sampling method under different maximum error thresholds;

[0271] Figure 4 This figure compares the path planning accuracy of the embodiment of the present invention and the traditional image privacy protection method under different image feature dimensions;

[0272] Figure 5 This figure compares the path planning accuracy of the embodiment of the present invention and the traditional model parameter protection method under different privacy budgets;

[0273] Figure 6 A comparison chart of the path planning accuracy of the embodiment of the present invention and the traditional Q-learning path planning method under different gradient learning rates;

[0274] The purpose, features and advantages of this drawing will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0275] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0276] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0277] Unless otherwise specified, all embodiments and optional embodiments of the present application can be combined with each other to form a new technical solution.

[0278] Unless otherwise specified, all technical features and optional technical features of this application can be combined with each other to form a new technical solution.

[0279] Unless otherwise specified, all steps of the present application may be performed sequentially or randomly, preferably sequentially. For example, the method includes steps (a) and (b), indicating that the method may include steps (a) and (b) performed sequentially, or may include steps (b) and (a) performed sequentially. For example, the method may further include step (c), indicating that step (c) may be added to the method in any order, for example, the method may include steps (a), (b) and (c), or may include steps (a), (c) and (b), or may include steps (c), (a) and (b), etc.

[0280] Unless otherwise specified, the terms "include" and "comprising" used in this application may be open-ended or closed-ended. For example, "include" and "comprising" may mean that other components not listed may also be included or that only the listed components are included.

[0281] Unless otherwise specified, the term "or" is used in this application to be inclusive. For example, the phrase "A or B" means "A, B, or both A and B." More specifically, the condition "A or B" is satisfied if any of the following conditions are met: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); or both A and B are true (or exist).

[0282] In order to better understand the solutions of the embodiments of the present application, some relevant terms and concepts that may be involved in the embodiments of the present application are first introduced below.

[0283] (1) The Q network is a deep neural network that takes in the state of the environment (such as sensor data or images) and outputs the Q value of each possible action, that is, the value estimate of the state-action pair. For example, in DQN, the input of the Q network is the state vector, and the number of neurons in the output layer is equal to the dimension of the action space. Each neuron corresponds to the Q value of an action. It usually consists of an input layer, a hidden layer (such as a fully connected layer, an activation function), and an output layer. For example, a stacked structure of fully connected layers using ReLU activation is often divided into an online network (updated in real time for action selection) and a target network (regularly synchronized parameters for calculating the target value of the time delay) in DQN to stabilize training.

[0284] (2) Loss function. In the process of training a deep neural network, because we hope that the output of the deep neural network is as close as possible to the value we really want to predict, we can compare the current network's predicted value with the target value we really want, and then update the weight vector of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value, and the adjustment is continued until the deep neural network can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function. They are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0285] (3) Kalman Filter is an efficient recursive estimation algorithm used to estimate the system state from noisy observation data in dynamic systems. Its core idea is to combine prediction (based on the system model) and update (based on the observation data) to achieve optimal estimation by minimizing the mean square error (MSE).

[0286] like Figure 1 As shown, in this embodiment, the traffic optimal path planning method based on sparse completion collaborative Q deep learning includes the following steps:

[0287] Step 1: Based on the adaptive selection method of weighted error control, sparse sampling is performed on the key areas. The sparsity is controlled to ensure quality by calculating the change rate of adjacent sampled data.

[0288] Step 2: Based on the transformer-based traffic map feature privacy protection method, the traffic image feature matrix is obtained and differential privacy protection is performed;

[0289] Step 3: A sparse traffic map feature matrix completion method based on KNN weighted average is used to complete the feature matrix by calculating missing data through weighted average;

[0290] Step 4: Q-deep learning adaptive gradient protection method for sparse traffic map pre-training, which performs differential privacy protection on model parameters based on gradient change trends;

[0291] Step 5: Optimize the model parameters and perform optimal path planning based on the Kalman filter-based sparse traffic flow optimal path planning method.

[0292] In this embodiment, the specific operation flow chart of the present invention is as follows Figure 2 shown.

[0293] Furthermore, step one specifically includes:

[0294] Assume that in the kth cycle sampling, there are a total of x paths in a certain area, and their set is expressed as:

[0295] ;

[0296] The sampling selected y paths, and the set is represented as:

[0297] ;

[0298] Set Y is a subset of set X. The traffic flow of path z is calculated using the traffic density K and the average traffic speed V. The larger the traffic flow, the more important the corresponding path is in this area and during this period. The normalized result is used as the weight value of path z.

[0299] Step 1.1: Based on the adaptive selection method, dynamic weights are assigned to different paths at different times. The traffic flow and weights of the paths are calculated and normalized. The weights of the paths selected in the current cycle are summed to obtain the overall weight. The formula is as follows:

[0300] ;

[0301] ;

[0302] ;

[0303] in, is the traffic flow of path z;

[0304] is the traffic density of path z;

[0305] is the average speed of traffic on path z;

[0306] is the weight of path z;

[0307] is the total number of paths in the selected area;

[0308] is the traffic flow of path i;

[0309] is the number of selected paths;

[0310] is the overall weight;

[0311] Step 1.2: Use the obtained overall weight to perform weighted error control to reduce the interference of outliers on the overall optimization process. Then calculate the data change rate between two adjacent cycles. When the data change rate is less than the specified threshold, stop this round of sparse sampling. The formula is as follows:

[0312] ;

[0313] ;

[0314] Stop sampling conditions:

[0315] ;

[0316] in, is the overall weight;

[0317] is the difference between the actual path data and the predicted path data in the kth cycle;

[0318] is the actual path data of the kth cycle;

[0319] is the predicted path data of the kth cycle;

[0320] is the maximum allowed error threshold;

[0321] is the maximum tolerance of relative error;

[0322] is the actual path data of the k-1th cycle;

[0323] is the specified threshold.

[0324] Furthermore, step 2 specifically includes:

[0325] The transformer-based traffic map feature privacy protection method uses the convolutional layer of a CNN to perform preliminary processing on the M images in the collected sample to obtain a feature map of each image. Then, convolution kernels with different expansion rates are used to extract local features from the feature map of each image. At the same time, the global features of the feature map are extracted through a key area scoring mechanism. The obtained local features and global features are fused and used as the input of the transformer to obtain the feature matrix of each image. Finally, the feature matrices of all the images are noised and transmitted to the edge nodes for model training.

[0326] Step 2.1: The input image set is I={I(1), I(2), ..., I(M)}, where , H is the length, W is the width, the number of channels is 3, Represents the image information of the i-th picture. Directly performing convolution operation on the image may cause some information loss. Therefore, zero padding is first used to complete the image data to obtain a new image.

[0327] After the image is processed by the convolution layer, a feature map is obtained. These feature maps are then spliced together, and then the spliced feature maps are reduced in dimension using a 1×1 convolution with a channel size of 3. The feature map of the i-th image is obtained. , which is expressed as:

[0328] ;

[0329] in: is the feature map of the i-th picture;

[0330] The j-scale feature map input for the i-th image;

[0331] To perform a splicing operation on all feature maps in the jth image;

[0332] Then the feature map set , the convolution kernels with different expansion rates of 1, 2, and 3 are used to process the feature map set, and the three feature maps of the i-th picture are decomposed in the i-th feature map, which are , , , then these three feature maps are applied with a 1×1 convolution with a channel number of 3, and then Softmax is used to obtain the general distribution of the three feature maps. Finally, the Softmax result is split into three different feature maps along the channel dimension, and each feature map corresponds to a weight w, so we can get , the formula is as follows:

[0333] ;

[0334] Afterwards, in order to enhance the comparability between image data indicators, MIN-MAX normalization is used to process the obtained output features and extract local features in the preliminary sparse sampling traffic feature matrix. .

[0335] The formula is as follows:

[0336] ;

[0337] Step 2.2: Feature Map It is achieved by segmenting the feature map of each image along n dimensions using 1×1 convolution, and then performing n importance score maps. Calculation and segmentation are performed to collect a set of key positions, and feature weighting is applied to the key position set to obtain the final global feature. The formula is as follows:

[0338] ;

[0339] ;

[0340] Acquiring global features: When detecting traffic signs, distinctive areas such as intersections, near schools, and construction zones can serve as key locations for global features. By strengthening the information collection of these key locations, the required global features can be obtained, enhancing the recognition of key features in the image. The present invention uses a global maximum pooling operation to determine the maximum value in each convolution kernel channel to return the importance score of the position (x, y) and channel. Subsequently, a key location set with n key features is derived based on the importance score, and the collected features are highlighted and enhanced based on this key location set.

[0341] Step 2.3: Concatenate the obtained feature maps of global features and local features, and then use the transformer's self-attention mechanism to calculate the output feature matrix for each position. The formula is as follows:

[0342] ;

[0343] In the present invention, the traffic flow feature is specifically manifested as the real-time number of vehicles in each lane recorded in a column of the feature matrix.

[0344] Step 2.4: Define Sensitivity For function The maximum value among adjacent datasets Norm change. Set adjacent data sets and Only when the difference of a certain element is at most 1, then the matrix The norm difference is ,in for norm, so The sensitivity of .

[0345] For vector , , and are adjacent datasets, and the adjacent matrix is limited to a single element difference of 1, given a privacy budget and failure probability , , privacy budget Controls the strength of privacy protection. The smaller the value, the higher the privacy protection and the probability of failure. It represents the probability that the mechanism may violate the definition of differential privacy in extreme cases.

[0346] For the feature matrix , from the Gaussian distribution Generate a dimensional noise matrix , the feature matrix is protected by the Gaussian mechanism of differential privacy, and the noise feature matrix is obtained , the formula is as follows:

[0347] ;

[0348] ;

[0349] Set a feature matrix , there is only one pair , making , and for all , both , The corresponding feature matrix is .

[0350] In order to ensure the privacy of the feature matrix set uploaded to the edge node, it is necessary to theoretically prove that the designed Gaussian mechanism based on differential privacy satisfies Differential privacy conditions:

[0351] Proof: Prove that the privacy protection mechanism adopted in this invention satisfies the differential privacy condition, that is, prove that the noise scale added by the Gaussian mechanism to each element in each feature matrix is , , , and It is the feature matrix with noise added corresponding to the matrices of two adjacent data sets.

[0352] Proof required . make , for any feature matrix , for any measurable set ,

[0353] Noise standard deviation Need to meet , perform upper bound analysis on the probability ratio of the output distribution:

[0354] ;in ,and , .

[0355] Consider adjacent datasets and The noise output is and , the difference .because And the noise is independent, the probability ratio can be further simplified as:

[0356] ;

[0357] in .

[0358] Define the privacy loss random variable:

[0359] ;

[0360] because , its moment generating function decomposes the expectation and uses the properties of the Gaussian moment generating function to obtain:

[0361] ;

[0362] For any , in the conversion process The exponential term about Take the derivative and set it to 0, and simplify it. Verifiable about The inequality holds, so we can conclude that:

[0363] ;

[0364] The tail probability of the Gaussian distribution satisfies:

[0365] ;

[0366] Due to the properties of Gaussian distribution, for any ,have:

[0367] ;

[0368] make ,but:

[0369] ;

[0370] When the noise parameter satisfy When any observable collection ,have:

[0371] ;

[0372] Therefore, it can be proved that the above privacy protection process satisfies the formula of differential privacy and has the effect of privacy protection.

[0373] Furthermore, in step three, in order to complete the above traffic Figure 2 The KNN-based weighted average interpolation method is used to predict missing data by leveraging the similarity of known data. Compared with traditional interpolation methods, the KNN weighted average interpolation method assigns different weights to adjacent samples, making the interpolation results more accurate and fully utilizing the similarity between data. It has obvious advantages when facing a large number of missing values, including:

[0374] Step 3.1: According to the KNN algorithm, for the noise feature matrix f(i)' containing missing data x, select the K data closest to it in the feature matrix and calculate the weights of the K feature matrix data respectively. The formula is as follows:

[0375] ;

[0376] Step 3.2: Accumulate and sum the weights to obtain the K feature matrix data closest to the missing data x in the i-th traffic map feature matrix and calculate the weight sum , the missing feature matrix data is estimated by weighted average algorithm, the formula is as follows:

[0377] ;

[0378] (i=1,2,3...,M);

[0379] Step 3.3: Construct a complete traffic feature matrix based on the missing feature matrix data , the set of traffic map feature matrices is , which is expressed as follows:

[0380] ;

[0381] Furthermore, in step four, a deep Q-learning framework is employed to combine individual and local value networks, using the feature matrix completed in step three as input. The individual network optimizes the routing strategy for individual vehicles, while the local network evaluates the benefits of multi-vehicle collaboration. A cosine similarity coordination mechanism is used to balance individual and global benefits. The model parameters are then optimized using a gradient descent algorithm. Furthermore, differential privacy protection (e.g., adaptive gradient noise) is applied to the model parameters to prevent the cloud from inferring sensitive data on the edge, ensuring secure model upload.

[0382] Step 4.1: Construct individual Q-network and local Q-network and process vehicle observations and regional status , the individual Q network generates the optimal path action, the local Q network generates the optimal joint action, and then records the state transition data and stores it in the buffer to provide training samples for network updates.

[0383] For the traffic map feature matrix set Contains observations for individual vehicles and the local state within the region For each vehicle, its observation information , including the coordinates of the current location, speed, path cost and traffic status and other basic data, after standardization, are input into the individual Q network to generate individual path selection strategy, local state It represents the traffic environment information in the edge node area, which is input into the local Q network after standardization.

[0384] Two core network architectures are designed in this invention: individual Q network and local Q network.

[0385] The main task of the individual Q network is to estimate the individual value of each vehicle under different path selection actions and dynamically adjust its path selection strategy according to the local environment of the vehicle, thereby improving the flexibility and accuracy of path planning. Its input is the observation of the vehicle and individual actions , the output is the value estimate of the individual's corresponding action, and its Q value is updated by the following formula:

[0386] ;

[0387] The task of the local Q network is to evaluate the overall value of the joint actions of multiple vehicles in the local area, further optimize the global cooperation efficiency, and avoid congestion caused by multiple vehicles choosing the same path. Its input is the local state and joint path selection actions , the output is the benefit estimate of the local joint path selection action, and its Q value is updated by the following formula:

[0388] ;

[0389] Then design the reward function of the two Q networks, the individual reward value and local reward value ,

[0390] The design of individual reward values combines factors such as safety, efficiency, and smoothness. Safety rewards ensure basic driving safety, efficiency rewards improve driving efficiency, and smoothness rewards improve driving stability. The formula for individual comprehensive rewards is:

[0391] ;

[0392] 、 and is the weight coefficient of the three reward items, if Larger, more emphasis is placed on safety rewards, if If the larger the value, the more emphasis is placed on efficiency rewards. A larger value places more emphasis on smoothness rewards.

[0393] Safety Rewards The purpose is to prevent vehicles from choosing paths that may lead to collisions or dangers, encourage vehicles to maintain a safe distance, and ensure driving safety. The safety reward is defined by the following formula:

[0394] ;

[0395] in: Reward for safety;

[0396] is the minimum distance to surrounding vehicles;

[0397] is the safety distance threshold;

[0398] It is a smoothing term to prevent the reward value from approaching negative infinity;

[0399] Efficiency Rewards It aims to shorten travel time and improve traffic efficiency. It guides vehicles to choose more efficient routes by quantifying path length, average speed, and waiting time. The efficiency reward is defined by the following formula:

[0400] ;

[0401] in: Reward for efficiency;

[0402] is the traffic density of path segment p;

[0403] Smoothness Reward Reduce the negative impact of road conditions on the vehicle. The straighter and flatter the path, the smoother the vehicle's ride will be, thus increasing the reward and improving the driving experience. The formula is as follows:

[0404] ;

[0405] in: Reward for smoothness;

[0406] is the curvature of the path segment p;

[0407] is the flatness of the path segment p;

[0408] and is the weight coefficient, if If it is larger, more emphasis will be placed on reducing the impact of path curvature. Larger ones place more emphasis on the flatness of the path.

[0409] Local reward value The design combines local traffic efficiency and congestion conditions to optimize the coordination of multi-vehicle joint path selection. The formula is as follows:

[0410] ;

[0411] in: is the local reward value;

[0412] and are the weight coefficients of the reward items respectively. If If it is larger, it will focus more on improving overall efficiency. On the contrary, Larger ones focus more on reducing congestion costs;

[0413] Efficiency Gain Bonus It represents the improvement of local path efficiency after the joint action a is executed. The formula is as follows:

[0414] ;

[0415] in: rewards for efficiency gains;

[0416] represents the path length of vehicle j;

[0417] is the average speed of vehicle j before the action;

[0418] is the average speed of vehicle j after the action;

[0419] Congestion Cost Rewards Used to punish traffic congestion that may be caused by joint actions. The formula is as follows:

[0420] ;

[0421] in: is the traffic density of road section p;

[0422] is the total number of road sections within the local area;

[0423] In order to achieve the balance between individual optimization and local cooperation in path planning, the network collects training data through T iterations, gradually optimizes the individual Q network and the local Q network, and each vehicle uses the individual Q network to calculate its observation value. Process and select the optimal path action:

[0424] ;

[0425] The vehicles then calculate their priorities based on the surrounding traffic conditions. Vehicles with high priorities will perform the optimal path selection action for each individual, while vehicles with low priorities provide candidate actions for the local Q network to optimize joint actions and avoid local conflicts. The priority formula is as follows:

[0426] ;

[0427] in: is the priority of vehicle j;

[0428] is the average speed of vehicle j before the action;

[0429] is the traffic density of surrounding vehicles;

[0430] are weight parameters respectively, if The larger the size, the more emphasis is placed on vehicle speed, and vice versa. If it is larger, it will pay more attention to the surrounding traffic density;

[0431] when When the priority is set to high, otherwise it is set to low. Assuming the initial setting Always greater than the current round , Always less than , then .

[0432] Then the edge node summarizes the optimal path selection actions of all vehicles , combining them into joint actions , the joint path selection action is evaluated through the local Q network, and the joint path solution with the greatest benefit is selected:

[0433] ;

[0434] Finally, execute the action and record the state transfer data and , stored in the experience buffer for network update.

[0435] Step 4.2: From the buffer and Randomly sample small batches of data in to calculate the loss function of the individual Q network and the local Q network loss function. The formula is as follows:

[0436] ;

[0437] ;

[0438] ;

[0439] ;

[0440] In order to update the model parameters of the network, the network is updated by sampling data from the buffer through T iterations, each performing P learning steps.

[0441] Step 4.3: To avoid conflicts between the optimal path of individual vehicles and global traffic efficiency, a coordination mechanism based on cosine similarity is designed to force the individual strategy to be consistent with the global strategy. The pre-trained loss function is obtained and the model parameters are optimized using the gradient descent algorithm. The formula is as follows:

[0442] ;

[0443] ;

[0444] Optimize model parameters through gradient descent algorithm:

[0445] ;

[0446] ;

[0447] Step 4.4: Gradients from the t+1th round of model training Norm prediction value to get the adjustment factor, for the gradient of the model parameters in the t+1th round Norm prediction can use the first three rounds of gradients of the parameter in round t+1 norm , , ;

[0448] The gradient of the t+1 round of model training is predicted by the second-order exponential smoothing mechanism norm, and then according to the gradient of the t+1th round and the tth round The norm is used to obtain the gradient change trend of the t+1th round, and then the adjustment factor is calculated. , the formula for calculating the adjustment factor is as follows:

[0449] ;

[0450] Step 4.5: Adjust the gradient clipping threshold according to the adjustment factor. If the gradient trend increases, the clipping threshold will be increased, adding more noise. Conversely, the clipping threshold will be lowered, adding less noise. Finally, the adaptive gradient Laplace noise is used to generate the noisy model parameter vector. The formula is as follows:

[0451] ;

[0452] ;

[0453] ;

[0454] In differential privacy, function sensitivity S(f) describes the maximum impact that a change in a single data point may have on the function output. The clipping threshold C is used to limit the gradient contribution of each data point. Therefore, the intensity of Laplace noise is proportional to the clipping threshold, and the clipping threshold can be adjusted by adjusting the factor ,If the gradient change trend is rising, the clipping threshold will be increased and more noise will be added. Otherwise, the clipping threshold will be lowered and less noise will be added.

[0455] According to the privacy budget of each parameter and clipping threshold , generate the Laplace noise vector Y that satisfies differential privacy, and finally pass the adaptive gradient Laplace noise according to the formula: , generate the model parameter vector after noise addition .

[0456] To ensure the privacy of uploaded model parameters, it is necessary to theoretically prove that the designed adaptive gradient noise protection mechanism meets Differential privacy conditions.

[0457] Proof: Prove that the adaptive gradient noise protection method proposed in this invention satisfies The differential privacy condition is to prove that the Laplace mechanism adds noise of scale to the model parameters uploaded by each edge node. noise, satisfying .

[0458] Given any adjacent dataset and , with model parameters and Make ,

[0459] ;

[0460] Then each model parameter is noised to meet Differential privacy, based on the serial principle characteristics of differential privacy, the designed adaptive gradient noise protection method meets Differential privacy conditions.

[0461] Furthermore, step five specifically includes:

[0462] Step 5.1: The cloud receives the noise parameters from N edge nodes ,The cloud uses the federated averaging algorithm to aggregate multiple edge models and update the global model parameters. The formula is as follows:

[0463] ;

[0464] The global model parameters obtained When sending it to the edge node for optimal path planning, the presence of noise may cause errors in the optimal path planning. We need to use Kalman filter to Smoothing and optimization processing is performed, which includes a prediction step and an update step. The prediction step predicts the state and error covariance of the current moment by the state of the previous moment.

[0465] Step 5.2: Use the Kalman filter to smooth and optimize the obtained global model parameters, including the prediction step and the update step. Set the state transfer matrix and the observation matrix to the identity matrix. The formula in the prediction step is as follows:

[0466] ;

[0467] ;

[0468] ;

[0469] Step 5.3: In the update step, the predicted state is combined with the noise parameter to correct the predicted value, and finally the global model parameters optimized by the Kalman filter are obtained. , the formula is as follows:

[0470] ;

[0471] ;

[0472] ;

[0473] ;

[0474] Step 5.4: The global parameters optimized by Kalman filter , used for model update, define a noise loss function to reduce the impact of noise on training, and then use the weighted loss function to comprehensively consider the pre-training loss function and the noise loss function, and optimize the above weighted loss function through the gradient descent algorithm. The formula is as follows:

[0475] ;

[0476] Weighted loss function: ;

[0477] Gradient Descent Algorithm:

[0478] ;

[0479] ;

[0480] Step 5.5: After Kalman filtering the noise parameters and optimizing the noise loss function, use the updated model to calculate the individual optimal path for each vehicle. Then, optimize the local optimal path planning based on the local value network to obtain the overall optimal path. The formula is as follows:

[0481] ;

[0482] ;

[0483] In this embodiment, the path planning accuracy comparison between the embodiment of the present invention and the traditional sparse data sampling method under different maximum error thresholds is shown in the figure below: Figure 3 As shown in the figure below; the path planning accuracy comparison between the embodiment of the present invention and the traditional image privacy protection method under different image feature dimensions is shown in the figure below. Figure 4 As shown in the figure below; the comparison of the path planning accuracy of the embodiment of the present invention and the traditional model parameter protection method under different privacy budgets is shown in the figure below. Figure 5 As shown in the figure; the path planning accuracy comparison of the embodiment of the present invention and the traditional Q-learning path planning method under different gradient learning rates is shown in the figure Figure 6 As shown;

[0484] Depend on Figures 3 to 6 It can be seen that under different maximum error thresholds, different image feature dimensions, different privacy budgets, and different gradient learning rates, the path planning accuracy of the embodiments of the present invention is higher than that of the currently commonly used traditional methods.

[0485] It should be noted that the present application is not limited to the above-mentioned embodiments. The above-mentioned embodiments are merely examples, and any embodiments having substantially the same structure and effect as the technical concept within the scope of the present application are all included in the technical scope of the present application. In addition, without departing from the scope of the present application, any other embodiments that can be conceived by those skilled in the art and that combine some of the constituent elements in the embodiments are also included in the scope of the present application.

Claims

1. A traffic optimal path planning method based on sparse completion collaborative Q deep learning, characterized by: The following steps are involved: Step 1: Based on the adaptive selection method of weighted error control, sparse sampling is performed on the key areas. The sparsity is controlled to ensure quality by calculating the change rate of adjacent sampled data. The step 1 specifically includes: Step 1.1: Based on the adaptive data collection method, dynamic weights are assigned to different paths at different times. The traffic flow and weights of the paths are calculated and normalized. The weights of the paths selected in the current cycle are accumulated and summed to obtain the overall weight. Step 1.2: Use the overall weight to perform weighted error control to reduce the interference of outliers on the overall optimization process, perform relative error limits, calculate the data change rate between two adjacent cycles, and stop this round of sparse sampling when the data change rate is less than the specified threshold; Step 2: Based on the transformer-based traffic map feature privacy protection method, the traffic image feature matrix is obtained and differential privacy protection is performed; Step 3: A sparse traffic map feature matrix completion method based on KNN weighted average is used to complete the feature matrix by calculating missing data through weighted average; Step 4: Q-deep learning adaptive gradient protection method for sparse traffic map pre-training, which performs differential privacy protection on model parameters based on gradient change trends; The fourth step specifically includes: Step 4.1: Based on the complete set of traffic map feature matrices, individual Q networks and local Q networks are constructed. The individual Q networks generate the optimal path action, while the local Q networks generate the optimal joint action. After the action is executed, the data is transferred and stored in the experience buffer to provide training data for subsequent network updates. Step 4.2: Sample data from the experience buffer, set the target value, and calculate the loss function of the individual Q network and the local Q network loss function; Step 4.3: Design a cosine similarity coordination mechanism, calculate the individual and local Q-value directional consistency loss, weightedly integrate it into the pre-training loss, and optimize the parameters through gradient descent to balance the conflict between individual and global strategies; Step 4.4: Use the second-order exponential smoothing mechanism to predict the gradient norm of the model training in the t+1th round. Then, use the gradient norms of the t+1th round and the tth round to obtain the gradient change trend of the t+1th round, and then calculate the adjustment factor. Step 4.5: Adjust the gradient clipping threshold according to the adjustment factor, and then generate the noisy model parameter vector through adaptive gradient Laplace noise; Step 5: Optimize the model parameters and perform optimal path planning based on the Kalman filter-based sparse traffic flow optimal path planning method.

2. The traffic optimal path planning method based on sparse completion collaborative Q deep learning according to claim 1 is characterized in that: The second step specifically includes: Step 2.1: After sampling, extract the feature map of the sparsely sampled area, and then use the convolution layer of CNN to process it to obtain the preliminary feature map; The preliminary feature map is further processed based on the convolution kernel to obtain local features; Step 2.2: Perform convolutional segmentation on the feature map in each image to obtain the importance score map; The importance score graph is then calculated and segmented to collect the location set of key locations and obtain the key location set; The final global feature is obtained by weighting the key position set; Step 2.3: Concatenate the obtained feature maps of global features and local features, and then use the transformer’s self-attention mechanism to calculate the output feature matrix for each position; Step 2.4: Use the Gaussian mechanism of differential privacy to protect the privacy of the feature matrix and obtain the noise feature matrix .

3. The traffic optimal path planning method based on sparse completion collaborative Q deep learning according to claim 2 is characterized in that: The convolution kernels use convolution kernels with expansion rates of 1, 2, and 3 respectively.

4. The traffic optimal path planning method based on sparse completion collaborative Q deep learning according to claim 2 is characterized in that: The step 2.2 is specifically to obtain n importance score maps by segmenting the feature map in each image along n dimensions using 1×1 convolution, and then calculating and segmenting the n importance score maps to collect a position set of key positions, and applying feature weighting to the key position set to obtain the final global feature.

5. The traffic optimal path planning method based on sparse completion collaborative Q deep learning according to claim 2 is characterized in that: The step 2.4 is specifically for the feature matrix , from the Gaussian distribution Generate a dimensional noise matrix , the feature matrix is protected by the Gaussian mechanism of differential privacy, and the noise feature matrix is obtained .

6. The traffic optimal path planning method based on sparse completion collaborative Q deep learning according to claim 1 is characterized in that: The step three specifically includes: Step 3.1: For the noise feature matrix containing missing data x , select the K feature matrix data closest to the missing data x, and calculate the corresponding weights of these K feature matrix data; Step 3.2: Accumulate and sum the weights to calculate the sum of the weights of the K feature matrix data closest to the missing data x in the i-th traffic map feature matrix , missing data were estimated by weighted average algorithm; Step 3.3: Construct a complete traffic feature matrix based on the missing feature matrix data , the set of traffic map feature matrices is .

7. The traffic optimal path planning method based on sparse completion collaborative Q deep learning according to claim 1 is characterized in that: The gradient clipping threshold is adjusted according to the adjustment factor. Specifically, if the gradient change trend is rising, the clipping threshold will be increased to add more noise; otherwise, the clipping threshold will be lowered to add less noise.

8. The traffic optimal path planning method based on sparse completion collaborative Q deep learning according to claim 1 is characterized in that: The step five specifically includes: Step 5.1: The cloud receives the noisy model parameters from N edge nodes, aggregates multiple edge models using the federated averaging algorithm, and updates the global model parameters. Step 5.2: Use Kalman filter to smooth and optimize the obtained global model parameters, including prediction step and update step; Step 5.3: In the update step, the predicted state is combined with the noise parameter to correct the predicted value, and finally the global model parameters optimized by Kalman filter are obtained; Step 5.4: Use the global parameters optimized by the Kalman filter to update the model and define a noise loss function to reduce the impact of noise on training. Then, use a weighted loss function to comprehensively consider the pre-training loss function and the noise loss function, and optimize the weighted loss function using the gradient descent algorithm. Step 5.5: After Kalman filtering the noise parameters and optimizing the noise loss function, use the updated model to calculate the individual optimal path for each vehicle. Then, optimize the local optimal path planning based on the local value network to obtain the overall optimal path.

Images