Trip Destination Prediction Method Integrating Convolution, Attention, and MLP

Abstract

The present invention proposes a travel destination prediction method that integrates convolution, attention, and MLP, including the following steps: Step S1, extract vehicle travel trajectory data and perform trajectory reconstruction; Step S2, using a feature embedding method, represent the vehicle license plate number, trajectory travel time, and the driving speed, driving direction angle, and time corresponding to the last trajectory point obtained after trajectory reconstruction as feature data representations with equal dimensions, and splice them with the longitude and latitude position features of the trajectory points obtained by trajectory reconstruction, and then perform normalization processing; Step S3, construct a travel destination prediction model that integrates convolution, attention, and a multi-layer perceptron MLP, and calculate the destination feature weight matrix; Step S4, use the MeanShift algorithm to cluster the travel destinations, and perform matrix operations with the feature weight matrix obtained by the destination prediction model to obtain the longitude and latitude coordinate values of the travel destinations. The present invention can improve the accuracy of taxi destination prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation, and in particular to a travel destination prediction method that integrates convolution, attention, and MLP. Background Art

[0002] With the wide application of global positioning system (GPS)-enabled vehicle systems and mobile devices, the massive trajectory data collected by vehicles equipped with positioning devices can be used to study travel mobility. The accumulation of trajectory data provides opportunities for many applications that may rely on vehicle location information, among which destination analysis and prediction are important parts of location prediction. By mining vehicle location data and extracting travel information for travel destination prediction, it is of great significance for ensuring the sustainable development of urban transportation.

[0003] The main travel destination prediction methods at home and abroad are the Markov model based on probability statistics, the prediction method based on ensemble learning algorithms, and the prediction method based on neural networks. The existing destination prediction methods have the following problems: only partial trajectories are utilized, and the information of historical trajectories is not fully extracted; the influence brought by the sparsity of travel trajectory data is not solved. Summary of the Invention

[0004] The present invention proposes a travel destination prediction method that integrates convolution, attention, and MLP, which can improve the accuracy of taxi destination prediction.

[0005] The present invention adopts the following technical solutions.

[0006] A travel destination prediction method that integrates convolution, attention, and MLP includes the following steps;

[0007] Step S1: Extract vehicle travel trajectory data and perform trajectory reconstruction;

[0008] Step S2: Using the feature embedding method, represent the vehicle license plate number, the travel time of the trajectory, and the driving speed, driving direction angle, and time corresponding to the last trajectory point obtained after trajectory reconstruction as feature data with equal dimensions. After splicing with the longitude and latitude position features of the trajectory points obtained by trajectory reconstruction, perform normalization processing;

[0009] Step S3: Construct a travel destination prediction model that integrates convolution, attention, and a multi-layer perceptron (MLP), and calculate the destination feature weight matrix;

[0010] Step S4: Use the MeanShift algorithm to cluster the trajectory destinations, and perform matrix operations with the destination feature weight matrix obtained by the destination prediction model to obtain the longitude and latitude coordinate values of the travel destination.

[0011] The trajectory data reconstruction in step S1 is specifically as follows:

[0012] Step S11: Select the first m trajectory points of the trajectory and the n trajectory points before the destination point, and record the longitude and latitude positions of these m + n trajectory points as the result of the first reconstruction of the trajectory.

[0013] Step S12: Move the destination point forward by one trajectory point, and repeat S11 until the starting point of the n trajectory points before the destination point reaches the (m + 1)th position of the trajectory, and then stop the reconstruction.

[0014] In step S3, construct a trip destination prediction model that combines convolution, attention, and MLP. The specific steps are as follows:

[0015] Step S31: Convert the feature data into the input form of the attention model through convolution.

[0016] Step S32: Use the Convolutional Block Attention Module (CBAM) to learn information in the channel and spatial dimensions, sequentially obtain the channel attention vector and the spatial attention vector along the channel dimension and the spatial dimension respectively, and multiply the obtained feature weights by the input features to complete the adaptive feature optimization.

[0017] Step S33: Use a multi-layer perceptron structure with three hidden layers for non-linear feature learning to obtain the weight matrix of the trajectory destination point.

[0018] In step S31, convert the feature data into the input form of the attention model through convolution. For each batch of training data X ∈ RM×N, the convolution kernel W ∈ R U×V , where U << M, V << N, and the convolution calculation formula is as follows:

[0019]

[0020] where, w uv and x i-u+1,j-v+1 are the elements in W and X respectively;

[0021] In step S32, the CBAM module sequentially obtains the channel attention vector and the spatial attention vector along the channel dimension and the spatial dimension respectively, and then multiplies the obtained feature weights by the input features to complete the adaptive feature optimization. The CBAM contains two branches: channel attention and spatial attention. The calculation formula of the channel attention branch is as follows:

[0022]

[0023] where M c (F) is the generated channel attention feature, F avg is the result of global average pooling, F maxis the result of global maximum pooling. F is the result obtained by convolving the feature data and serves as the input to the channel attention module;

[0024] The calculation formula for the spatial attention branch is as follows:

[0025]

[0026] where M s (F') is the generated channel attention feature, is the result of global average pooling, is the result of global maximum pooling. [] represents the channel concatenation operation, f represents the convolution operation, and F' is the result obtained through the channel branch and serves as the input to the spatial attention branch;

[0027] In step S33, a multi-layer perceptron structure with three hidden layers is used to learn non-linear features. The training process of the feature weights of the multi-layer perceptron is divided into two steps: one is the forward propagation of the signal, and the other is the backpropagation of the error. The forward propagation calculation formula of the multi-layer perceptron is as follows:

[0028]

[0029] where f represents the activation function, is determined by the output of the previous layer, the weight and the bias The neural network continuously adjusts and Finally, the predicted output continuously approaches the target output. The error backpropagation formula of the multi-layer perceptron is as follows:

[0030]

[0031] where the error is calculated backward from the error of the next layer. By calculating the error of the output layer, the errors of all the previous neurons are calculated backward.

[0032] In step S4, the MeanShift clustering algorithm is used to combine the results of step S3 to obtain the predicted destination longitude and latitude coordinates. The specific steps are as follows:

[0033] Step S41: Use the MeanShift clustering algorithm to generate destination clustering points for the trajectories of the selected model training set.

[0034] Step S42: Perform matrix operations on the clustering results and the feature weight matrix obtained in S3 to obtain the final predicted destination longitude and latitude coordinate values.

[0035] In step S4, the feature weight matrix obtained through the MLP and the MeanShift clustering result of the trajectory destination point are subjected to matrix operations to obtain the final predicted destination longitude and latitude coordinate values. The MeanShift calculation formula is as follows:

[0036]

[0037] where x is the center point; x i is the point within the bandwidth; n is the number of trajectory destination points within the bandwidth, G(x) is a unit kernel function, and w(x i ) ≥ 0 represents the weight of each sample.

[0038] The fields of the vehicle trajectory data include license plate number, time, longitude, latitude, vehicle speed, driving direction angle, and operation status; there is only one positioning data for the same vehicle at the same time, and the unique identifier of each data is composed of the vehicle ID and time.

[0039] Through the reconstruction and refined preprocessing of vehicle trajectory data, the present invention effectively overcomes the influence of data sparsity; by integrating convolutional, attention, and MLP prediction models, it predicts the trip destination on the basis of fully extracting each trajectory information, improving the accuracy of taxi destination prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention will be further described in detail below with reference to the drawings and specific embodiments:

[0041] Att Figure 1 is a schematic diagram of trajectory data reconstruction;

[0042] Att Figure 2 is a schematic diagram showing the clustering result of trajectory destination points in the test area;

[0043] Att Figure 3 is a schematic diagram of the trip destination prediction model architecture integrating convolution, attention, and MLP;

[0044] Att Figure 4 is a schematic diagram of the comparison experiment results of weekdays and non - weekdays based on different models;

[0045] Att Figure 5 is a schematic diagram of the comparison experiment results of different completeness trajectories based on different models;

[0046] Att Figure 6 is a schematic diagram of the distance error experiment results based on arbitrary trajectories. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] As shown in the figure, the trip destination prediction method integrating convolution, attention, and MLP includes the following steps;

[0048] Step S1: Extract the vehicle travel trajectory data and perform trajectory reconstruction;

[0049] Step S2: Using the feature embedding method, represent the vehicle license plate number, the travel time of the trajectory, and the driving speed, driving direction angle, and time corresponding to the last trajectory point obtained after trajectory reconstruction as feature data representations with equal dimensions. After splicing with the longitude and latitude position features of the trajectory points obtained by trajectory reconstruction, perform standardization processing;

[0050] Step S3: Construct a travel destination prediction model that integrates convolution, attention, and the multi-layer perceptron MLP, and calculate the destination feature weight matrix;

[0051] Step S4: Use the MeanShift algorithm to cluster the trajectory destinations, and perform matrix operations with the feature weight matrix obtained by the destination prediction model to obtain the longitude and latitude coordinate values of the travel destination.

[0052] The trajectory data reconstruction in step S1 is specifically as follows:

[0053] Step S11: Select the first m trajectory points of the trajectory and the n trajectory points before the destination point, and record the longitude and latitude positions of these m + n trajectory points as the first trajectory reconstruction result.

[0054] Step S12: Move the destination point forward by one trajectory point, and repeat S11 until the starting point of the n trajectory points before the destination point reaches the m + 1 position of the trajectory to stop the reconstruction.

[0055] In step S3, the construction of the travel destination prediction model that integrates convolution, attention, and MLP is specifically as follows:

[0056] Step S31: Through the convolution method, convert the feature data into the input form of the attention model;

[0057] Step S32: Use the convolutional block attention module CBAM to learn information in the channel and spatial dimensions, sequentially obtain the channel attention vector and the spatial attention vector along the channel dimension and the spatial dimension respectively, and multiply the obtained feature weights with the input features to complete the adaptive feature optimization;

[0058] Step S33: Use the multi-layer perceptron structure with three hidden layers to perform non-linear feature learning to obtain the weight matrix of the trajectory destination point.

[0059] In step S31, through convolution, the feature data is converted into the input form of the attention model. For each batch of training data X ∈ R M×N , the convolution kernel W ∈ R U×V , where U << M, V << N, and the convolution calculation formula is as follows:

[0060]

[0061] Among them, w uv and x i-u+1,j-v+1 are elements in W and X respectively;

[0062] In step S32, the CBAM module sequentially obtains the channel attention vector and the spatial attention vector along the channel dimension and the spatial dimension respectively, and then multiplies the obtained feature weights by the input features to complete the adaptive feature optimization. The CBAM includes two branches: channel attention and spatial attention. The calculation formula of the channel attention branch is as follows:

[0063]

[0064] Among them, M c (F) is the generated channel attention feature, F avg is the result of global average pooling, F max is the result of global max pooling, and F is the result obtained by convolving the feature data and serves as the input of the channel attention module;

[0065] The calculation formula of the spatial attention branch is as follows:

[0066]

[0067] Among them, M s (F') is the generated channel attention feature, is the result of global average pooling, is the result of global max pooling, [] represents the channel concatenation operation, f represents the convolution operation, and F' is the result obtained through the channel branch and serves as the input of the spatial attention branch;

[0068] In step S33, a multi-layer perceptron structure with three hidden layers is used to learn non-linear features. The training process of the feature weights of the multi-layer perceptron is divided into two steps: one is the forward propagation of the signal, and the other is the backpropagation of the error. The forward propagation calculation formula of the multi-layer perceptron is as follows:

[0069]

[0070] Among them, f represents the activation function, is determined by the output of the previous layer, the weight and the bias . The neural network continuously adjusts and Finally, the predicted output continuously approaches the target output. The error backpropagation formula of the multi-layer perceptron is as follows:

[0071]

[0072] Among them, the error is calculated by backpropagating the error of the next layer. By calculating the error of the output layer the errors of all the previous neurons are deduced backwards.

[0073] In step S4, the MeanShift clustering algorithm is used in combination with the result of step S3 to obtain the predicted destination longitude and latitude coordinates. The specific steps are as follows:

[0074] Step S41: Apply the MeanShift clustering algorithm to the trajectory destinations of the selected model training set to generate destination clustering points.

[0075] Step S42: Perform matrix operations on the clustering result and the feature weight matrix obtained in S3 to obtain the finally predicted destination longitude and latitude coordinate values.

[0076] In step S4, the feature weight matrix obtained through the MLP and the MeanShift clustering result of the trajectory destination points are used to perform matrix operations to obtain the finally predicted destination longitude and latitude coordinate values. The MeanShift calculation formula is as follows:

[0077]

[0078] Among them, x is the center point; x i is the point within the bandwidth range; n is the number of trajectory destination points within the bandwidth range, G(x i ) greater than or equal to 0 represents the weight of each sample.

[0079] The fields of the vehicle trajectory data include license plate number, time, longitude, latitude, vehicle speed, driving direction angle, and operation status; there is only one positioning data for the same vehicle at the same time, and the unique identifier of each data is composed of the vehicle ID and time.

[0080] Example:

[0081] This example provides a method for predicting trip destinations that combines convolution, attention, and MLP; the specific implementation is as follows:

[0082] ​Step S1: Collect data. The experiment uses Xiamen Island as the research area, and the experimental data used is taxi trajectory data. The trajectory contains data from June 18, 2020 to June 27, 2020, a total of ten days of data, with daily data of about 20 million. Among them, 18, 19, 22, 23, and 24 are working days, 20, 21 (weekends) 25, 26, and 27 (Dragon Boat Festival holidays) are non-working days. The original taxi trajectory data is shown in Table 1, where RUNNING_STATUS represents the operating status, including 1 (empty car), 2 (passenger), 4 (call), 8 (out of service), 16 (shift), and 32 (charter). The taxi trajectory data records the coordinates and status of the taxi at a certain time. The sample is shown in Table 1. The fields include license plate number, time, longitude, latitude, speed, driving direction angle, and operating status, a total of 7 fields. The same vehicle can only have one positioning data at the same time, so the vehicle ID and time constitute the unique identifier of each data, and the license plate number is displayed as desensitized characters.

[0083] Step S2: First, extract each trajectory based on the original trajectory point data and sort them based on time and license plate number. Then, according to the geographic spatial correlation, filter the experimental area data based on the trajectory destination, and eliminate abnormal data in the trajectory data driving direction angle, speed and time.

[0084] Step S3: Map the driving direction angle, speed, license plate number and time of the taxi to feature values, and convert the training set and test set into a form that is easier to train and understand. The original license plate number is composed of some random letters and numbers. It is difficult to directly perform numerical operations on the string during training. It is necessary to map the speed direction angle, speed, and license plate number to corresponding unique integer values, and map the starting time of the trajectory to the corresponding features of the day of the week, the week of the year, and the time of the day.

[0085] Step S4: This paper uses trajectory reconstruction to fully extract the information of each trajectory, considering the importance of the trajectories near the start and end points of each trajectory to the destination prediction, and also considering the process information of each trajectory. By expanding the amount of trajectory data through trajectory reconstruction, the information of each trajectory is retained, effectively overcoming the impact of data sparsity on the prediction of the trip destination. The effectiveness of the method is verified by different trajectory proportions. The specific steps of trajectory truncation are as follows:

[0086] 1) Calculate the number of trajectory points for each trajectory (the trajectory does not include the final destination).

[0087] 2) Truncate each trajectory. Set m and n as the flags to stop trajectory truncation. Fix the first m points of the trajectory, and select the last n points (excluding the destination point of the trajectory) as the result of the first truncation of the trajectory. Move the entire last n points forward by one trajectory point and combine them with the fixed first m points as the result of the second truncation of the trajectory. Sequentially move the last n points of the trajectory forward in this way until the starting point of the last n points reaches the m+1 position of the trajectory to stop truncation.

[0088] 3) Figure 1 Taking a trajectory with 16 trajectory points as an example, m = 5 and n = 5 are the parameters for the final selected trajectory interception, and a total of 6 new trajectory segments are intercepted. The new trajectory segments obtained through trajectory reconstruction are part of the original trajectory. Based on the different positions intercepted, they contain different spatial information of the original trajectory, reducing the impact on destination prediction by only using the starting part and the ending part of the trajectory while ignoring the intermediate process information, and improving the accuracy and reliability of model prediction. Taking No. 18 as an example, after processing, the original data of about 200,000 pieces is amplified to about 4 million pieces, and on average, each original trajectory is amplified by about 20 new trajectory segments.

[0089] Step S5: Based on the data preprocessing and trajectory amplification in steps 1-4, the input features for the destination prediction model are obtained, including: the longitude and latitude position features of the first m points and the last n points of the trajectory, as well as the driving direction angle, driving speed, license plate number, time feature corresponding to the last point of the trajectory, and the driving time of each trajectory.

[0090] Step S6: Debug the model parameters, including the learning rate, the size of the clustering bandwidth of the trajectory destination point, the selection of the optimizer, batch_size, Embedding size, etc., to determine the optimal model training parameters.

[0091] Step 7: Determine the model input data and parameters through steps 1-6, and perform the prediction of the travel trajectory destination based on the fusion convolutional, attention, and MLP prediction model. The model architecture is as Figure 3 , furthermore, the fusion convolutional, attention, and MLP prediction model structure mainly includes three major modules: model feature embedding, splicing and convolution, attention module, and MLP module.

[0092] Step 71: Feature embedding and concatenation. Feature embedding has the ability to jointly learn the embedding of feature data. First, randomly initialize the original features to be embedded, with the initialized dimension set to 10. Store the dimension of each meta-feature in advance. During the feature embedding process, each time a piece of data is taken, perform a look-up table operation on each feature data according to the mapping table to obtain the Embedding result, and then concatenate the results of each feature after Embedding. After the data undergoes feature embedding and feature concatenation, normalization is required. Features have different dimensions and dimension units, and the different-scale data distributions among different features will affect the model training results. To eliminate the dimensionality impact between features, it is necessary to perform standardization processing on the data so that the features are at the same order of magnitude. At the same time, use convolution for dimension conversion to obtain the input form of the attention module.

[0093] Step 72: Attention module. The attention module selects two types of attention and conducts experimental comparisons using channel domain attention (ECA and SE) and hybrid domain attention (CBAM and BAM) respectively. The attention module can remember more important information that helps with auxiliary judgment and ignore some irrelevant information, that is, it can focus on important information with high weights and ignore irrelevant information with low weights. Verify the impact of different attentions on the destination prediction result through channel domain attention SE and ECA and hybrid domain attention CBAM and BAM, and specifically illustrate using SE channel domain attention and CBAM hybrid domain attention respectively.

[0094] 1) The SE (Squeeze-and-Excitation) attention module mainly contains two important operations, Squeeze and Excitation. The main steps combined with this model are as follows:

[0095] The first step is Squeeze(F sq ): Given an input: U ∈ R H×W×C , let it undergo a transformation through F sq to obtain an output: Z ∈ R 1×1×C . The calculation formula is as follows:

[0096]

[0097] Adopt global average pooling to compress the corresponding spatial information on each channel into a single value in the corresponding channel. At this time, one pixel represents one channel, and the final dimension is converted into a vector of 1×1×C.

[0098] The second step is Excitation(F ex ): Given an input: Z ∈ R 1×1×C , let it undergo a transformation through Fex Get an output: S ∈ R 1×1×C , and the calculation formula is as follows:

[0099] S = F ex (Z, W) = σ(g(z, w)) = σ(w2δ(w1z))

[0100] Pass the Z obtained in the first step through two fully connected layers, namely w1 and w2 in the formula, to improve the non-linearity of the model. Here, δ is the activation function Relu, and σ is the activation function sigmoid. Finally, the weight value result S is obtained.

[0101] 2) CBAM (Convolutional Block Attention Module) belongs to hybrid-domain attention and is an attention module that includes spatial and channel branches. The specific process is as follows:

[0102] Channel attention branch: Given an input: F ∈ R H×W×C , and simultaneously pass through global average pooling and global max pooling operations based on width and height to obtain different spatial semantic description operators respectively. Pass the two through a shared perceptron (a two-layer neural network). The number of neurons in the first layer is C / r (r is the reduction rate), and the activation function is ReLU. The number of neurons in the second layer is C, and this two-layer neural network is shared. Subsequently, add the two channel attention feature vectors through matrix addition and fusion, and obtain the channel attention vector M c ∈ R C×1×1 , and the calculation formula is as follows:

[0103]

[0104] Spatial attention branch: The spatial attention module takes the output feature map of the channel attention module as the input feature map of this module. Pass through global average pooling and global max pooling operations simultaneously along the channel dimension to make up for the insufficient information embedding caused by a single pooling operation, and obtain two different channel feature description operators respectively. Then splice the two, and perform a convolution operation with a 7*7 convolution kernel to achieve dimensionality reduction and increase the receptive field. The dimensionality is reduced to H×W×1, and then pass through the sigmoid activation function to obtain the spatial attention feature vector M s (F), and the calculation formula is as follows:

[0105]

[0106] Among them, f represents the convolution operation, [] represents the channel concatenation operation, and finally, a multiplication operation is performed on this vector and the input feature map of this module. The structure adopts a series connection form. The original feature map first passes through the channel attention module to obtain channel attention, and then passes through the spatial attention module to obtain spatial attention.

[0107] Step 73: MLP module. The MLP structure has a great impact on destination prediction by setting different numbers of hidden layers. Theoretically speaking, the more hidden layers are constructed, the stronger the non-linear learning ability of the multi-layer perceptron. If the number of given hidden layers is small, the non-linear learning ability for features is weak. Among them, the setting of neurons in each hidden layer is also one of the influencing factors. Currently, there is no definite theory to determine the selection of the number of hidden layers. Adjustment based on the experimental process is the best way to determine the hidden layers. In the experiment, setting too many hidden layers, although having a strong non-linear learning ability for features, mostly learns high-dimensional information generated, which is instead not good for the destination prediction result. While selecting a small number of hidden layers has insufficient non-linear learning ability for features and cannot learn good feature information, which will also lead to a poor destination prediction result. Based on the experiment, it is finally determined that selecting three hidden layers can better learn feature information and improve the destination prediction accuracy.

[0108] Step 8: Trajectory destination point clustering. As Figure 2 Figures 9(a) and 9(b) show the results before and after clustering the destination points of taxi trajectories using the MeanShift clustering algorithm. The clustering process of the trajectory destination points first selects the destination points of all training sets and sets the bandwidth of clustering. The larger the bandwidth, the fewer the clustering points obtained by clustering, and the overall distribution of the clustering points is sparse but the training parameters are few and the training time is short. If the bandwidth is set smaller, the generated clustering points are denser, but it does not mean that the prediction result is more accurate. After experiments, a suitable bandwidth of 0.001 is selected, and 1743 clustering points are obtained.

[0109] Step 9: Weighted average of destination prediction results. The value to be finally predicted consists of two scalar values (longitude and latitude). However, directly predicting longitude and latitude is relatively difficult. Therefore, the clustering algorithm is used to cluster the destinations of the training set to obtain the clustering results of the trajectory destination points. The features finally obtain a weight matrix after being trained in Step 7. The obtained weight matrix and the clustering results of the trajectory destination points are subjected to matrix operations to obtain the destination prediction values of each trajectory. The formula for calculating the weighted average of the destination cluster center is:

[0110]

[0111] where C represents the number of clusters, c i represents the longitude and latitude of each cluster center, p i is the probability that the network predicts each destination, is the predicted result. The evaluation index uses an approximate equal rectangle formula to calculate the distance between two locations, and the calculation formula is:

[0112]

[0113] where and λ x are the latitude and longitude of point x respectively, and λ y are the latitude and longitude of y respectively.

[0114] Step 10: Based on the processes of Steps 1 - 9, select the SE, ECA, BAM, and CBAM attention modules to conduct a comparative experiment between weekdays and non - weekdays. The comparison results are as Figure 4 . Conduct a statistical analysis on the results obtained from processing taxi trajectories to obtain the operation quantity and average operation time of taxis in each period. It can be seen that there are differences in the operation quantity of taxis between weekdays and non - weekdays mainly during the morning rush hour for work, and the average operation time has differences during the morning rush hour and the evening rush hour. The data distributions of the two are different. Through the destination prediction model, relatively high and similar prediction accuracies are obtained for both weekdays and non - weekdays, reflecting the strong generalization ability of the fusion convolutional, attention, and MLP prediction models. From the experimental comparison between the hybrid - domain attention module and the channel attention module, it can be seen that the predicted distance error of the prediction model based on the hybrid - domain attention module is about 1.5 km, and the predicted distance error of the prediction model based on the channel attention module is about 1.6 km. The experimental results show that different attention modules will have an impact on the destination prediction results, and the prediction accuracy of the model using the hybrid - domain attention module is higher than that of the model using the channel attention module.

[0115] Step 14: Comparison of different trajectory completeness. Figure 5 is the experimental result of destination prediction based on different trajectory completeness. Taking the non - weekday dataset (20, 21, 25, 26, 27) as an example, 20, 21, 25 are the training sets, 26 is used as the validation set, and 27 is used as the test set. Experiments are conducted using the first ten trajectory points and trajectories with a completeness of 40%, 60%, and 80% of each first 40% respectively. Figure 6These are experimental results obtained based on arbitrary trajectories. From the results obtained by the two model methods, it can be seen that the overall regularity of the results obtained by the two models is consistent, that is, the prediction accuracy of selecting the first ten-point trajectories is lower than that of the prediction accuracy with a trajectory integrity of 40%, which is lower than that of the prediction accuracy with a trajectory integrity of 60%, which is lower than that of the prediction accuracy with a trajectory integrity of 80%. Among them, the overall distance losses of the predictions based on the trajectories of the first ten points, with an integrity of 40%, 60%, and 80% are 1.516 km, 1.331 km, 1.016 km, and 0.729 km respectively. The average distance error of selecting arbitrary trajectories is 1.07 km. As the provided trajectory integrity is higher, the accuracy of destination prediction also shows an increasing trend. The more complete the trajectory is, the more trajectory information is provided, which is conducive to the model learning more feature rules from more information and improving the accuracy of destination prediction. In real life, different application services can be provided according to different trajectory integrities.

[0116] Table 1 shows the partial trajectories of the original vehicles in the experimental area

[0117] Table 1

[0118]

[0119]

Claims

1. A travel destination prediction method that integrates convolution, attention, and MLP, characterized in that: It includes the following steps; Step S1: Extract vehicle travel trajectory data and perform trajectory reconstruction; Step S2: Using the feature embedding method, represent the vehicle license plate number, trajectory travel time, and driving speed, driving direction angle, and moment corresponding to the last trajectory point obtained after trajectory reconstruction as feature data with equal dimensions. After splicing with the longitude and latitude position features of the trajectory points obtained by trajectory reconstruction, perform standardization processing; Step S3: Build a travel destination prediction model that integrates convolution, attention, and multi-layer perceptron MLP, and calculate the destination feature weight matrix; Step S4: Use the MeanShift algorithm to cluster the trajectory destinations, and perform matrix operations with the feature weight matrix obtained by the destination prediction model to obtain the longitude and latitude coordinate values of the travel destination; The trajectory data reconstruction in step S1 is specifically as follows: Step S11: Select the first m trajectory points of the trajectory and n trajectory points before the destination point, and record the longitude and latitude positions of these m + n trajectory points as the first trajectory reconstruction result; Step S12: Move the destination point forward by one trajectory point, and repeat S11 until the starting point of the n trajectory points before the destination point reaches the m + 1 position of the trajectory and stops reconstruction; In step S3, the construction of the travel destination prediction model that integrates convolution, attention, and MLP is specifically as follows: Step S31: Convert the feature data into the input form of the attention model through the convolution method; Step S32: Use the Convolutional Block Attention Module (CBAM) to learn information in the channel and spatial dimensions, sequentially obtain the channel attention vector and spatial attention vector along the channel dimension and spatial dimension respectively, and multiply the obtained feature weights with the input features to complete adaptive feature optimization; Step S33: Use a multi-layer perceptron structure with three hidden layers to perform non-linear feature learning to obtain the weight matrix of the trajectory destination point; In step S4, the MeanShift clustering algorithm is used to combine the results of step S3 to obtain the predicted longitude and latitude coordinate values of the destination, specifically as follows: Step S41: Use the MeanShift clustering algorithm to generate destination cluster points for the trajectory destinations in the selected model training set; Step S42: Perform matrix operations on the clustering results and the feature weight matrix obtained in S3 to obtain the finally predicted longitude and latitude coordinate values of the destination.

2. The itinerary destination prediction method integrating convolution, attention, and MLP according to claim 1, characterized in that: In step S31, the feature data is converted into the input form of the attention model through convolution. Each batch of training data X ∈ R M×N , and the convolution kernel W ∈ R U×V , where U << M, V << N, and the convolution calculation formula is as follows: where w uv and x i-u+1,j-v+1 are elements in W and X respectively; In step S32, the CBAM module sequentially obtains the channel attention vector and spatial attention vector along the channel dimension and spatial dimension respectively, and then multiplies the obtained feature weights with the input features to complete adaptive feature optimization. CBAM contains two branches: channel attention and spatial attention. The calculation formula for the channel attention branch is as follows: Among which M c (F) is the generated channel attention feature, F avg is the result of global average pooling, F max is the result of global max pooling, F is the result obtained by convolving the feature data and serves as the input to the channel attention module; The calculation formula for the spatial attention branch is as follows: Among which M s (F') is the generated channel attention feature, which is the result of global average pooling, which is the result of global max pooling, [] represents the channel concatenation operation, f represents the convolution operation, and F' is the result obtained through the channel branch as the input of the spatial attention branch; In step S33, a multi-layer perceptron structure with three hidden layers is used to perform non-linear feature learning. The training process of the feature weights of the multi-layer perceptron is divided into two steps: one is the forward propagation of the signal, and the other is the backpropagation of the error. The forward propagation calculation formula of the multi-layer perceptron is as follows: where f represents the activation function, is determined by the output of the previous layer weights and biases. Through continuous training, the neural network adjusts and finally makes the predicted output continuously approach the target output. The error backpropagation formula for the multi-layer perceptron is as follows: Among them, the error is calculated backward from the error of the next layer. By calculating the error of the output layer, the errors of all the previous neurons are deduced backward.

3. The itinerary destination prediction method integrating convolution, attention, and MLP according to claim 1, characterized in that: In step S4, the feature weight matrix obtained by the MLP and the MeanShift clustering result of the trajectory destination point are subjected to matrix operations to obtain the final predicted destination longitude and latitude coordinate values. The MeanShift calculation formula is as follows: Among them, x is the center point; x i is a point within the bandwidth range; n is the number of trajectory destination points within the bandwidth range, G(x) is a unit kernel function, and w(x i ) ≥ 0 indicates the weight of each sample.

4. The itinerary destination prediction method integrating convolution, attention and MLP according to claim 1, characterized in that: The fields of the vehicle travel trajectory data include license plate number, time, longitude, latitude, vehicle speed, driving direction angle, and operation status; there is only one positioning data for the same vehicle at the same time, and the unique identifier of each data is composed of the vehicle ID and time.

Images