Model training method and trajectory prediction method based on multi-dimensional feature fusion

Abstract

The invention provides a multi-dimensional feature fusion-based model training method and a trajectory prediction method, which can be applied to the technical field of artificial intelligence. The model training method comprises the following steps: clustering a plurality of track sequences to obtain track clusters respectively corresponding to a plurality of track behavior rules; performing bidirectional time feature extraction and spatial feature extraction on the plurality of track clusters; performing semantic coding on respective auxiliary semantic information of the plurality of track sequences by utilizing a word embedding model to obtain respective auxiliary feature sets of the plurality of track clusters; on the basis of the incidence relation among the bidirectional time feature sets, the spatial feature sets and the auxiliary feature sets of the multiple track clusters, feature fusion is conducted on the multiple track clusters, the obtained multi-dimensional fusion feature sets are used for training the initial model, and target prediction sub-models corresponding to the multiple track behavior rules are obtained; the misjudgment of prediction only depending on a bidirectional time feature set and a spatial feature set in the prior art on a complex scene is made up.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a model training method and trajectory prediction method based on multi-dimensional feature fusion. Background Art

[0002] Predicting the behavioral trajectory of the target object can assist in decision-making on risk avoidance and order management.

[0003] In the process of realizing the concept of the present invention, it was found through research that the prediction models in related technologies are difficult to accurately explore the behavior patterns of complex and changeable target objects, resulting in low prediction accuracy. Summary of the Invention

[0004] In view of this, the present invention provides a model training method and a trajectory prediction method based on multi-dimensional feature fusion.

[0005] One aspect of the present invention provides a model training method based on multidimensional feature fusion, comprising:

[0006] Clustering is performed on multiple trajectory sequences to obtain trajectory clusters corresponding to the multiple trajectory behavior patterns. Spatiotemporal features are extracted from the multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets for each of the multiple trajectory clusters, where the bidirectional temporal feature set includes forward features and backward features in the temporal dimension, and the spatial feature set includes features in the spatial dimension. A word embedding model is used to semantically encode the auxiliary semantic information of each of the multiple trajectory sequences to obtain an auxiliary feature set for each of the multiple trajectory clusters. Based on the correlation between the bidirectional temporal feature sets, spatial feature sets, and auxiliary feature sets of each of the multiple trajectory clusters, feature fusion is performed on each of the multiple trajectory clusters to obtain a multidimensional fused feature set for each of the multiple trajectory clusters. The initial model is trained using the multidimensional fused feature sets of each of the multiple trajectory clusters until the model parameters meet predetermined convergence conditions, thereby obtaining a target prediction model, where the target prediction model includes target prediction sub-models corresponding to each of the multiple trajectory behavior patterns.

[0007] According to an embodiment of the present invention, clustering is performed on multiple trajectory sequences to obtain trajectory clusters corresponding to multiple trajectory behavior patterns, including: segmenting the multiple preprocessed trajectory sequences based on a multidimensional predetermined strategy to obtain multiple target trajectory subsequences; extracting features from the multiple target trajectory subsequences based on a sliding window to obtain feature sequences corresponding to the multiple target trajectory subsequences; and clustering the multiple trajectory sequences based on the multiple feature sequences to obtain multiple trajectory clusters corresponding to the multiple trajectory behavior patterns.

[0008] According to an embodiment of the present invention, spatiotemporal feature extraction is performed on multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets for each of the multiple trajectory clusters. The method includes: using a multi-layer convolutional neural network to extract features in the spatial dimension of the multiple trajectory clusters to obtain spatial feature sets for each of the multiple trajectory clusters; using a bidirectional long short-term memory network to extract bidirectional features in the temporal dimension of multiple target trajectory subsequences to obtain bidirectional temporal features for each of the multiple target trajectory subsequences; and concatenating the bidirectional temporal features of the multiple target trajectory subsequences to obtain bidirectional temporal feature sets for each of the multiple trajectory clusters. The length of the bidirectional temporal features in the bidirectional temporal feature sets of each of the multiple trajectory clusters is equal to the length of the spatial features in the spatial feature sets.

[0009] According to an embodiment of the present invention, the multidimensional predetermined strategy includes a speed strategy and a heading strategy, the trajectory sequence includes multiple trajectory point data, the trajectory point data includes heading angle data and acceleration data, and the target trajectory subsequence includes a variable speed linear trajectory subsequence, a uniform speed linear trajectory subsequence, a variable speed non-linear trajectory subsequence or a uniform speed non-linear trajectory subsequence; based on the multidimensional predetermined strategy, the multiple pre-processed trajectory sequences are cut and processed respectively to obtain multiple target trajectory subsequences, including: for each pre-processed trajectory sequence, based on the heading strategy and the multiple heading angle data of the pre-processed trajectory sequence, the trajectory sequence is cut to obtain a linear trajectory subsequence and a non-linear trajectory subsequence corresponding to the trajectory sequence; based on the speed strategy, the multiple acceleration data of the linear trajectory subsequence and the multiple acceleration data of the non-linear trajectory subsequence, the linear trajectory subsequence and the non-linear trajectory subsequence are cut respectively to obtain at least one of a variable speed linear trajectory subsequence, a uniform speed linear trajectory subsequence, a variable speed non-linear trajectory subsequence or a uniform speed non-linear trajectory subsequence.

[0010] According to an embodiment of the present invention, a bidirectional long short-term memory network includes a first network, a second network, a third network, and a fourth network; using the bidirectional long short-term memory network, bidirectional feature extraction is performed on multiple target trajectory subsequences in the time dimension, and the bidirectional time features of the multiple target trajectory subsequences are respectively spliced ​​to obtain bidirectional time feature sets of multiple trajectory clusters, including: inputting the speed-changing linear trajectory subsequence into the first network, extracting the time sequence and time reversal features of the speed-changing linear trajectory subsequence, and obtaining the bidirectional time features of the speed-changing linear trajectory subsequence; inputting the uniform speed linear trajectory subsequence into the second network, and extracting the time sequence and time reversal features of the uniform speed linear trajectory subsequence. The bidirectional time features of the uniform speed linear trajectory subsequence are obtained; the uniform speed non-linear trajectory subsequence is input into the third network, and the time sequence and time reverse feature extraction are performed on the uniform speed non-linear trajectory subsequence to obtain the bidirectional time features of the uniform speed non-linear trajectory subsequence; the variable speed non-linear trajectory subsequence is input into the fourth network, and the time sequence and time reverse feature extraction are performed on the variable speed non-linear trajectory subsequence to obtain the bidirectional time features of the variable speed non-linear trajectory subsequence; the bidirectional time features of the variable speed linear trajectory subsequence, the uniform speed linear trajectory subsequence, the variable speed non-linear trajectory subsequence, and the uniform speed non-linear trajectory subsequence are projected into the same dimensional space to obtain the bidirectional time feature sets of multiple trajectory clusters.

[0011] According to an embodiment of the present invention, a multi-layer convolutional neural network includes a cascaded first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer; using the multi-layer convolutional neural network, spatial feature extraction is performed on multiple trajectory clusters to obtain spatial feature sets for each of the multiple trajectory clusters, including: converting multiple trajectory sequences into multiple trajectory images respectively; using the first convolutional layer and the first pooling layer to perform local shallow feature extraction on the multiple trajectory images to obtain a first feature map; using the second convolutional layer and the second pooling layer to perform global deep feature extraction on the first feature map to obtain spatial feature sets for each of the multiple trajectory clusters.

[0012] According to an embodiment of the present invention, a word embedding model is used to semantically encode the auxiliary semantic information of multiple trajectory sequences to obtain auxiliary feature sets for multiple trajectory clusters. The method includes: mapping the auxiliary semantic information of each of the multiple trajectory sequences from text form to feature vectors using an embedding lookup table or the embedding layer of a word embedding model; and concatenating the feature vectors corresponding to the auxiliary semantic information of each of the multiple trajectory sequences to obtain auxiliary feature sets for each of the multiple trajectory clusters.

[0013] According to an embodiment of the present invention, based on multiple feature sequences, multiple trajectory sequences are clustered to obtain trajectory clusters corresponding to multiple trajectory behavior patterns, including: using a target deep autoencoder to compress and reconstruct the multiple feature sequences to obtain a low-dimensional reconstructed sequence, wherein the target deep autoencoder is obtained by adjusting the parameters of the initial autoencoder based on a regularization norm, and the regularization norm is determined based on the reconstruction error between the reconstructed data and the sample data, and the reconstructed data is obtained by inputting the sample data into the initial autoencoder; using the low-dimensional reconstructed sequence, the multiple trajectory sequences are clustered to obtain trajectory clusters corresponding to the multiple trajectory behavior patterns.

[0014] Another aspect of the present invention provides a trajectory prediction method based on multi-dimensional feature fusion, comprising:

[0015] Based on the dynamic time warping algorithm, the target trajectory behavior pattern corresponding to the trajectory sequence to be predicted is determined. Based on the target trajectory behavior pattern, the target prediction sub-model corresponding to the trajectory sequence to be predicted is called from the target prediction model, wherein the target prediction model is trained according to the above-mentioned model training method. The trajectory data of the trajectory sequence to be predicted in the current time period is input into the target prediction sub-model to obtain the target prediction trajectory corresponding to the predetermined time period after the current time period.

[0016] Another aspect of the present invention provides an electronic device, comprising:

[0017] one or more processors;

[0018] a memory for storing one or more programs,

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0020] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0021] Another aspect of the present invention provides a computer program product, comprising computer executable instructions, which are used to implement the method described above when the instructions are executed.

[0022] According to an embodiment of the present invention, multiple trajectory sequences are first clustered to group trajectory sequences with similar trajectory behavior patterns into the same trajectory cluster. Then, spatiotemporal feature extraction is performed on the multiple trajectory clusters to obtain the features of the trajectory sequences in the spatial dimension, as well as the forward and backward features in the temporal dimension. This captures the changes in the temporal and spatial dimensions of specific trajectory behavior patterns and embeds the semantic information of the trajectory sequences to obtain an auxiliary feature set. Based on the correlation between the bidirectional temporal feature set, the spatial feature set, and the auxiliary feature set, multi-source feature fusion is performed to obtain a multidimensional fused feature set with spatiotemporal correlation corresponding to different trajectory behavior patterns. The multidimensional fused feature set is then used for training until a predetermined convergence condition is met, thereby obtaining target prediction sub-models corresponding to different trajectory behavior patterns. This allows the corresponding target prediction sub-model to be selected for prediction based on the trajectory behavior pattern of the trajectory to be predicted, effectively improving the prediction accuracy and better meeting the complex and changing trajectory prediction needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0024] Figure 1 The flowchart of the model training method based on multi-dimensional feature fusion according to an embodiment of the present invention is schematically shown.

[0025] Figure 2 The figure schematically shows a dual-path feature fusion according to an embodiment of the present invention.

[0026] Figure 3 The diagram schematically shows a schematic diagram of clustering multiple trajectory sequences according to an embodiment of the present invention.

[0027] Figure 4 A schematic diagram of multi-dimensional feature extraction and fusion in model training according to an embodiment of the present invention is schematically shown.

[0028] Figure 5 The flowchart of a trajectory prediction method based on multi-dimensional feature fusion according to another embodiment of the present invention is schematically shown.

[0029] Figure 6 A block diagram of an electronic device suitable for implementing a model training method and a trajectory prediction method based on multi-dimensional feature fusion according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0030] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0031] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the presence of the features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0032] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0033] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0034] During the implementation of the present invention, research revealed that existing methods for predicting target trajectory are primarily based on two perspectives: data-driven and behavior-driven. Data-driven methods primarily utilize massive amounts of historical trajectory data to uncover behavioral characteristics of mobile targets, which are then integrated and matched with current location data to predict the target's motion trends. Behavior-driven methods, on the other hand, predict the target's trajectory within a certain timeframe based on its relevant motion characteristics. These methods can generally be categorized into two main categories: dynamic models and intent recognition.

[0035] However, data-driven models require a large number of samples for model learning, and feature extraction relies heavily on the model's own learning iterations, making it difficult to quickly adapt to dynamic behavioral changes and analyze behavioral intent. Behavior-driven prediction models are limited to specific scenarios with clear intent, and suffer from low prediction accuracy in real-world applications.

[0036] In view of this, embodiments of the present invention provide a model training method for trajectory prediction. This method clusters multiple trajectory sequences to obtain trajectory clusters corresponding to multiple trajectory behavior patterns. This method breaks the technical barrier of the related art to unified modeling of multiple trajectory behavior patterns and performs bidirectional temporal and spatial feature extraction on multiple trajectory clusters. That is, in the temporal dimension, not only can future trajectories be inferred from prior trajectories, but prior trajectories can also be inferred from future trajectories. This improves the accuracy of feature extraction for complex trajectories. At the same time, auxiliary semantic information is embedded through semantic encoding, which compensates for the misjudgment of complex scenes caused by the related art relying solely on bidirectional temporal and spatial feature sets. At the same time, in the process of multi-source feature fusion, it no longer relies solely on fixed weight distribution or simple splicing, but simultaneously considers the correlation between the bidirectional temporal feature set, spatial feature set, and auxiliary feature set, ensuring that the obtained multi-dimensional fusion feature set is more comprehensive, thereby improving the training accuracy of the initial model and ensuring that each prediction sub-model focuses on its own behavior pattern during the learning process, avoiding mutual interference between the features of different trajectory behavior patterns. As a result, the obtained target prediction sub-models corresponding to multiple trajectory behavior patterns have better robustness and prediction accuracy.

[0037] Figure 1 The flowchart of the model training method based on multi-dimensional feature fusion according to an embodiment of the present invention is schematically shown.

[0038] like Figure 1 As shown, the model training method 100 includes operations S110 to S150.

[0039] In operation S110 , clustering is performed on the plurality of trajectory sequences to obtain trajectory clusters corresponding to the plurality of trajectory behavior patterns.

[0040] According to an embodiment of the present invention, multiple trajectory sequences can come from different areas (including but not limited to at sea or in the air) and different types of target objects (such as aircraft and ships), and specifically include multiple trajectory point data arranged in chronological order. The trajectory point data can include static attribute data (such as longitude and latitude) and dynamic attribute data (such as acceleration, speed, and heading angle).

[0041] Clustering in related technologies focuses solely on the path shape of trajectories through static attributes, without considering dynamic attributes. This can lead to overly limited or overly broad clustering results. Therefore, we identify trajectory behavior patterns based on data from multiple trajectory points, clustering trajectories with similar behavior patterns into the same trajectory cluster.

[0042] Trajectory behavior patterns can include: direct flight, circling, circling, climbing, landing, turning, hovering, anchoring, escorting, etc. Therefore, it is difficult to distinguish between circling and hovering for clustering that does not consider dynamic attributes. This results in sample trajectories of hovering and circling being misclassified into the same cluster, affecting subsequent prediction accuracy.

[0043] In operation S120, spatiotemporal features are extracted from the multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets of the multiple trajectory clusters. The bidirectional temporal feature set includes forward features and backward features in the temporal dimension, and the spatial feature set includes features in the spatial dimension.

[0044] According to an embodiment of the present invention, spatiotemporal feature extraction is performed on each of the multiple trajectory clusters generated by clustering to avoid mutual interference between trajectory sequences with different trajectory behavior patterns.

[0045] For example, circling, hovering, and turning all involve heading changes, and mixed extraction may lead to error accumulation.

[0046] According to an embodiment of the present invention, spatiotemporal feature extraction may include feature extraction in the time dimension and feature extraction in the space dimension. For feature extraction in the time dimension, it may include extraction from front to back and extraction from back to front, and at the same time cover local time feature extraction and global time feature extraction.

[0047] In operation S130 , a word embedding model is used to semantically encode the auxiliary semantic information of each of the plurality of trajectory sequences to obtain auxiliary feature sets of each of the plurality of trajectory clusters.

[0048] According to an embodiment of the present invention, the auxiliary semantic information is in text form, including semantic information corresponding to the trajectory sequence, which may include dynamic attribute information (such as kinematic parameters such as speed, acceleration, and heading angle), typical trajectory information (such as typical trajectory types of fighter jets, reconnaissance aircraft, and drones), related descriptive information (such as the equipment model, mission batch number, and typical activity area of ​​the target object), and auxiliary metadata (such as non-content information such as timestamps and data source identifiers). The auxiliary feature set is divided into a dynamic attribute feature set, a typical trajectory feature set, a related description feature set, and an auxiliary metadata feature set.

[0049] Since the auxiliary semantic information currently takes the form of a string of text, a word embedding model is needed to semantically encode the text and convert it into a feature format. Word embedding models can include Word2Vec (a related model used to generate word vectors) or FastText. Furthermore, the initial text may contain redundant symbols or inconsistent formatting, so structural cleaning is required to ensure that the word embedding model can effectively capture the semantics.

[0050] For example, remove punctuation marks from auxiliary semantic information and retain only the core content. Units for acceleration, speed, etc. also need to be unified.

[0051] In operation S140 , based on the association relationships among the bidirectional temporal feature sets, spatial feature sets, and auxiliary feature sets of the plurality of trajectory clusters, feature fusion is performed on the plurality of trajectory clusters to obtain multidimensional fused feature sets of the plurality of trajectory clusters.

[0052] According to an embodiment of the present invention, an adaptive feature interaction network can be used to achieve deep fusion of a bidirectional temporal feature set, a spatial feature set, and an auxiliary feature set. First, based on the association between the bidirectional temporal feature set, the spatial feature set, and the auxiliary feature set, the attention weight of one feature relative to the other two features is obtained, which can be specifically expressed as the following formula (1).

[0053] (1);

[0054] Among them, A mn Represents the target feature H m The target-related feature H n Relative to the target feature H m The attention weight of target feature is m, n is the target associated feature index, and target feature H is m Any feature from the bidirectional temporal feature set, spatial feature set, and auxiliary feature set, H m The target association feature H n and other related features H q With H m Belong to different modes, for example, H m is a typical trajectory feature, H q It can be bidirectional time features, spatial features, dynamic attribute features, related description features and auxiliary meta-features. ( ) represents the similarity function, which is used to calculate the cosine similarity between the target feature and the associated feature. Represents a variable weight parameter.

[0055] Then, based on the above formula (1), the target feature H is obtained as shown in the following formula (2): m The gate vector , the gating vector can reflect the fine-grained weight of a single feature.

[0056] (2);

[0057] in, ( ) represents an activation function, such as the Sigmoid activation function (logistic regression activation function), LeakyReLU( ) represents the LeakyReLU activation function (leaky rectified linear unit activation function), and P is the generation weight matrix of the gating vector.

[0058] According to an embodiment of the present invention, in feature fusion, related technologies only assign an overall weight to features of the same modality, while through the gating vector, it is equivalent to assigning a separate weight to each feature in each modality. Subsequently, for each trajectory cluster, multimodal feature fusion is performed based on the dual path to obtain the multidimensional fusion feature set shown in the following formula (3): .

[0059] (3);

[0060] Among them, LN() represents normalization processing, P' represents the residual transformation weight matrix, Indicates element-by-element multiplication, H skip Represents the mean of all features corresponding to the current trajectory cluster.

[0061] In order to better understand the specific process of dual-path feature fusion, the following will be Figure 2 To give a hint.

[0062] Figure 2 The figure schematically shows a dual-path feature fusion according to an embodiment of the present invention.

[0063] like Figure 2 As shown in the figure, for the trajectory cluster corresponding to each trajectory behavior law, the dynamic attribute feature set, typical trajectory feature set, related description feature set, auxiliary meta-feature set, bidirectional time feature set and spatial feature set are first input into the projection layer to unify the dimension, and then divided into path 1 and path 2 for processing. In path 1, the feature vector after dimension unification passes through the cascaded cross-feature attention layer, adaptive gating layer and gated application layer to obtain the product of the feature vector after dimension unification and its corresponding gate vector, while path 2 uses the feature averager to obtain the mean of each feature vector after dimension unification. Subsequently, the outputs of path 1 and path 2 are weighted aggregated in the path aggregation layer, and the weighted aggregation result is input into the normalization layer to obtain the multidimensional fusion feature set of the trajectory cluster.

[0064] In operation S150 , the initial model is trained using the multi-dimensional fusion feature sets of the plurality of trajectory clusters until the model parameters meet a predetermined convergence condition, thereby obtaining a target prediction model.

[0065] According to an embodiment of the present invention, the initial model can be a Transformer (an attention-based deep learning model architecture), which can efficiently mine features between trajectory contexts based on the attention mechanism while effectively addressing the problem of long-distance dependencies. The Transformer model is trained using feature data from a multidimensional fusion feature set corresponding to different trajectory behavior patterns to obtain corresponding prediction model weight parameters. The model weight parameters are continuously optimized through iterative training until a predetermined number of iterations is reached or L reaches a predetermined threshold, completing the training.

[0066] The initial model still uses an encoder-decoder structure. The encoder consists of a single input layer and multiple encoding layers stacked together. The decoder has a similar structure, consisting of a single input layer and multiple decoding layers stacked together. The encoder's encoding layer includes a multi-head self-attention mechanism and a feedforward network, both of which are followed by residual links and normalization. The decoder's decoding layer includes a multi-head self-attention mechanism (with mask), an encoder-decoder multi-head attention mechanism, and a feedforward network, both of which are followed by residual links and normalization. The three attention mechanisms, residual links, and normalization effectively mitigate potential vanishing or vanishing gradient problems. The multi-head self-attention mechanism (with mask) can mask temporal features by changing the attention weights.

[0067] According to an embodiment of the present invention, the target prediction model is essentially a target prediction sub-model corresponding to each of multiple trajectory behavior patterns. During application, the trajectory behavior pattern closest to the trajectory to be predicted can be determined first, and then the corresponding target prediction sub-model can be called.

[0068] According to an embodiment of the present invention, multiple trajectory sequences are first clustered to group trajectory sequences with similar trajectory behavior patterns into the same trajectory cluster. Then, spatiotemporal feature extraction is performed on the multiple trajectory clusters to obtain the features of the trajectory sequences in the spatial dimension, as well as the forward and backward features in the temporal dimension. This captures the changes in the temporal and spatial dimensions of specific trajectory behavior patterns and embeds the semantic information of the trajectory sequences to obtain an auxiliary feature set. Based on the correlation between the bidirectional temporal feature set, the spatial feature set, and the auxiliary feature set, multi-source feature fusion is performed to obtain a multidimensional fused feature set with spatiotemporal correlation corresponding to different trajectory behavior patterns. The multidimensional fused feature set is then used for training until a predetermined convergence condition is met, thereby obtaining target prediction sub-models corresponding to different trajectory behavior patterns. This allows the corresponding target prediction sub-model to be selected for prediction based on the trajectory behavior pattern of the trajectory to be predicted, effectively improving the prediction accuracy and better meeting the complex and changing trajectory prediction needs.

[0069] According to an embodiment of the present invention, clustering is performed on multiple trajectory sequences to obtain trajectory clusters corresponding to multiple trajectory behavior patterns, including: cutting the multiple preprocessed trajectory sequences based on a multidimensional predetermined strategy to obtain multiple target trajectory subsequences; extracting features from the multiple target trajectory subsequences based on a sliding window to obtain feature sequences corresponding to the multiple target trajectory subsequences; and clustering the multiple trajectory sequences using a clustering algorithm based on the multiple feature sequences to obtain multiple trajectory clusters corresponding to the multiple trajectory behavior patterns.

[0070] Among them, the multi-dimensional booking strategy includes speed strategy and heading strategy.

[0071] According to an embodiment of the present invention, the preprocessing of the trajectory sequence can filter out abnormal trajectory sequences, thereby providing a more reliable data source for subsequent trajectory sequence segmentation and feature extraction.

[0072] Based on the multi-dimensional predetermined strategy, the plurality of pre-processed trajectory sequences are cut and processed respectively to obtain a plurality of target trajectory subsequences, including: for each pre-processed trajectory sequence, firstly based on the heading strategy, according to the heading strategy and the plurality of heading angle data of the pre-processed trajectory sequence, the change of the heading angle data corresponding to each track point in the trajectory sequence is analyzed, thereby cutting the trajectory sequence to obtain a straight trajectory subsequence and a non-straight trajectory subsequence corresponding to the trajectory sequence, wherein the non-straight trajectory subsequence may include a turning trajectory subsequence; then based on the speed strategy, the plurality of added straight trajectory subsequences are cut and processed. The changes in speed data and multiple acceleration data of the non-linear trajectory subsequence are used to determine whether the linear trajectory subsequence and the non-linear trajectory subsequence are in acceleration, deceleration or uniform speed, so as to cut the linear trajectory subsequence and the non-linear trajectory subsequence respectively, divide the linear trajectory subsequence into a variable speed linear trajectory subsequence and a uniform speed linear trajectory subsequence, and divide the non-linear trajectory subsequence into a variable speed non-linear trajectory subsequence and a uniform speed non-linear trajectory subsequence, and finally obtain at least one of the variable speed linear trajectory subsequence, the uniform speed linear trajectory subsequence, the variable speed non-linear trajectory subsequence and the uniform speed non-linear trajectory subsequence.

[0073] That is, based on the multi-dimensional pre-defined strategy, each pre-processed trajectory sequence can be divided into one or more types of target trajectory sub-sequences.

[0074] Figure 3 The diagram schematically shows a schematic diagram of clustering multiple trajectory sequences according to an embodiment of the present invention.

[0075] like Figure 3As shown in the figure, the quality of the initial trajectory sequences varies, and the time intervals between each trajectory point in not all trajectory sequences are equal. Some are too large, and some are too small. Therefore, low-quality trajectory sequences need to be deleted during the preprocessing process of the preprocessing layer to prevent these useless data from affecting the accuracy of the subsequent model.

[0076] Subsequently, the pre-processed multiple trajectory sequences can be cut using a multi-dimensional predetermined strategy to obtain multiple target trajectory subsequences.

[0077] According to an embodiment of the present invention, multiple target trajectory subsequences corresponding to multiple trajectory sequences are input into a feature extraction layer to extract movement behavior features, and each target trajectory subsequence is converted into a feature sequence based on a sliding window.

[0078] According to an embodiment of the present invention, based on multiple feature sequences, multiple trajectory sequences are clustered to obtain trajectory clusters corresponding to multiple behavioral patterns, including: using a target deep autoencoder to compress and reconstruct the multiple feature sequences to obtain a low-dimensional reconstructed sequence, wherein the target deep autoencoder is obtained by adjusting the parameters of the initial autoencoder based on a regularization norm, and the regularization norm is determined based on the reconstruction error between the reconstructed data and the sample data, and the reconstructed data is obtained by inputting the sample data into the initial autoencoder.

[0079] Because the target object's trajectory operates in a complex environment, the data flow within the feature sequence is highly nonlinear. Therefore, a Deep Autoencoder (DAE) can be used to reduce the dimensionality of the feature sequence. DAE is an unsupervised learning algorithm whose expected output and input dimensions are the same. DAE can be applied to image recognition, noise removal, and data dimensionality reduction, and has derived various functions. Compared to the linear dimensionality reduction achieved by Principal Component Analysis (PCA), DAE has better nonlinear dimensionality reduction effects.

[0080] See Figure 3 The number of neurons in the hidden layers of the encoding phase decreases layer by layer. This indicates that after the feature sequence enters the DAE, it is input into the hidden layers of the encoding phase. Through weight learning, key features are gradually extracted. It is then input into the bottleneck layer and compressed into a low-dimensional nonlinear vector, achieving nonlinear dimensionality reduction. The low-dimensional vector is then gradually restored using the hidden layers of the decoding phase to obtain a low-dimensional reconstructed sequence. This effectively extracts the complex distribution of data in the feature sequence and improves clustering accuracy.

[0081] According to an embodiment of the present invention, in a target deep autoencoder, the relationship between the reconstructed data output by the decoder and the data input by the encoder can be expressed as the following formula (4).

[0082] (4);

[0083] Among them, E() represents the encoding process, D[] represents the decoding process, and X represents the data matrix input by the encoder. Represents the reconstructed data matrix output by the decoder. The connection function between neurons can be a logistic function, that is, the data transmission process between neurons can be expressed as the encoder output result shown in the following formula (5): And the decoder output shown in formula (6) .

[0084] (5);

[0085] (6);

[0086] Among them, x∈X, θ is the parameter matrix of the target deep autoencoder, W is the weight matrix, is the bias matrix corresponding to the encoder, is the bias matrix corresponding to the decoder, logit represents the logistic function, and T is the matrix transpose symbol.

[0087] In fact, X can be composed of two parts, L and S, that is, X=L+S, where L represents the vector that can be well represented by the hidden layer of the autoencoder, and S represents the abnormal vector that is difficult to represent. The error between and X requires the removal of abnormal vectors to ensure that the decoding stage can focus more on the reconstruction of L, that is, to improve the convergence conditions of the target deep autoencoder. Therefore, the regularization norm can be introduced in the training process of the deep initial autoencoder. The objective function of the autoencoder is It can be expressed as the following formula (7).

[0088] (7);

[0089] Among them, λ is a parameter used to adjust the sparsity in S, Represents the L0 norm of S. To simplify the operation, the L1 norm can be used ( ) Regularized convex relaxation (converting non-convex optimization problems into convex optimization problems) instead ,norm is used to optimize the clustering results, that is, it can make the trajectory sequences with the same trajectory behavior law more easily condensed into the same trajectory cluster and separated from the trajectory sequences with other trajectory behavior laws.

[0090] According to an embodiment of the present invention, in order to further prevent clustering from being disturbed by abnormal vectors, the norm of the matrix column vector can be introduced, which can be specifically expressed as the following formula (8).

[0091] (8);

[0092] in, L representing X 2,1 norm, represents the j-th column vector in X, express The L2 norm of , J is the total number of columns of X, represents the j-th column and i-th row vector in X, and I represents the total number of rows in X. One of the predetermined constraints can be expressed as the following formula (9).

[0093] (9);

[0094] Here, st stands for "subject to," meaning subject to constraints. Furthermore, the current intermediate autoencoder can only be determined as the target deep autoencoder after the loss function of the intermediate autoencoder obtained after iterative training converges.

[0095] According to an embodiment of the present invention, the initial autoencoder is trained by introducing a regularization norm to ensure that the obtained target deep autoencoder can effectively filter out abnormal parts, so that the decoding stage focuses on reconstructing normal vectors, that is, the reconstruction error is smaller. At the same time, the regularization norm can ensure that trajectory sequences with the same or similar trajectory behavior patterns are more cohesive, thereby enhancing the clustering effect.

[0096] According to an embodiment of the present invention, spatiotemporal feature extraction is performed on multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets for each of the multiple trajectory clusters. The method includes: using a multi-layer convolutional neural network to extract features in the spatial dimension of the multiple trajectory clusters to obtain spatial feature sets for each of the multiple trajectory clusters; using a bidirectional long short-term memory network to extract bidirectional features in the temporal dimension of multiple target trajectory subsequences to obtain bidirectional temporal features for each of the multiple target trajectory subsequences; and concatenating the bidirectional temporal features of the multiple target trajectory subsequences to obtain bidirectional temporal feature sets for each of the multiple trajectory clusters. The temporal feature dimension in the temporal feature set of each of the multiple trajectory clusters is equal to the spatial feature dimension in the spatial feature set.

[0097] According to an embodiment of the present invention, a multi-layer convolutional neural network includes a cascaded first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer; using the multi-layer convolutional neural network, spatial feature extraction is performed on multiple trajectory clusters to obtain spatial feature sets for each of the multiple trajectory clusters, including: converting multiple trajectory sequences into multiple trajectory images respectively; using the first convolutional layer and the first pooling layer to perform local shallow feature extraction on the multiple trajectory images to obtain a first feature map; using the second convolutional layer and the second pooling layer to perform global deep feature extraction on the first feature map to obtain spatial feature sets for each of the multiple trajectory clusters.

[0098] According to an embodiment of the present invention, a one-dimensional trajectory sequence to two-dimensional trajectory image generation algorithm is first used to convert multiple trajectory sequences into multiple trajectory images. Then, the convolution kernel sliding mechanism of a multi-layer convolutional neural network (CNN) is used to extract spatial features at different depths of the trajectory images, capturing spatial correlation information such as the position distribution and motion trend of the target object in the trajectory image.

[0099] According to an embodiment of the present invention, the trajectory image is first input into the first convolutional layer Cov1 to perform local shallow spatial feature extraction. The output of each neuron in the first convolutional layer can be expressed as the following formula (10).

[0100] (10);

[0101] in, represents the output result of the neuron in the cth output channel, αth row, and βth column in Cov1. σ(⋅) represents the activation function, which can be RELU (Rectified Linear Unit). For the input Y, there can be . L represents the side length of the convolution kernel, represents the convolution kernel parameter, k represents the convolution step size, Represents the pixel of the input trajectory image, b c represents the bias term of the c-th output channel, v and are the coordinate indices of the convolution kernel on rows and columns, respectively, and .

[0102] According to an embodiment of the present invention, it can be seen from the above formula (10) that for neurons at various positions in the first convolutional layer, the convolution kernel slides on the input, traverses all positions within the kernel, and multiplies the parameters of each position with the input pixel, accumulates them, adds a bias term, and then processes them with an activation function to retain useful information and suppress invalid information.

[0103] According to an embodiment of the present invention, the first pooling layer Pooling1 samples the output of the first convolutional layer Cov1 to generate a larger proportion of higher-level features to obtain a first feature map. The output of each neuron in the first pooling layer can be expressed as the following formula (11).

[0104] (11);

[0105] in, represents the output result of the neuron in the cth output channel, αth row, and βth column in Pooling1, represents the output result of the neuron in the cth output channel, α·k+vth row, and β·k+ωth column in Cov1, and t is the size of the pooling window of Pooling1.

[0106] According to an embodiment of the present invention, it can be seen from the above formula (11) that the function of the first pooling layer is to extract the local maximum value from the output of the first convolutional layer.

[0107] According to an embodiment of the present invention, the first pooling layer may be followed by a second convolutional layer Cov2 and a second pooling layer Pooling2 for extracting global deep features. The processing of the output of the first pooling layer by the second convolutional layer Cov2 and the second pooling layer Pooling2 can refer to the first convolutional layer Cov1 and the first pooling layer Pooling1, which will not be repeated here.

[0108] According to an embodiment of the present invention, the second feature map output by the second pooling layer Pooling2 needs to be flattened to convert the multi-dimensional second feature map into a spatial feature set in the form of a one-dimensional vector.

[0109] According to an embodiment of the present invention, a bidirectional long short-term memory network is used to extract bidirectional features from multiple target trajectory subsequences in the time dimension. The bidirectional temporal features of the multiple target trajectory subsequences are then concatenated to obtain temporal feature sets for multiple trajectory clusters, including:

[0110] The variable speed linear trajectory subsequence is input into the first network, and the time sequence and time reverse sequence features of the variable speed linear trajectory subsequence are extracted to obtain the bidirectional time features of the variable speed linear trajectory subsequence; the uniform speed linear trajectory subsequence is input into the second network, and the time sequence and time reverse sequence features of the uniform speed linear trajectory subsequence are extracted to obtain the bidirectional time features of the uniform speed linear trajectory subsequence; the uniform speed non-linear trajectory subsequence is input into the third network, and the time sequence and time reverse sequence features of the uniform speed non-linear trajectory subsequence are extracted to obtain the bidirectional time features of the uniform speed non-linear trajectory subsequence; the variable speed non-linear trajectory subsequence is input into the fourth network, and the time sequence and time reverse sequence features of the variable speed non-linear trajectory subsequence are extracted to obtain the bidirectional time features of the variable speed non-linear trajectory subsequence; the bidirectional time features of the variable speed linear trajectory subsequence, the uniform speed linear trajectory subsequence, the variable speed non-linear trajectory subsequence, and the uniform speed non-linear trajectory subsequence are projected into the same dimensional space to obtain the time feature sets of multiple trajectory clusters.

[0111] According to an embodiment of the present invention, the first network, the second network, the third network, and the fourth network respectively correspond to the dimensions of different types of target trajectory subsequences. For example, the first network corresponds to the accelerated straight line trajectory subsequence, which may be 64-dimensional, the second network corresponds to the uniform speed straight line trajectory subsequence, which may be 128-dimensional, and the third network corresponds to the accelerated non-linear trajectory subsequence, which may be 96-dimensional. In addition, the first network, the second network, the third network, and the fourth network each include a forward network and a backward network, which can simultaneously perform forward extraction and backward extraction of temporal features, thereby simultaneously capturing the influence of the trajectory front-end on the trajectory back-end and the influence of the trajectory back-end on the trajectory front-end of the corresponding target trajectory subsequence. Since the time step of each forward network and the backward network represents the number of sequence units processed at one time, in order to ensure that all target trajectory subsequences are fully read and the temporal relationship of the target trajectory subsequences is accurately constructed, it is necessary to ensure that the time step is equal to the number of target trajectory subsequences of the corresponding category to avoid information loss or the introduction of invalid data.

[0112] The bidirectional temporal features of multiple target trajectory subsequences are concatenated to obtain the bidirectional temporal feature set of the corresponding trajectory cluster The process can be expressed as the following formula (12).

[0113] (12);

[0114] in, Representing bidirectional temporal features of subsequences of non-linear trajectories with variable speed The corresponding feature weights, Representation and bidirectional temporal features of variable speed linear trajectory subsequences The corresponding feature weights, Represents the bidirectional temporal characteristics of the uniform linear trajectory subsequence The corresponding feature weights, Represents the feature weight corresponding to the bidirectional time feature of the uniform non-linear trajectory subsequence .

[0115] Feature Weight { , , , }∈G, G can be 256-dimensional, and G can be calculated as follows (13).

[0116] (13);

[0117] Among them, || represents feature splicing, represents the weight matrix corresponding to the feature weight, and Ω represents the bias matrix corresponding to the feature weight.

[0118] In addition, the channel length of the bidirectional long short-term memory network can match the length of the one-dimensional vector in the above-mentioned spatial feature set to balance the weights of subsequent feature fusion, that is, the vector length of the temporal feature in the obtained bidirectional temporal feature set is equal to the length of the spatial vector in the spatial feature set.

[0119] According to an embodiment of the present invention, a word embedding model is used to semantically encode the auxiliary semantic information of multiple trajectory sequences to obtain auxiliary feature sets for multiple trajectory clusters. The method includes: mapping the auxiliary semantic information of each of the multiple trajectory sequences from text form to feature vectors using an embedding lookup table or the embedding layer of a word embedding model; and concatenating the feature vectors corresponding to the auxiliary semantic information of each of the multiple trajectory sequences to obtain auxiliary feature sets for each of the multiple trajectory clusters.

[0120] The above has explained the extraction and fusion of multidimensional features. For ease of understanding, the following will be Figure 4 To give a hint.

[0121] Figure 4 A schematic diagram of multi-dimensional feature extraction and fusion in model training according to an embodiment of the present invention is schematically shown.

[0122] like Figure 4As shown in the figure, in the spatial feature extraction stage, multiple trajectory sequences are first converted into multiple trajectory images and input into CNN for spatial feature extraction of different depths (shallow and deep) to obtain a spatial feature set; in the temporal feature extraction stage, multiple trajectory subsequences are input into the bidirectional long short-term memory network, and the time features of time sequence and time reverse order are extracted through the forward network and backward network respectively to obtain a bidirectional temporal feature set; in the auxiliary feature extraction stage, the dynamic attribute information, typical trajectory information, related description information and auxiliary metadata corresponding to the trajectory sequence are obtained, and converted into dynamic attribute feature set, typical trajectory feature set, related description feature set and auxiliary metadata through the word embedding model, and then deep feature fusion is performed with the bidirectional temporal feature set and spatial feature set just obtained to obtain a multidimensional fusion feature set.

[0123] According to an embodiment of the present invention, the process of mapping the auxiliary semantic information of each of the plurality of trajectory sequences from text form to feature vectors using the embedding layer or embedding lookup table of the word embedding model can be expressed as the following formula (14).

[0124] (14);

[0125] Among them, h_text represents the feature vector corresponding to the auxiliary semantic information text, EmbeddingLayer() represents the word embedding process, R d Represents a space of vector dimension d.

[0126] According to an embodiment of the present invention, the above formula (14) converts the originally sparse, high-dimensional text into a semantically expressive feature vector suitable for deep neural network processing, providing a basis for subsequent feature fusion. Taking ten types of auxiliary semantic information as an example, the length of the feature vector corresponding to the auxiliary semantic information can be one-tenth of the length of the time vector and the space vector. Therefore, the feature vectors corresponding to the auxiliary semantic information of multiple trajectory sequences are spliced ​​in the feature dimension to obtain an auxiliary fusion vector with a length equal to the length of the time vector and the space vector, and then the auxiliary feature sets of multiple trajectory clusters are obtained.

[0127] Figure 5 The flowchart of a trajectory prediction method based on multi-dimensional feature fusion according to another embodiment of the present invention is schematically shown.

[0128] like Figure 5 As shown, the prediction method 500 includes operations S510 to S530.

[0129] In operation S510 , a target trajectory behavior pattern corresponding to the trajectory sequence to be predicted is determined based on a dynamic time warping algorithm.

[0130] The Dynamic Time Warping (DTW) algorithm iteratively finds the optimal matching path that minimizes the cumulative distance among all possible nonlinear alignment paths, thereby accurately measuring the similarity of time series. Therefore, DTW can be used to calculate the similarity distance between the current trajectory sequence to be predicted and the trajectory clusters corresponding to each trajectory behavior pattern. The trajectory behavior pattern corresponding to the trajectory cluster with the smallest similarity distance is then determined as the target trajectory behavior pattern for the trajectory sequence to be predicted.

[0131] For example, for the trajectory sequence to be predicted , it is necessary to select part of the trajectory sequence from multiple trajectory clusters and calculate the similarity distance with the sequence to be predicted. For example, the DTW algorithm distance between E and F It can be expressed as the following formula (15).

[0132] (15);

[0133] Among them, ρ is the trajectory point index, and ρ≤g, ρ≤h, Represents the trajectory point e1 and the trajectory point f ρ The distance between Represents the trajectory point e ρ The distance from the trajectory point f1, Represents the trajectory point e g With trajectory point f h The distance between , . It can be seen that when there is only one trajectory point in E, the distances between the only trajectory point e1 in E and all trajectory points in F are traversed and calculated and summed. Correspondingly, when there is only one trajectory point in F, the distances between f1 and all trajectory points in E are traversed and calculated and summed. When there are multiple trajectory points in E and F, the distance between the last trajectory point in each of the two trajectory sequences is first calculated, and the DTW algorithm distance between Head(E) and F, the DTW algorithm distance between E and Head(F), and the DTW algorithm distance between Head(E) and Head(F) are calculated. The minimum value of the multiple DTW algorithm distances is selected from the calculated distances and the distance between the last trajectory point in the two trajectory sequences is summed.

[0134] Since the calculation of the above formula (15) is also affected by the length of the trajectory sequence, the similarity distance can be obtained by the following formula (16): .

[0135] (16).

[0136] In operation S520, based on the target trajectory behavior pattern, the target prediction sub-model corresponding to the trajectory sequence to be predicted is called from the target prediction model. The training process of the target prediction sub-model has been described above and will not be repeated here.

[0137] In operation S530 , the trajectory data of the trajectory sequence to be predicted within the current period is input into the target prediction sub-model to obtain a target predicted trajectory corresponding to a predetermined period after the current period.

[0138] According to an embodiment of the present invention, a teacher-forcing strategy can be adopted to use the known data of the trajectory sequence to be predicted as the input of the target prediction sub-model. For example, if the trajectory sequence from the 1st moment to the 5th moment is known, to predict the trajectory sequence from the 6th moment to the 9th moment, the data of the 6th moment can be predicted by the data of the 5th moment, and the data of the 1st moment to the 5th moment as a whole must be considered to predict the data of the 6th moment.

[0139] Specifically, the input of the target prediction sub-model must pass through a fully connected layer. The known trajectory sequence is used as the mapping result and is input into the decoder of the target prediction sub-model. The decoder then deeply explores the dependencies between the known trajectory sequence and the predicted trajectory sequence, discovering more profound patterns in the data and leveraging the long-range correlations in the trajectory data to generate more accurate trajectory prediction results. At the same time, the output of the target prediction sub-model also needs to pass through a fully connected layer, outputting prediction data corresponding to multiple time points in the predetermined time period.

[0140] Figure 6 A block diagram of an electronic device suitable for implementing a model training method and a trajectory prediction method based on multi-dimensional feature fusion according to an embodiment of the present invention is schematically shown. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0141] like Figure 6 As shown, an electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0142] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 executes the programs in the ROM 602 and / or RAM 603 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also execute the programs stored in the one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0143] According to an embodiment of the present invention, electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to bus 604. Electronic device 600 may also include one or more of the following components connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or modem. Communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. Removable media 611, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 610 as needed, so that computer programs read from the removable media can be installed into storage section 608 as needed.

[0144] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0145] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0146] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or the RAM 603 described above and / or one or more memories other than the ROM 602 and the RAM 603 .

[0148] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the model training method and trajectory prediction method provided by the embodiment of the present invention.

[0149] When the computer program is executed by the processor 601, the above functions defined in the system / device of the embodiment of the present invention are performed. According to the embodiment of the present invention, the above-described systems, devices, modules, units, etc. can be implemented by computer program modules.

[0150] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0151] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, as well as the combination of boxes in the block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or may be implemented using a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention may be combined and / or coupled in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0153] The above describes embodiments of the present invention. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A model training method based on multi-dimensional feature fusion, characterized in that: include: Clustering is performed on multiple trajectory sequences to obtain trajectory clusters corresponding to the behavior patterns of multiple trajectories; Performing spatiotemporal feature extraction on each of the plurality of trajectory clusters to obtain a bidirectional temporal feature set and a spatial feature set for each of the plurality of trajectory clusters, wherein the bidirectional temporal feature set includes forward features and backward features in the temporal dimension, and the spatial feature set includes features in the spatial dimension; Using a word embedding model, semantically encode the auxiliary semantic information of each of the plurality of trajectory sequences to obtain auxiliary feature sets for each of the plurality of trajectory clusters; Based on the correlation between the bidirectional temporal feature sets, spatial feature sets, and auxiliary feature sets of the multiple trajectory clusters, feature fusion is performed on the multiple trajectory clusters to obtain multidimensional fusion feature sets of the multiple trajectory clusters; The initial model is trained using the multidimensional fusion feature sets of each of the multiple trajectory clusters until the model parameters meet a predetermined convergence condition, thereby obtaining a target prediction model, wherein the target prediction model includes target prediction sub-models corresponding to each of the multiple trajectory behavior patterns.

2. The method according to claim 1, characterized in that The clustering process of the plurality of trajectory sequences to obtain trajectory clusters corresponding to the plurality of trajectory behavior patterns includes: Based on the multi-dimensional predetermined strategy, the pre-processed multiple trajectory sequences are cut and processed to obtain multiple target trajectory subsequences; Based on the sliding window, feature extraction is performed on the multiple target trajectory subsequences to obtain feature sequences corresponding to each of the multiple target trajectory subsequences; Based on the multiple feature sequences, clustering processing is performed on the multiple trajectory sequences to obtain multiple trajectory clusters corresponding to the multiple trajectory behavior patterns.

3. The method according to claim 2, characterized in that The extracting spatiotemporal features of the plurality of trajectory clusters to obtain respective bidirectional temporal feature sets and spatial feature sets of the plurality of trajectory clusters includes: Using a multi-layer convolutional neural network, extracting features of the plurality of trajectory clusters in a spatial dimension to obtain spatial feature sets of the plurality of trajectory clusters; Using a bidirectional long short-term memory network, bidirectional feature extraction is performed on the multiple target trajectory subsequences in the time dimension to obtain bidirectional time features of the multiple target trajectory subsequences; concatenating the bidirectional temporal features of the plurality of target trajectory subsequences to obtain bidirectional temporal feature sets of the plurality of trajectory clusters; The bidirectional time feature length in the bidirectional time feature set of each of the plurality of trajectory clusters is equal to the spatial feature length in the spatial feature set.

4. The method according to claim 3, characterized in that The multidimensional predetermined strategy includes a speed strategy and a heading strategy, the trajectory sequence includes a plurality of trajectory point data, the trajectory point data includes heading angle data and acceleration data, and the target trajectory subsequence includes a variable speed linear trajectory subsequence, a uniform speed linear trajectory subsequence, a variable speed non-linear trajectory subsequence, or a uniform speed non-linear trajectory subsequence; The multi-dimensional predetermined strategy is based on which the pre-processed multiple trajectory sequences are cut to obtain multiple target trajectory subsequences, including: For each preprocessed trajectory sequence, Based on the heading strategy and the multiple heading angle data of the preprocessed trajectory sequence, the trajectory sequence is cut to obtain a linear trajectory subsequence and a non-linear trajectory subsequence corresponding to the trajectory sequence; Based on the speed strategy, multiple acceleration data of the straight line trajectory subsequence and multiple acceleration data of the non-linear trajectory subsequence, the straight line trajectory subsequence and the non-linear trajectory subsequence are cut respectively to obtain at least one of the variable speed straight line trajectory subsequence, the uniform speed straight line trajectory subsequence, the variable speed non-linear trajectory subsequence or the uniform speed non-linear trajectory subsequence.

5. The method according to claim 4, characterized in that The bidirectional long short-term memory network includes a first network, a second network, a third network, and a fourth network. The bidirectional long short-term memory network is used to extract bidirectional features from the multiple target trajectory subsequences in the time dimension, and the bidirectional time features of the multiple target trajectory subsequences are concatenated to obtain bidirectional time feature sets for the multiple trajectory clusters, including: Inputting the speed-changing linear trajectory subsequence into the first network, performing time sequence and time reverse feature extraction on the speed-changing linear trajectory subsequence to obtain a bidirectional time feature of the speed-changing linear trajectory subsequence; Inputting the uniform linear trajectory subsequence into the second network, performing time sequence and time reverse feature extraction on the uniform linear trajectory subsequence to obtain a bidirectional time feature of the uniform linear trajectory subsequence; Inputting the uniform non-linear trajectory subsequence into the third network, performing time sequence and time reverse feature extraction on the uniform non-linear trajectory subsequence to obtain a bidirectional time feature of the uniform non-linear trajectory subsequence; Inputting the speed-changing non-linear trajectory subsequence into the fourth network, performing time sequence and time reverse feature extraction on the speed-changing non-linear trajectory subsequence to obtain a bidirectional time feature of the speed-changing non-linear trajectory subsequence; The bidirectional time features of the speed-varying linear trajectory subsequence, the uniform speed linear trajectory subsequence, the speed-varying non-linear trajectory subsequence, and the uniform speed non-linear trajectory subsequence are projected into the same dimensional space to obtain the bidirectional time feature sets of the multiple trajectory clusters.

6. The method according to claim 3, characterized in that The multi-layer convolutional neural network includes a cascaded first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer; the multi-layer convolutional neural network is used to extract spatial features from the multiple trajectory clusters to obtain spatial feature sets for the multiple trajectory clusters, including: Converting the plurality of trajectory sequences into a plurality of trajectory images respectively; Performing local shallow feature extraction on the multiple trajectory images using the first convolutional layer and the first pooling layer to obtain a first feature map; The second convolutional layer and the second pooling layer are used to perform global deep feature extraction on the first feature map to obtain spatial feature sets of each of the multiple trajectory clusters.

7. The method according to any one of claims 1 to 3, characterized in that The word embedding model is used to semantically encode the auxiliary semantic information of each of the plurality of trajectory sequences to obtain auxiliary feature sets of each of the plurality of trajectory clusters, including: Mapping the auxiliary semantic information of each of the plurality of trajectory sequences from text form to a feature vector using an embedding lookup table or an embedding layer of the word embedding model; The feature vectors corresponding to the auxiliary semantic information of each of the plurality of trajectory sequences are concatenated to obtain auxiliary feature sets of each of the plurality of trajectory clusters.

8. The method according to claim 2, characterized in that The clustering process of the plurality of trajectory sequences based on the plurality of feature sequences to obtain trajectory clusters corresponding to the plurality of trajectory behavior patterns includes: Using a target deep autoencoder, compressing and reconstructing the plurality of feature sequences to obtain a low-dimensional reconstructed sequence, wherein the target deep autoencoder is obtained by adjusting parameters of an initial autoencoder based on a regularization norm, the regularization norm being determined based on a reconstruction error between reconstructed data and sample data, the reconstructed data being obtained by inputting the sample data into the initial autoencoder; The plurality of trajectory sequences are clustered using the low-dimensional reconstruction sequence to obtain trajectory clusters corresponding to the plurality of trajectory behavior patterns.

9. A trajectory prediction method based on multi-dimensional feature fusion, characterized in that: include: Based on the dynamic time warping algorithm, the target trajectory behavior pattern corresponding to the trajectory sequence to be predicted is determined; Based on the target trajectory behavior law, calling a target prediction sub-model corresponding to the trajectory sequence to be predicted from a target prediction model, wherein the target prediction model is trained according to any one of claims 1 to 8; The trajectory data of the trajectory sequence to be predicted in the current time period is input into the target prediction sub-model to obtain a target prediction trajectory corresponding to a predetermined time period after the current time period.

10. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 9.

Images