Model training and trajectory prediction methods based on multidimensional feature fusion

Abstract

This invention provides a model training method and trajectory prediction method based on multi-dimensional feature fusion, which can be applied to the field of artificial intelligence technology. The model training method includes: clustering multiple trajectory sequences to obtain trajectory clusters corresponding to various trajectory behavior patterns; extracting bidirectional temporal and spatial features from each trajectory cluster; semantically encoding the auxiliary semantic information of each trajectory sequence using a word embedding model to obtain auxiliary feature sets for each trajectory cluster; fusing features from the bidirectional temporal, spatial, and auxiliary feature sets of each trajectory cluster based on their respective relationships; and training an initial model using the obtained multi-dimensional fused feature sets to obtain target prediction sub-models corresponding to various trajectory behavior patterns. This overcomes the misjudgment in complex scenarios caused by predictions relying solely on bidirectional temporal and spatial feature sets in related technologies.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a model training method and trajectory prediction method based on multi-dimensional feature fusion. Background Technology

[0002] Predicting the behavioral trajectory of a target can assist in decision-making for risk avoidance and order management.

[0003] In the process of realizing the concept of this invention, it was found through research that the prediction models in related technologies are difficult to accurately uncover the behavioral patterns of complex and ever-changing target objects, resulting in low prediction accuracy. Summary of the Invention

[0004] In view of this, the present invention provides a model training method and a trajectory prediction method based on multi-dimensional feature fusion.

[0005] One aspect of the present invention provides a model training method based on multi-dimensional feature fusion, comprising:

[0006] Multiple trajectory sequences are clustered to obtain trajectory clusters corresponding to the behavioral patterns of each trajectory. Spatiotemporal features are extracted from each trajectory cluster to obtain bidirectional temporal and spatial feature sets. The bidirectional temporal feature set includes forward and backward features in the time dimension, and the spatial feature set includes features in the spatial dimension. Using a word embedding model, the auxiliary semantic information of each trajectory sequence is semantically encoded to obtain auxiliary feature sets for each trajectory cluster. Based on the correlation between the bidirectional temporal, spatial, and auxiliary feature sets of each trajectory cluster, feature fusion is performed to obtain multidimensional fused feature sets for each trajectory cluster. The initial model is trained using the multidimensional fused feature sets of each trajectory cluster until the model parameters meet the predetermined convergence conditions to obtain the target prediction model. The target prediction model includes target prediction sub-models corresponding to the behavioral patterns of each trajectory.

[0007] According to an embodiment of the present invention, clustering multiple trajectory sequences to obtain trajectory clusters corresponding to multiple trajectory behavior patterns includes: cutting the preprocessed multiple trajectory sequences according to a multi-dimensional predetermined strategy to obtain multiple target trajectory subsequences; extracting features from the multiple target trajectory subsequences based on a sliding window to obtain feature sequences corresponding to each of the multiple target trajectory subsequences; and clustering the multiple trajectory sequences based on the multiple feature sequences to obtain multiple trajectory clusters corresponding to each of the multiple trajectory behavior patterns.

[0008] According to an embodiment of the present invention, spatiotemporal feature extraction is performed on multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets for each trajectory cluster. This includes: using a multi-layer convolutional neural network to extract spatial dimension features from multiple trajectory clusters to obtain spatial feature sets for each trajectory cluster; using a bidirectional long short-term memory network to extract bidirectional features from multiple target trajectory subsequences in the time dimension to obtain bidirectional temporal features for each target trajectory subsequence; and concatenating the bidirectional temporal features of each target trajectory subsequence to obtain bidirectional temporal feature sets for each trajectory cluster. The length of the bidirectional temporal features in the bidirectional temporal feature sets of each trajectory cluster is equal to the length of the spatial features in the spatial feature sets.

[0009] According to an embodiment of the present invention, the multidimensional predetermined strategy includes a velocity strategy and a heading strategy. The trajectory sequence includes multiple trajectory point data, and the trajectory point data includes heading angle data and acceleration data. The target trajectory subsequence includes a variable-speed straight trajectory subsequence, a uniform-speed straight trajectory subsequence, a variable-speed non-linear trajectory subsequence, or a uniform-speed non-linear trajectory subsequence. Based on the multidimensional predetermined strategy, the preprocessed multiple trajectory sequences are segmented to obtain multiple target trajectory subsequences, including: for each preprocessed trajectory sequence, the trajectory sequence is segmented based on the heading strategy and multiple heading angle data of the preprocessed trajectory sequence to obtain a straight trajectory subsequence and a non-linear trajectory subsequence corresponding to the trajectory sequence; based on the velocity strategy, multiple acceleration data of the straight trajectory subsequence, and multiple acceleration data of the non-linear trajectory subsequence, the straight trajectory subsequence and the non-linear trajectory subsequence are segmented to obtain at least one of the variable-speed straight trajectory subsequence, uniform-speed straight trajectory subsequence, variable-speed non-linear trajectory subsequence, or uniform-speed non-linear trajectory subsequence.

[0010] According to an embodiment of the present invention, the bidirectional long short-term memory network includes a first network, a second network, a third network, and a fourth network. Using the bidirectional long short-term memory network, bidirectional feature extraction is performed on multiple target trajectory subsequences in the time dimension, and the bidirectional time features of each of the multiple target trajectory subsequences are concatenated to obtain bidirectional time feature sets for multiple trajectory clusters. This includes: inputting a variable-speed straight-line trajectory subsequence into the first network, performing time-order and time-reverse feature extraction on the variable-speed straight-line trajectory subsequence to obtain bidirectional time features of the variable-speed straight-line trajectory subsequence; inputting a uniform-speed straight-line trajectory subsequence into the second network, performing time-order and time-reverse feature extraction on the uniform-speed straight-line trajectory subsequence... The bidirectional time features of the uniform linear trajectory subsequence are obtained by taking the uniform non-linear trajectory subsequence as input into the third network and extracting features in both time order and time inversion. The bidirectional time features of the variable non-linear trajectory subsequence are obtained by taking the variable non-linear trajectory subsequence as input into the fourth network and extracting features in both time order and time inversion. The bidirectional time features of the variable linear trajectory subsequence, uniform linear trajectory subsequence, variable non-linear trajectory subsequence, and uniform non-linear trajectory subsequence are projected onto the same dimensional space to obtain the bidirectional time feature sets of multiple trajectory clusters.

[0011] According to an embodiment of the present invention, a multi-layer convolutional neural network includes a cascaded first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer. Using the multi-layer convolutional neural network, spatial feature extraction is performed on multiple trajectory clusters to obtain spatial feature sets for each trajectory cluster. This includes: converting multiple trajectory sequences into multiple trajectory images; using the first convolutional layer and the first pooling layer to perform local shallow feature extraction on the multiple trajectory images to obtain a first feature map; and using the second convolutional layer and the second pooling layer to perform global deep feature extraction on the first feature map to obtain spatial feature sets for each trajectory cluster.

[0012] According to an embodiment of the present invention, a word embedding model is used to semantically encode the auxiliary semantic information of multiple trajectory sequences to obtain auxiliary feature sets of multiple trajectory clusters, including: using an embedding lookup table or the embedding layer of a word embedding model to map the auxiliary semantic information of multiple trajectory sequences from text form to feature vectors respectively; and concatenating the feature vectors corresponding to the auxiliary semantic information of multiple trajectory sequences to obtain auxiliary feature sets of multiple trajectory clusters.

[0013] According to an embodiment of the present invention, multiple trajectory sequences are clustered based on multiple feature sequences to obtain trajectory clusters corresponding to each of the multiple trajectory behavior patterns. This includes: compressing and reconstructing multiple feature sequences using a target depth autoencoder to obtain a low-dimensional reconstruction sequence, wherein the target depth autoencoder is obtained by adjusting the parameters of an initial autoencoder based on a regularization norm, the regularization norm being determined based on the reconstruction error between the reconstructed data and the sample data, and the reconstructed data being obtained by inputting the sample data into the initial autoencoder; and then clustering the multiple trajectory sequences using the low-dimensional reconstruction sequence to obtain trajectory clusters corresponding to each of the multiple trajectory behavior patterns.

[0014] Another aspect of the present invention provides a trajectory prediction method based on multidimensional feature fusion, comprising:

[0015] Based on the dynamic time warping algorithm, the behavior pattern of the target trajectory corresponding to the trajectory sequence to be predicted is determined; based on the behavior pattern of the target trajectory, the target prediction sub-model corresponding to the trajectory sequence to be predicted is called from the target prediction model, wherein the target prediction model is trained according to the above model training method; the trajectory data of the trajectory sequence to be predicted in the current time period is input into the target prediction sub-model to obtain the target prediction trajectory corresponding to the predetermined time period after the current time period.

[0016] Another aspect of the present invention provides an electronic device comprising:

[0017] One or more processors;

[0018] Memory, used to store one or more programs.

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0020] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the method described above.

[0021] Another aspect of the present invention provides a computer program product comprising computer-executable instructions which, when executed, are used to implement the method described above.

[0022] According to an embodiment of the present invention, multiple trajectory sequences are first clustered to group trajectory sequences with similar trajectory behavior patterns into the same trajectory cluster. Then, spatiotemporal features are extracted from multiple trajectory clusters to obtain the spatial features, forward features, and backward features of the trajectory sequences in the temporal dimension. The changes of specific trajectory behavior patterns in the temporal and spatial dimensions are captured, and the semantic information of the trajectory sequences is embedded to obtain an auxiliary feature set. Based on the correlation between the bidirectional temporal feature set, spatial feature set, and auxiliary feature set, multi-source feature fusion is performed to obtain a multi-dimensional fusion feature set with spatiotemporal correlation corresponding to different trajectory behavior patterns. The multi-dimensional fusion feature set is used for training until a predetermined convergence condition is met to obtain target prediction sub-models corresponding to different trajectory behavior patterns. Thus, the corresponding target prediction sub-model can be selected for prediction based on the trajectory behavior pattern of the trajectory to be predicted, effectively improving the prediction accuracy and better meeting the complex and ever-changing trajectory prediction needs. Attached Figure Description

[0023] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0024] Figure 1 A flowchart illustrating a model training method based on multidimensional feature fusion according to an embodiment of the present invention is shown.

[0025] Figure 2 A schematic diagram illustrating dual-path feature fusion according to an embodiment of the present invention is shown.

[0026] Figure 3 The illustration shows a schematic diagram of clustering multiple trajectory sequences according to an embodiment of the present invention.

[0027] Figure 4 The diagram illustrates multidimensional feature extraction and fusion during model training according to an embodiment of the present invention.

[0028] Figure 5 A flowchart illustrating a trajectory prediction method based on multidimensional feature fusion according to another embodiment of the present invention is shown.

[0029] Figure 6 The diagram illustrates an electronic device suitable for implementing a model training method and a trajectory prediction method based on multidimensional feature fusion according to an embodiment of the present invention. Detailed Implementation

[0030] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0032] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0033] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0034] In realizing the inventive concept of this invention, research revealed that existing target trajectory prediction methods mainly focus on two aspects: data-driven and behavior-driven. Data-driven trajectory prediction methods primarily utilize massive amounts of historical trajectory data to mine the hidden behavioral characteristics of moving targets, then fuse and match this data with current location data to predict the target's movement trend. Behavior-driven trajectory prediction methods, on the other hand, can predict the trajectory of a moving target within a certain future time period based on the relevant motion characteristics of the target object, and can generally be divided into two main categories: dynamic models and intent recognition.

[0035] However, data-driven approaches require a large number of samples for model learning, and feature extraction relies heavily on the model's own learning iterations, making it difficult to quickly adapt to dynamic changes in target behavior and analyze behavioral intent. Behavior-driven prediction models are limited to specific scenarios with clearly defined intents, and suffer from low prediction accuracy in real-world applications.

[0036] In view of this, embodiments of the present invention provide a model training method for trajectory prediction. This method clusters multiple trajectory sequences to obtain trajectory clusters corresponding to the behavioral patterns of each trajectory. This breaks through the technical barriers of related technologies in uniformly modeling multiple trajectory behavioral patterns. It performs bidirectional temporal and spatial feature extraction on multiple trajectory clusters. That is, in the temporal dimension, it can not only infer future trajectories from previous trajectories but also infer previous trajectories from future trajectories, improving the accuracy of feature extraction for complex trajectories. Simultaneously, it embeds auxiliary semantic information through semantic encoding, compensating for the misjudgments in complex scenarios caused by relying solely on bidirectional temporal and spatial feature sets in related technologies. Furthermore, in the process of multi-source feature fusion, it no longer relies solely on fixed weight allocation or simple concatenation but simultaneously considers the correlation between bidirectional temporal feature sets, spatial feature sets, and auxiliary feature sets, ensuring a more comprehensive multi-dimensional fused feature set. This improves the training accuracy of the initial model and ensures that each prediction sub-model focuses on its own behavioral patterns during the learning process, avoiding mutual interference between features of different trajectory behavioral patterns. This results in target prediction sub-models corresponding to multiple trajectory behavioral patterns having better robustness and prediction accuracy.

[0037] Figure 1 A flowchart illustrating a model training method based on multidimensional feature fusion according to an embodiment of the present invention is shown.

[0038] like Figure 1 As shown, the model training method 100 includes operations S110 to S150.

[0039] In operation S110, multiple trajectory sequences are clustered to obtain trajectory clusters corresponding to the behavioral patterns of each trajectory.

[0040] According to embodiments of the present invention, multiple trajectory sequences may originate from different regions (including but not limited to sea or air) and different types of target objects (e.g., aircraft, ships). Specifically, they include multiple trajectory point data arranged in chronological order. The trajectory point data may include static attribute data (e.g., longitude, latitude) and dynamic attribute data (e.g., acceleration, speed, heading angle).

[0041] Clustering techniques in related technologies focus solely on the path shape of trajectories through static attributes, neglecting dynamic attributes. This leads to clustering results that are either too limited or too broad. Therefore, this approach identifies patterns in trajectory behavior based on multiple trajectory point data, thereby grouping trajectories with similar behavioral patterns into the same trajectory cluster.

[0042] Trajectory behavior patterns can include: direct flight, circling, hovering, climbing, landing, turning, loitering, anchoring, and escorting. Therefore, clustering that does not consider dynamic attributes makes it difficult to distinguish between hovering and loitering, leading to the misclassification of loitering and hovering trajectories into the same cluster, thus affecting subsequent prediction accuracy.

[0043] In operation S120, spatiotemporal features are extracted from multiple trajectory clusters, resulting in bidirectional temporal feature sets and spatial feature sets for each trajectory cluster. The bidirectional temporal feature set includes forward and backward features in the time dimension, while the spatial feature set includes features in the spatial dimension.

[0044] According to an embodiment of the present invention, spatiotemporal features are extracted from multiple trajectory clusters generated by clustering to avoid mutual interference between trajectory sequences with different trajectory behavior patterns.

[0045] For example, circling, hovering, loitering, and turning all involve changing course, and mixing these behaviors in extraction may lead to error accumulation.

[0046] According to embodiments of the present invention, spatiotemporal feature extraction may include feature extraction in the time dimension and feature extraction in the spatial dimension. For feature extraction in the time dimension, it may include extraction from front to back and extraction from back to front, and simultaneously cover local temporal feature extraction and global temporal feature extraction.

[0047] In operation S130, the word embedding model is used to semantically encode the auxiliary semantic information of multiple trajectory sequences to obtain the auxiliary feature sets of multiple trajectory clusters.

[0048] According to an embodiment of the present invention, the auxiliary semantic information is in text form, including semantic information corresponding to the trajectory sequence, which may include dynamic attribute information (e.g., kinematic parameters such as speed, acceleration, and heading angle), typical trajectory information (e.g., typical trajectory types of fighter jets, reconnaissance aircraft, and UAVs), relevant descriptive information (e.g., equipment model, mission batch number, and typical activity area of ​​the target object), and auxiliary metadata (e.g., non-content information such as timestamps and data source identifiers). The auxiliary feature set is then divided into a dynamic attribute feature set, a typical trajectory feature set, a relevant descriptive feature set, and an auxiliary metadata feature set.

[0049] Since the auxiliary semantic information is currently in the form of a string of text, it needs to be semantically encoded using a word embedding model to convert it into a feature form. The word embedding model can be Word2Vec (a model used to generate word vectors) or FastText. Additionally, the initial text may contain redundant symbols or inconsistent formatting, so it needs to be structurally cleaned first to ensure that the word embedding model can effectively capture the semantics.

[0050] For example, punctuation marks in auxiliary semantic information should be removed, and only the core content should be retained. Units for acceleration, velocity, etc., should also be standardized.

[0051] In operation S140, based on the correlation between the bidirectional time feature sets, spatial feature sets, and auxiliary feature sets of multiple trajectory clusters, feature fusion is performed on the multiple trajectory clusters respectively to obtain the multidimensional fused feature sets of the multiple trajectory clusters.

[0052] According to an embodiment of the present invention, an adaptive feature interaction network can be used to achieve deep fusion of bidirectional temporal feature sets, spatial feature sets, and auxiliary feature sets. First, based on the correlation between the bidirectional temporal feature sets, spatial feature sets, and auxiliary feature sets, the attention weights of one feature relative to the other two features are obtained, which can be specifically expressed as the following formula (1).

[0053] (1);

[0054] Among them, A mn Representing target feature H m Target association features H n Relative to target feature H m The attention weights are defined as follows: m is the target feature index, n is the target association feature index, and H is the target feature. m H is any feature derived from the bidirectional temporal feature set, spatial feature set, and auxiliary feature set. m Target association features H n Other related features H q With H m Belonging to different modes, for example, H m As a typical trajectory feature, H q These can be bidirectional temporal features, spatial features, dynamic attribute features, relevant descriptive features, and auxiliary meta-features. ( ) represents the similarity function, used to calculate the cosine similarity between the target feature and the associated features. This represents a variable weight parameter.

[0055] Subsequently, based on the above formula (1), the target feature H is obtained as shown in the following formula (2). m gating vector Gating vectors can reflect the fine-grained weights of individual features.

[0056] (2);

[0057] in, ( ) represents the activation function, such as the Sigmoid activation function (logistic regression activation function), and LeakyReLU() represents the LeakyReLU activation function (leakage correction linear unit activation function). P is the generation weight matrix of the gate vector.

[0058] According to an embodiment of the present invention, the related technology assigns a single overall weight to features of the same modality only during feature fusion, while the gating vector is used to assign a separate weight to each feature in each modality. Subsequently, for each trajectory cluster, multimodal feature fusion is performed based on dual paths to obtain the multidimensional fused feature set shown in the following formula (3). .

[0059] (3);

[0060] Where LN() represents the normalization process, and P' represents the residual transformation weight matrix. H represents element-wise multiplication. skip This represents the mean of all features corresponding to the current trajectory cluster.

[0061] To better understand the specific process of dual-path feature fusion, the following will explain... Figure 2 To illustrate.

[0062] Figure 2 A schematic diagram illustrating dual-path feature fusion according to an embodiment of the present invention is shown.

[0063] like Figure 2 As shown, for each trajectory cluster corresponding to a trajectory behavior pattern, the dynamic attribute feature set, typical trajectory feature set, relevant descriptive feature set, auxiliary meta-feature set, bidirectional temporal feature set, and spatial feature set are first input into the projection layer to unify the dimensions. Then, the processing is divided into path 1 and path 2. In path 1, the dimension-unified feature vectors are processed through a cascaded cross-feature attention layer, adaptive gating layer, and gating application layer to obtain the product of the dimension-unified feature vectors and their corresponding gating vectors. In path 2, the mean of each dimension-unified feature vector is obtained by using a feature averager. Then, the outputs of path 1 and path 2 are weighted and aggregated in the path aggregation layer, and the weighted aggregation result is input into the normalization layer to obtain the multi-dimensional fusion feature set of the trajectory cluster.

[0064] In operation S150, the initial model is trained using the multidimensional fusion feature sets of multiple trajectory clusters until the model parameters meet the predetermined convergence conditions, thus obtaining the target prediction model.

[0065] According to an embodiment of the present invention, the initial model can be a Transformer (a deep learning model architecture based on attention mechanism), which can efficiently mine features between trajectory contexts based on the attention mechanism, while effectively solving the problem of long-distance dependency. The Transformer model is trained using feature data from a multi-dimensional fusion feature set corresponding to different trajectory behavior patterns to obtain the corresponding prediction model weight parameters. The model's weight parameters are continuously optimized through iterative training until a predetermined number of iterations is reached or L reaches a predetermined threshold, thus completing the training.

[0066] The initial model remains an encoder-decoder structure. The encoder consists of a single input layer and multiple stacked encoding layers. The decoder has a similar structure, consisting of one input layer and multiple stacked decoding layers. The encoder's encoding layers include a multi-head self-attention mechanism and a feedforward network, followed by residual connections and normalization structures. The decoder's decoding layers include a multi-head self-attention mechanism (with masking), an encoder-decoder multi-head attention mechanism, and a feedforward network, with residual connections and normalization structures added after the feedforward network. These three attention mechanisms, residual connections, and normalization structures effectively mitigate potential gradient vanishing or gradient diffusion problems. The multi-head self-attention mechanism (with masking) can mask temporal features by changing the attention weights.

[0067] According to an embodiment of the present invention, the target prediction model is essentially a target prediction sub-model corresponding to multiple trajectory behavior patterns. In the application process, the trajectory behavior pattern closest to the trajectory to be predicted can be determined first, and then the corresponding target prediction sub-model can be called.

[0068] According to an embodiment of the present invention, multiple trajectory sequences are first clustered to group trajectory sequences with similar trajectory behavior patterns into the same trajectory cluster. Then, spatiotemporal features are extracted from multiple trajectory clusters to obtain the spatial features, forward features, and backward features of the trajectory sequences in the temporal dimension. The changes of specific trajectory behavior patterns in the temporal and spatial dimensions are captured, and the semantic information of the trajectory sequences is embedded to obtain an auxiliary feature set. Based on the correlation between the bidirectional temporal feature set, spatial feature set, and auxiliary feature set, multi-source feature fusion is performed to obtain a multi-dimensional fusion feature set with spatiotemporal correlation corresponding to different trajectory behavior patterns. The multi-dimensional fusion feature set is used for training until a predetermined convergence condition is met to obtain target prediction sub-models corresponding to different trajectory behavior patterns. Thus, the corresponding target prediction sub-model can be selected for prediction based on the trajectory behavior pattern of the trajectory to be predicted, effectively improving the prediction accuracy and better meeting the complex and ever-changing trajectory prediction needs.

[0069] According to an embodiment of the present invention, clustering multiple trajectory sequences to obtain trajectory clusters corresponding to multiple trajectory behavior patterns includes: cutting the preprocessed multiple trajectory sequences according to a multi-dimensional predetermined strategy to obtain multiple target trajectory subsequences; extracting features from the multiple target trajectory subsequences based on a sliding window to obtain feature sequences corresponding to each of the multiple target trajectory subsequences; and using a clustering algorithm to cluster the multiple trajectory sequences based on the multiple feature sequences to obtain multiple trajectory clusters corresponding to each of the multiple trajectory behavior patterns.

[0070] Among them, the multidimensional pre-planning strategy includes speed strategy and heading strategy.

[0071] According to an embodiment of the present invention, the preprocessing of trajectory sequences can filter out abnormal trajectory sequences, providing a more reliable data source for subsequent trajectory sequence segmentation and feature extraction.

[0072] Based on a multi-dimensional pre-defined strategy, multiple preprocessed trajectory sequences are segmented to obtain multiple target trajectory subsequences. This includes: for each preprocessed trajectory sequence, firstly, based on a heading strategy, analyzing the changes in heading angle data corresponding to each waypoint in the trajectory sequence according to the heading strategy and multiple heading angle data of the preprocessed trajectory sequence, thereby segmenting the trajectory sequence to obtain straight trajectory subsequences and non-straight trajectory subsequences corresponding to the trajectory sequence. The non-straight trajectory subsequences may include turning trajectory subsequences. Subsequently, based on a speed strategy and multiple accelerations of the straight trajectory subsequences... By analyzing the changes in velocity data and multiple acceleration data of non-linear trajectory subsequences, it is determined whether the linear trajectory subsequence and the non-linear trajectory subsequence belong to acceleration, deceleration, or uniform motion, respectively. This allows for the segmentation of the linear trajectory subsequence into variable-speed linear trajectory subsequence and uniform-speed linear trajectory subsequence, and the non-linear trajectory subsequence into variable-speed non-linear trajectory subsequence and uniform-speed non-linear trajectory subsequence. Finally, at least one of the following is obtained: variable-speed linear trajectory subsequence, uniform-speed linear trajectory subsequence, variable-speed non-linear trajectory subsequence, and uniform-speed non-linear trajectory subsequence.

[0073] In other words, based on a multi-dimensional pre-defined strategy, each preprocessed trajectory sequence can be divided into one or more types of target trajectory subsequences.

[0074] Figure 3 The illustration shows a schematic diagram of clustering multiple trajectory sequences according to an embodiment of the present invention.

[0075] like Figure 3As shown, the initial trajectory sequences are of varying quality, and the time intervals between trajectory points are not equal in all trajectory sequences; some are too large and some are too small. Therefore, low-quality trajectory sequences need to be deleted during the preprocessing process of the preprocessing layer to avoid these useless data affecting the accuracy of subsequent models.

[0076] Subsequently, the preprocessed trajectory sequences can be segmented using a multidimensional pre-defined strategy to obtain multiple target trajectory subsequences.

[0077] According to an embodiment of the present invention, multiple target trajectory subsequences corresponding to multiple trajectory sequences are input into a feature extraction layer for motion behavior feature extraction. Based on a sliding window, each target trajectory subsequence is converted into a feature sequence.

[0078] According to an embodiment of the present invention, multiple trajectory sequences are clustered based on multiple feature sequences to obtain trajectory clusters corresponding to multiple behavioral patterns, including: using a target depth autoencoder to compress and reconstruct multiple feature sequences to obtain a low-dimensional reconstruction sequence, wherein the target depth autoencoder is obtained by adjusting the parameters of an initial autoencoder based on a regularization norm, the regularization norm is determined based on the reconstruction error between the reconstructed data and the sample data, and the reconstructed data is obtained by inputting the sample data into the initial autoencoder.

[0079] Because the trajectory of the target object operates in a complex environment, the data flow within the feature sequence is highly nonlinear. Therefore, a deep autoencoder (DAE) can be used to reduce the dimensionality of the feature sequence. DAE is an unsupervised learning algorithm where the expected output of the model has the same dimensionality as the input. DAE can be applied to image recognition, noise removal, and data dimensionality reduction, and has derived various functionalities. Compared to the linear dimensionality reduction achieved by Principal Component Analysis (PCA), it has better nonlinear dimensionality reduction performance.

[0080] See details Figure 3 In the encoding stage, the number of neurons in the hidden layers decreases layer by layer. This indicates that after the feature sequence enters the DAE, it is input into the hidden layers of the encoding stage. Through weight learning, key features are gradually extracted, and then input into the bottleneck layer where they are compressed into low-dimensional nonlinear vectors, achieving nonlinear dimensionality reduction. Subsequently, the hidden layers of the decoding stage are used to gradually restore the low-dimensional vectors, obtaining a low-dimensional reconstructed sequence. This effectively extracts the complex distribution of data in the feature sequence and improves the accuracy of clustering.

[0081] According to an embodiment of the present invention, in a target depth autoencoder, the relationship between the reconstructed data output by the decoder and the data input by the encoder can be expressed as the following formula (4).

[0082] (4);

[0083] Where E() represents the encoding process, D[] represents the decoding process, and X represents the data matrix input to the encoder. This represents the reconstructed data matrix output by the decoder. The connection function between neurons can be the logistic function (Rogistic function), that is, the data transmission process between neurons can be represented by the encoder output as shown in the following formula (5). The decoder output shown in formula (6) .

[0084] (5);

[0085] (6);

[0086] Where x∈X, θ is the parameter matrix of the target depth autoencoder, and W is the weight matrix. This is the bias matrix corresponding to the encoder. Let T be the bias matrix corresponding to the decoder, logit denotes the logistic function, and T is the matrix transpose symbol.

[0087] In fact, X can include two parts, L and S, i.e., X = L + S, where L represents the vector that can be well represented by the hidden layer of the autoencoder, and S represents the outlier vector that is difficult to represent. To reduce... The error between X and L needs to be addressed by removing outlier vectors to ensure the decoding stage can focus more on reconstructing L, thus improving the convergence condition of the target depth autoencoder. Therefore, a regularization norm can be introduced during the training of the initial depth autoencoder. The objective function of the autoencoder... It can be expressed as the following formula (7).

[0088] (7);

[0089] Where λ is a parameter used to adjust the sparsity in S. To represent the L0 norm of S, the L1 norm can be used to simplify calculations. Regularized convex relaxation (transforming a non-convex optimization problem into a convex optimization problem) substitution The norm is used to optimize the clustering results, which makes it easier for trajectory sequences with the same trajectory behavior pattern to cluster into the same trajectory cluster and separate them from trajectory sequences with other trajectory behavior patterns.

[0090] According to an embodiment of the present invention, in order to further avoid clustering being disturbed by abnormal vectors, the norm of the matrix column vector can be introduced, which can be specifically expressed as the following formula (8).

[0091] (8);

[0092] in, L represents X 2,1 Norm, Let X represent the j-th column vector. express The L2 norm of X, where J is the total number of columns in X. Let I represent the vector in the j-th column and i-th row of X, and let I represent the total number of rows in X. One of the predetermined constraints can be expressed as the following formula (9).

[0093] (9);

[0094] Here, "st" is an abbreviation for "subject to," meaning subject to. Furthermore, the intermediate autoencoder can only be identified as the target depth autoencoder after the loss function of the intermediate autoencoder obtained through iterative training converges.

[0095] According to an embodiment of the present invention, by introducing a regularization norm to train the initial autoencoder, it is ensured that the obtained target depth autoencoder can effectively filter out abnormal parts, so that the decoding stage focuses on the reconstruction of normal vectors, that is, the reconstruction error is smaller. At the same time, the regularization norm can ensure that trajectory sequences with the same or similar trajectory behavior patterns are more cohesive, thus strengthening the clustering effect.

[0096] According to an embodiment of the present invention, spatiotemporal feature extraction is performed on multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets for each trajectory cluster. This includes: using a multi-layer convolutional neural network to extract spatial dimension features from multiple trajectory clusters to obtain spatial feature sets for each trajectory cluster; using a bidirectional long short-term memory network to extract bidirectional features from multiple target trajectory subsequences in the time dimension to obtain bidirectional temporal features for each target trajectory subsequence; and concatenating the bidirectional temporal features of each target trajectory subsequence to obtain bidirectional temporal feature sets for each trajectory cluster; wherein the temporal feature dimension in the temporal feature sets of each trajectory cluster is equal to the spatial feature dimension in the spatial feature sets.

[0097] According to an embodiment of the present invention, a multi-layer convolutional neural network includes a cascaded first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer. Using the multi-layer convolutional neural network, spatial feature extraction is performed on multiple trajectory clusters to obtain spatial feature sets for each trajectory cluster. This includes: converting multiple trajectory sequences into multiple trajectory images; using the first convolutional layer and the first pooling layer to perform local shallow feature extraction on the multiple trajectory images to obtain a first feature map; and using the second convolutional layer and the second pooling layer to perform global deep feature extraction on the first feature map to obtain spatial feature sets for each trajectory cluster.

[0098] According to an embodiment of the present invention, firstly, a one-dimensional trajectory sequence to two-dimensional trajectory image generation algorithm is used to convert multiple trajectory sequences into multiple trajectory images respectively. Then, the convolution kernel sliding mechanism of multi-layer convolutional neural networks (CNN) is used to extract spatial features at different depths of the trajectory images, and capture spatial correlation information such as the position distribution and motion trend of the target object in the trajectory images.

[0099] According to an embodiment of the present invention, the trajectory image is first input into the first convolutional layer Cov1 to extract local shallow spatial features. The output of each neuron in the first convolutional layer can be expressed as the following formula (10).

[0100] (10);

[0101] in, This represents the output of the neuron in the c-th output channel, α-th row, and β-th column of Cov1. σ(⋅) represents the activation function, which can be ReLU (Rectified Linear Unit). For the input Y, it can have... L represents the side length of the convolution kernel. The parameter represents the kernel parameter, and k represents the stride. b represents the number of pixels in the input trajectory image. c This represents the bias term of the c-th output channel, v and These are the row and column coordinate indices of the convolution kernel, respectively. .

[0102] According to an embodiment of the present invention, as can be seen from the above formula (10), for neurons at each position in the first convolutional layer, the convolutional kernel slides on the input, traverses all positions within the kernel, multiplies the parameters of each position with the input pixels, adds them together and adds a bias term, and then processes them with an activation function to retain useful information and suppress invalid information.

[0103] According to an embodiment of the present invention, the first pooling layer Pooling1 samples the output of the first convolutional layer Cov1 to generate a larger proportion of higher-level features and obtain a first feature map. The output of each neuron in the first pooling layer can be expressed as the following formula (11).

[0104] (11);

[0105] in, This represents the output result of the neuron in the c-th output channel, α-th row, and β-th column of Pooling1. This represents the output result of the neuron in the c-th output channel, the α·k+v-th row, and the β·k+ω-th column of Cov1, where t is the size of the pooling window of Pooling1.

[0106] According to an embodiment of the present invention, as can be seen from the above formula (11), the function of the first pooling layer is to extract the local maximum value from the output of the first convolutional layer.

[0107] According to an embodiment of the present invention, a second convolutional layer Cov2 and a second pooling layer Pooling2 can be installed after the first pooling layer to extract global deep features. The processing of the output of the first pooling layer by the second convolutional layer Cov2 and the second pooling layer Pooling2 can refer to the first convolutional layer Cov1 and the first pooling layer Pooling1, which will not be described in detail here.

[0108] According to an embodiment of the present invention, the second feature map output by the second pooling layer Pooling2 needs to be flattened to convert the multidimensional second feature map into a spatial feature set in the form of a one-dimensional vector.

[0109] According to an embodiment of the present invention, a bidirectional long short-term memory network is used to extract bidirectional features from multiple target trajectory subsequences in the time dimension, and the bidirectional time features of each of the multiple target trajectory subsequences are concatenated to obtain time feature sets for multiple trajectory clusters, including:

[0110] The variable-speed linear trajectory subsequence is input into the first network, and its temporal sequence and temporal reverse sequence features are extracted to obtain bidirectional temporal features. The uniform-speed linear trajectory subsequence is input into the second network, and its temporal sequence and temporal reverse sequence features are extracted to obtain bidirectional temporal features. The uniform-speed non-linear trajectory subsequence is input into the third network, and its temporal sequence and temporal reverse sequence features are extracted to obtain bidirectional temporal features. The variable-speed non-linear trajectory subsequence is input into the fourth network, and its temporal sequence and temporal reverse sequence features are extracted to obtain bidirectional temporal features. The bidirectional temporal features of each of the variable-speed linear trajectory subsequence, uniform-speed linear trajectory subsequence, variable-speed non-linear trajectory subsequence, and uniform-speed non-linear trajectory subsequence are projected onto the same dimensional space to obtain the temporal feature sets of multiple trajectory clusters.

[0111] According to embodiments of the present invention, the first network, the second network, the third network, and the fourth network correspond to the dimensions of different types of target trajectory subsequences. For example, the first network corresponds to the linear trajectory subsequence with varying velocity and can be 64-dimensional; the second network corresponds to the linear trajectory subsequence with uniform velocity and can be 128-dimensional; and the third network corresponds to the non-linear trajectory subsequence with varying velocity and can be 96-dimensional. Each of the first, second, third, and fourth networks includes a forward network and a backward network, enabling simultaneous forward and backward extraction of temporal features. This allows for the simultaneous capture of the influence of the trajectory's beginning on its ending and the influence of its ending on its beginning. Since the time step of each of the forward and backward networks represents the number of sequence units processed in one operation, to ensure the complete reading of all target trajectory subsequences and the accurate construction of the temporal relationship between the target trajectory subsequences, it is necessary to ensure that the time step is equal to the number of target trajectory subsequences of the corresponding category, thus avoiding information loss or the introduction of invalid data.

[0112] The bidirectional temporal features of multiple target trajectory subsequences are concatenated to obtain the bidirectional temporal feature set of the corresponding trajectory cluster. The process can be expressed as the following formula (12).

[0113] (12);

[0114] in, Representing the bidirectional time characteristics of the nonlinear trajectory subsequence with variable speed. The corresponding feature weights, Represents the bidirectional time characteristics of the variable-speed linear trajectory subsequence. The corresponding feature weights, Represents the bidirectional time characteristics of a subsequence of uniform linear trajectories. The corresponding feature weights, The feature weights represent the bidirectional time features corresponding to the uniform nonlinear trajectory subsequence. .

[0115] Feature weights { , , , }∈G, G can be 256-dimensional, and G can be calculated by the following formula (13).

[0116] (13);

[0117] Where || represents feature concatenation. Ω represents the weight matrix corresponding to the feature weights, and Ω represents the bias matrix corresponding to the feature weights.

[0118] In addition, the channel length of the bidirectional long short-term memory network can be matched with the length of the one-dimensional vector in the spatial feature set mentioned above, so as to balance the weights of subsequent feature fusion. That is, the vector length of the temporal feature in the bidirectional temporal feature set is equal to the length of the spatial vector in the spatial feature set.

[0119] According to an embodiment of the present invention, a word embedding model is used to semantically encode the auxiliary semantic information of multiple trajectory sequences to obtain auxiliary feature sets of multiple trajectory clusters, including: using an embedding lookup table or the embedding layer of a word embedding model to map the auxiliary semantic information of multiple trajectory sequences from text form to feature vectors respectively; and concatenating the feature vectors corresponding to the auxiliary semantic information of multiple trajectory sequences to obtain auxiliary feature sets of multiple trajectory clusters.

[0120] The extraction and fusion of multidimensional features have been explained above. For ease of understanding, the following will illustrate this further. Figure 4 To illustrate.

[0121] Figure 4 The diagram illustrates multidimensional feature extraction and fusion during model training according to an embodiment of the present invention.

[0122] like Figure 4As shown, in the spatial feature extraction stage, multiple trajectory sequences are first converted into multiple trajectory images and input into a CNN for spatial feature extraction at different depths (shallow and deep) to obtain a spatial feature set. In the temporal feature extraction stage, multiple trajectory subsequences are input into a bidirectional long short-term memory network, and temporal features are extracted in both forward and backward order through the feedforward and backward networks to obtain a bidirectional temporal feature set. In the auxiliary feature extraction stage, dynamic attribute information, typical trajectory information, related descriptive information, and auxiliary metadata corresponding to the trajectory sequences are obtained. Through a word embedding model, these are converted into dynamic attribute feature sets, typical trajectory feature sets, related descriptive feature sets, and auxiliary metadata sets, respectively. These are then fused with the bidirectional temporal feature set and spatial feature set obtained earlier to obtain a multidimensional fused feature set.

[0123] According to an embodiment of the present invention, the process of mapping the auxiliary semantic information of multiple trajectory sequences from text form to feature vectors using the embedding layer or embedding lookup table of the word embedding model can be expressed as the following formula (14).

[0124] (14);

[0125] Where h_text represents the feature vector corresponding to the auxiliary semantic information text, EmbeddingLayer() represents the word embedding process, and R d This represents a space with dimension d for a vector.

[0126] According to an embodiment of the present invention, the above formula (14) transforms the originally sparse, high-dimensional text into a feature vector with semantic expressiveness suitable for deep neural network processing, providing a foundation for subsequent feature fusion. Taking ten kinds of auxiliary semantic information as an example, the length of the feature vector corresponding to the auxiliary semantic information can be one-tenth of the length of the time vector and the space vector. Thus, the feature vectors corresponding to the auxiliary semantic information of multiple trajectory sequences are concatenated in the feature dimension to obtain an auxiliary fusion vector with a length equal to the length of the time vector and the space vector, thereby obtaining the auxiliary feature sets of multiple trajectory clusters.

[0127] Figure 5 A flowchart illustrating a trajectory prediction method based on multidimensional feature fusion according to another embodiment of the present invention is shown.

[0128] like Figure 5 As shown, the prediction method 500 includes operations S510 to S530.

[0129] When operating S510, the dynamic time warping algorithm is used to determine the target trajectory behavior pattern corresponding to the trajectory sequence to be predicted.

[0130] Dynamic Time Warping (DTW) algorithms can iteratively optimize all possible nonlinear alignment paths to find the optimal matching path that minimizes the cumulative distance, thereby achieving accurate measurement of time series similarity. Therefore, DTW can be used to calculate the similarity distance between the current trajectory sequence to be predicted and the trajectory clusters corresponding to each trajectory behavior pattern, thus identifying the trajectory behavior pattern corresponding to the trajectory cluster with the smallest similarity distance as the target trajectory behavior pattern for the trajectory sequence to be predicted.

[0131] For example, for the trajectory sequence to be predicted This requires selecting partial trajectory sequences from multiple trajectory clusters and calculating their similarity distance with the sequence to be predicted. Taking the trajectory sequences from the direct flight trajectory cluster as an example... For example, the DTW algorithm distance between E and F It can be expressed as the following formula (15).

[0132] (15);

[0133] Where ρ is the index of the trajectory point, and ρ≤g, ρ≤h. Represents the relationship between trajectory point e1 and trajectory point f ρ The distance between them Represents trajectory point e ρ Distance between the trajectory point f1 and the trajectory point f1 Represents trajectory point e g With trajectory point f h The distance between them , Therefore, when there is only one trajectory point in E, the distances between the unique trajectory point e1 in E and all trajectory points in F are calculated and summed. Similarly, when there is only one trajectory point in F, the distances between f1 and all trajectory points in E are calculated and summed. When there are multiple trajectory points in both E and F, the distance between the last trajectory point in each of the two trajectory sequences is first calculated, along with the DTW algorithm distances between Head(E) and F, E and Head(F), and Head(E) and Head(F). The minimum value of these calculated DTW algorithm distances is then summed with the distance between the last trajectory point in each of the two trajectory sequences.

[0134] Since the calculation of the above formula (15) is also affected by the different lengths of the trajectory sequences, the similarity distance can be obtained by the following formula (16). .

[0135] (16).

[0136] When operating the S520, based on the target trajectory behavior patterns, the target prediction sub-model corresponding to the trajectory sequence to be predicted is called from the target prediction model. The training process of the target prediction sub-model has been described above and will not be repeated here.

[0137] In operation S530, the trajectory data of the trajectory sequence to be predicted within the current time period is input into the target prediction sub-model to obtain the target prediction trajectory corresponding to the predetermined time period after the current time period.

[0138] According to an embodiment of the present invention, a teacher-forced strategy can be adopted, using known data of the trajectory sequence to be predicted as the input of the target prediction sub-model. For example, if the trajectory sequence from time 1 to time 5 is known, and the trajectory sequence from time 6 to time 9 is to be predicted, the data at time 6 can be predicted using the data at time 5. It is also necessary to consider the entire sequence from time 1 to time 5 and predict the data at time 6.

[0139] Specifically, the input to the target prediction sub-model needs to pass through a fully connected layer. The known trajectory sequence is input as a mapping result into the decoder within the target prediction sub-model. This allows for in-depth analysis of the dependencies between the known and predicted trajectory sequences, uncovering deeper patterns in the data, and leveraging long-range correlations in the trajectory data to generate more accurate trajectory prediction results. Simultaneously, the output of the target prediction sub-model also needs to pass through a fully connected layer, outputting prediction data corresponding to multiple moments within a predetermined time period.

[0140] Figure 6 The diagram illustrates an electronic device suitable for implementing a model training method and a trajectory prediction method based on multidimensional feature fusion according to an embodiment of the present invention. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0141] like Figure 6 As shown, an electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0142] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0143] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0144] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0145] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0146] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] For example, according to embodiments of the present invention, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0148] Embodiments of the present invention also include a computer program product, which includes a computer program containing program code for executing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the model training method and trajectory prediction method provided in the embodiments of the present invention.

[0149] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this embodiment of the invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0150] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0151] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or pairings fall within the scope of this invention.

[0153] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A model training method based on multi-dimensional feature fusion, characterized in that, include: Clustering is performed on multiple trajectory sequences to obtain trajectory clusters corresponding to the behavioral patterns of each trajectory, including: Based on a multi-dimensional pre-planning strategy, the pre-processed trajectory sequences are segmented to obtain multiple target trajectory subsequences. Based on a sliding window, feature extraction is performed on the multiple target trajectory subsequences to obtain feature sequences corresponding to each of the multiple target trajectory subsequences; Based on multiple feature sequences, clustering is performed on the multiple trajectory sequences to obtain multiple trajectory clusters corresponding to the behavior patterns of each of the multiple trajectories. The multidimensional predetermined strategy includes a velocity strategy and a heading strategy. The trajectory sequence includes multiple trajectory point data, and the trajectory point data includes heading angle data and acceleration data. The target trajectory subsequence includes a variable speed straight trajectory subsequence, a uniform speed straight trajectory subsequence, a variable speed non-linear trajectory subsequence, or a uniform speed non-linear trajectory subsequence. Spatiotemporal features are extracted from multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets for each trajectory cluster. The bidirectional temporal feature sets include forward and backward features in the time dimension, and the spatial feature sets include features in the spatial dimension. Using a word embedding model, the auxiliary semantic information of each of the multiple trajectory sequences is semantically encoded to obtain the auxiliary feature sets of each of the multiple trajectory clusters; Based on the correlation between the bidirectional time feature sets, spatial feature sets, and auxiliary feature sets of the multiple trajectory clusters, feature fusion is performed on the multiple trajectory clusters respectively to obtain the multidimensional fused feature sets of the multiple trajectory clusters; The initial model is trained using the multidimensional fusion feature sets of each of the multiple trajectory clusters until the model parameters meet the predetermined convergence conditions, thereby obtaining the target prediction model. The target prediction model includes target prediction sub-models corresponding to the behavior patterns of each of the multiple trajectories.

2. The method according to claim 1, characterized in that, The step of extracting spatiotemporal features from multiple trajectory clusters to obtain bidirectional temporal feature sets and spatial feature sets for each trajectory cluster includes: Using a multi-layer convolutional neural network, spatial dimension features are extracted from multiple trajectory clusters to obtain spatial feature sets for each trajectory cluster. Using a bidirectional long short-term memory network, bidirectional feature extraction is performed on the multiple target trajectory subsequences in the time dimension to obtain the bidirectional time features of each of the multiple target trajectory subsequences. The bidirectional time features of each of the multiple target trajectory subsequences are concatenated to obtain the bidirectional time feature sets of each of the multiple trajectory clusters; In this context, the bidirectional time feature length in the bidirectional time feature set of each of the multiple trajectory clusters is equal to the spatial feature length in the spatial feature set.

3. The method according to claim 1, characterized in that, The multi-dimensional pre-defined strategy involves segmenting the preprocessed trajectory sequences to obtain multiple target trajectory sub-sequences, including: For each preprocessed trajectory sequence Based on the heading strategy and multiple heading angle data of the preprocessed trajectory sequence, the trajectory sequence is segmented to obtain straight trajectory subsequences and non-straight trajectory subsequences corresponding to the trajectory sequence; Based on the velocity strategy, multiple acceleration data of the straight trajectory subsequence, and multiple acceleration data of the non-straight trajectory subsequence, the straight trajectory subsequence and the non-straight trajectory subsequence are respectively segmented to obtain at least one of the variable speed straight trajectory subsequence, the uniform speed straight trajectory subsequence, the variable speed non-straight trajectory subsequence, or the uniform speed non-straight trajectory subsequence.

4. The method according to claim 2, characterized in that, The bidirectional long short-term memory network includes a first network, a second network, a third network, and a fourth network. The bidirectional long short-term memory network is used to extract bidirectional features from the multiple target trajectory subsequences in the time dimension, and the bidirectional time features of each of the multiple target trajectory subsequences are concatenated to obtain the bidirectional time feature sets of each of the multiple trajectory clusters, including: The variable speed straight trajectory subsequence is input into the first network, and the time sequence and time reverse sequence features of the variable speed straight trajectory subsequence are extracted to obtain the bidirectional time features of the variable speed straight trajectory subsequence. The uniform linear trajectory subsequence is input into the second network, and the time sequence and time reverse sequence features of the uniform linear trajectory subsequence are extracted to obtain the bidirectional time features of the uniform linear trajectory subsequence. The uniform non-linear trajectory subsequence is input into the third network, and the time sequence and time reverse sequence features of the uniform non-linear trajectory subsequence are extracted to obtain the bidirectional time features of the uniform non-linear trajectory subsequence. The variable-speed non-linear trajectory subsequence is input into the fourth network, and the time sequence and time inversion features of the variable-speed non-linear trajectory subsequence are extracted to obtain the bidirectional time features of the variable-speed non-linear trajectory subsequence. The bidirectional time features of each of the variable-speed linear trajectory subsequence, the uniform-speed linear trajectory subsequence, the variable-speed non-linear trajectory subsequence, and the uniform-speed non-linear trajectory subsequence are projected onto the same dimensional space to obtain the bidirectional time feature sets of each of the multiple trajectory clusters.

5. The method according to claim 2, characterized in that, The multi-layer convolutional neural network includes a cascaded first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer; the spatial feature extraction of the multiple trajectory clusters using the multi-layer convolutional neural network to obtain the spatial feature sets of each trajectory cluster includes: The multiple trajectory sequences are converted into multiple trajectory images respectively; The first feature map is obtained by using the first convolutional layer and the first pooling layer to extract local shallow features from the multiple trajectory images; The second convolutional layer and the second pooling layer are used to perform global deep feature extraction on the first feature map to obtain the spatial feature sets of each of the multiple trajectory clusters.

6. The method according to claim 1 or 2, characterized in that, The step involves using a word embedding model to semantically encode the auxiliary semantic information of each of the multiple trajectory sequences, resulting in auxiliary feature sets for each of the multiple trajectory clusters, including: Using an embedding lookup table or the embedding layer of the word embedding model, the auxiliary semantic information of each of the multiple trajectory sequences is mapped from text form to feature vectors respectively; By concatenating the feature vectors corresponding to the auxiliary semantic information of the multiple trajectory sequences, the auxiliary feature sets of the multiple trajectory clusters are obtained.

7. The method according to claim 1, characterized in that, The step of clustering the multiple trajectory sequences based on multiple feature sequences to obtain trajectory clusters corresponding to the respective trajectory behavior patterns includes: Using a target depth autoencoder, multiple feature sequences are compressed and reconstructed to obtain a low-dimensional reconstruction sequence. The target depth autoencoder is obtained by adjusting the parameters of an initial autoencoder based on a regularization norm. The regularization norm is determined based on the reconstruction error between the reconstructed data and the sample data. The reconstructed data is obtained by inputting the sample data into the initial autoencoder. Using the low-dimensional reconstruction sequence, the multiple trajectory sequences are clustered to obtain trajectory clusters corresponding to the behavioral patterns of each of the multiple trajectories.

8. A trajectory prediction method based on multi-dimensional feature fusion, characterized in that, include: Based on the dynamic time warping algorithm, the behavior pattern of the target trajectory corresponding to the trajectory sequence to be predicted is determined; Based on the target trajectory behavior pattern, a target prediction sub-model corresponding to the trajectory sequence to be predicted is called from the target prediction model, wherein the target prediction model is trained by the method according to any one of claims 1 to 7; The trajectory data of the trajectory sequence to be predicted within the current time period is input into the target prediction sub-model to obtain the target prediction trajectory corresponding to the predetermined time period after the current time period.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 8.

Images