Multi-target interrupt trajectory completion method based on graph structure

Through the graph structure-based method, the multi-objective interrupt trajectory completion problem is solved in the existing technology, and the trajectory features are not fully utilized and the computational complexity is high, efficient correlation and completion of trajectories are achieved, and the multi-objective tracking system is improved. Accuracy and reliability.

CN119025824AInactive Publication Date: 2024-11-26HARBIN ENG UNIV
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411010846.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing trajectory completion method has not fully explored the trajectory features for traceability correlation and trajectory completion, and the model processing time is long and the computing power requirements are large.

Method used

Using the multi-objective interrupt trajectory completion method based on graph structure, the multi-objective interrupt trajectory data set is constructed to perform coarse and fine correlation, and the graph embedding module and feature pooling and full-connection layer module extract trajectory features to achieve fine trajectory Relevance and completion.

Benefits of technology

Effectively express and operate complex trajectory relationships, extract the structural characteristics of the entire trajectory, improve the accuracy and efficiency of trajectory completion, and reduce the complexity of model calculation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025824A_ABST
    Figure CN119025824A_ABST
Patent Text Reader

Abstract

The invention relates to a multi-target interrupt trajectory completion method, in particular to a multi-target interrupt trajectory completion method based on a graph structure. The method aims at solving the problems that an existing trajectory completion method cannot fully mine trajectory features for tracing association and trajectory completion, the model processing time is long, and the requirement for the computing power is large. The method comprises the following steps: step 1, constructing a multi-target interrupt trajectory data set; step 2, multi-target interrupt trajectory coarse correlation; step 3, carrying out fine association on multi-target interrupt tracks based on a graph structure; and step 4, multi-target interrupt trajectory completion based on a graph structure. The invention belongs to the technical field of multi-target tracking.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for completing interrupted trajectories of multiple targets, and belongs to the technical field of multiple target tracking. Background Art

[0002] In the field of multi-target tracking, target trajectory detection interruption is an important research direction. In practical applications, due to sensor failure, occlusion, environmental interference and other reasons, the trajectory data of the target is often interrupted, making it difficult to continuously track the target. Effectively processing and completing the interrupted trajectories of different targets is crucial to improving the accuracy and reliability of the multi-target tracking system. In recent years, many experts and scholars have proposed many innovative methods for completing the interrupted trajectories of multiple targets.

[0003] The invention patent with publication number CN115205334A and application date June 29, 2022 discloses a method for constructing global vehicle trajectories under incomplete perception data. This method combines the traffic characteristics of vehicle operation, tracks and extracts vehicle trajectories under multiple cameras and smoothes the vehicle trajectory data obtained by radar, extracts spatiotemporal features from vehicle trajectory data, unifies the spatiotemporal coordinates of multiple sensors, integrates the spatiotemporal interaction features of vehicle trajectories, constructs a vehicle trajectory completion model, and completes and repairs missing trajectories. This method does not fully explore trajectory features for traceability association and trajectory completion.

[0004] The invention patent with publication number CN116363546A and application date of November 4, 2022 discloses a dynamic sparse trajectory completion method based on time series. The method constructs a mutual mapping model for mutual conversion between monitoring video and trajectory vector and obtains static road features and abnormal dynamic features based on time series of the target area. Based on the mutual mapping model and the static road features and abnormal dynamic features of the target area, a trajectory tracing model about the target area is constructed to obtain abnormal dynamic trajectories based on time series corresponding to the target time period, and output abnormal dynamic trajectories, and finally obtains the monitoring video of the sub-area in the target area during the required time period and inputs it into the trajectory tracing model. This method requires the construction of static road features and abnormal dynamic monitoring, and the model processing time is long and the computing power requirements are large. Summary of the invention

[0005] In order to solve the problems that the existing trajectory completion methods fail to fully explore trajectory features for tracing association and trajectory completion, and the model processing time is long and the computing power requirements are large, the present invention further proposes a multi-target interruption trajectory completion method based on graph structure.

[0006] The technical solution adopted by the present invention to solve the above problems is: the present invention specifically includes:

[0007] Step 1: Construct a multi-target interruption trajectory dataset;

[0008] Step 2: Coarse association of multi-target interruption trajectories;

[0009] Step 3: Multi-target interruption trajectory fine association based on graph structure;

[0010] Step 4: Multi-target interrupted trajectory completion based on graph structure.

[0011] Furthermore, in step 1, the trajectory obtained by randomly intercepting 10-30 consecutive points of the motion trajectory is used as the interruption trajectory T m , the truncated motion trajectory is used as the trajectory before and after the interruption T i , and mark the associated flag bit label to obtain the multi-target interruption trajectory dataset.

[0012] Furthermore, in step 2, the known trajectories before and after the interruption are roughly associated with the multi-target interruption trajectories, and the speed matching ρ is calculated. v and velocity direction ρ d Matching reduces the candidate combinations of multi-target interruption trajectories.

[0013] Furthermore, in step 3, the multi-target interruption trajectory fine association model based on graph structure is composed of a graph embedding module and a feature pooling and fully connected layer module in series. The graph embedding module is composed of a graph convolutional neural network, a homogeneous graph neural network and an activation layer. The graph embedding module aggregates the adjacent node information of each node in the trajectory of the graph structure, and then generates multi-layer graph data under the calculation of multiple parameter combinations to finally realize the three-dimensional position vector (x i ,y i ,z i ) to the multidimensional feature vector (a1, a2, K, a n ) i Finally, the feature pooling and fully connected layer modules output the associated flag to represent the association relationship between the trajectories before and after the interruption.

[0014] Furthermore, in step 4, the multi-target interruption trajectory completion model based on graph structure is divided into a graph embedding module and a trajectory restoration module. The graph embedding module inputs trajectory graph data and outputs a high-dimensional feature graph after processing. The trajectory restoration module inputs the high-dimensional feature graph and outputs the completed trajectory graph. The trajectory pairs with the association flag bit between the trajectories before and after the interruption being 1 are input into the multi-target interruption trajectory completion model. Finally, the completed trajectory graph is converted into trajectory data, and the complete trajectory data is output.

[0015] The beneficial effects of the present invention are:

[0016] 1. The graph structure is very suitable for representing complex trajectory relationships. In the graph, nodes can represent trajectory segments, and edges can represent the relationship between trajectory segments (such as spatial distance, time interval, directional consistency, etc.). It can flexibly express and operate trajectory data in scenarios with multiple targets and frequent interruptions;

[0017] 2. Convert the trajectory into a graph structure, and then use graph convolutional neural network and isomorphic graph neural network to extract the structural features of the entire trajectory;

[0018] 3. When the correlation relationship between the trajectories before and after the interruption is unknown, the trajectories belonging to the same target are found through the correlation of the trajectories before and after the interruption of multiple targets to complete the trajectories. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of the present invention;

[0020] Figure 2 It is a multi-target interruption trajectory fine correlation model diagram based on graph structure;

[0021] Figure 3 It is a multi-objective interrupted trajectory completion model diagram based on graph structure. DETAILED DESCRIPTION

[0022] Specific implementation method 1: Figures 1 to 3 As shown, a multi-target interrupted trajectory completion method based on a graph structure includes the following specific steps:

[0023] Step 1: Construct a multi-target interruption trajectory dataset;

[0024] The training data set uses the GeoLife GPS data set to obtain the complete trajectory T = [p1, p2, ..., p n ];

[0025] The complete trajectory T of different targets c is truncated any number of times i and any continuous duration Δt to generate the trajectory T before and after the interruption i and interruption trajectory M i :

[0026]

[0027] For the multi-target interruption trajectory fine association model, the method of making the corresponding data set is as follows: the trajectories before and after the interruption from the same target form a pair of trajectories as positive samples, and are labeled with a positive label b=1; the trajectories before and after the interruption from different targets form a pair of trajectories as negative samples, and are labeled with a negative label b=0.

[0028] For the multi-target interruption trajectory completion model, the method of making the corresponding data set is as follows: Generate the trajectory T before and after the interruption iand interruption trajectory M i .

[0029] Finally, we get a dataset of interrupted trajectories that can be used to train multi-targets;

[0030] Step 2: Coarse association of multi-target interruption trajectories;

[0031] The rough correlation of trajectories is achieved through speed matching, which includes speed magnitude matching and speed direction matching:

[0032]

[0033] Among them, ρ v and ρ d They represent the confidence of velocity magnitude and velocity direction respectively. is a vector representing the average velocity of trajectory i and trajectory j, v miss is a vector representing the velocity of the interrupted trajectory.

[0034] The coarse association of multi-target interrupted trajectories is realized by using velocity information, which reduces the candidate combinations of fine association of multi-target interrupted trajectories, reduces the model calculation complexity and speeds up the calculation speed.

[0035] Step 3: Multi-target interruption trajectory fine association based on graph structure;

[0036] The multi-target interruption trajectory fine association model based on graph structure should have the following functions: on the one hand, it can extract the features of trajectory points, consider the trajectory point v i and its surrounding trajectory points v j ∈N(v i ) on the other hand, it can extract the structural features of the entire trajectory and consider the motion characteristics of the target behind the entire trajectory. Therefore, when the trajectory data is input into the multi-target interruption trajectory fine association model based on the graph structure, the three-dimensional coordinates need to be converted into graph data to match the graph neural network;

[0037] The multi-target interruption trajectory fine association model consists of a graph embedding module and a feature pooling and fully connected layer module in series;

[0038] The graph embedding module consists of a graph convolutional neural network, a homogeneous graph neural network, and an activation layer. The graph embedding module aggregates the adjacent node information of each node in the trajectory of the graph structure, and then generates multi-layer graph data under the calculation of multiple parameter combinations to finally realize the three-dimensional position vector (x i ,y i ,z i ) to the multidimensional feature vector (a1, a2, K, a n ) i The mapping of

[0039] The graph convolutional neural network uses convolution operations to aggregate node neighbor information and update node representation. Its basic formula is:

[0040]

[0041] Among them, H 0 =X is the input of the first layer, j∈N i , N i is the number of nodes in the graph, D is the dimension of each node feature vector, is the matrix after adding the self-loop to the adjacency matrix A, yes The degree matrix, H l is the node feature matrix of the lth layer, W l The weight matrix of the lth layer;

[0042] The homogeneous graph neural network aggregates the features of nodes and their neighbors more flexibly through the multi-layer perceptron (MLP) and learnable parameters ò, and describes the graph convolution from the node perspective as follows:

[0043]

[0044] Among them, MLP l+1 is the multilayer perceptron used for the l+1th layer, l+1 is a learnable parameter used to control the weight of the node’s own features. They are the feature representation j∈N of node v at layer l. i , N(v) represents the set of neighbor nodes of node v;

[0045] The output features of the graph convolutional neural network and the isomorphic graph neural network are concatenated and then activated by the LeakyReLU function, expressed as:

[0046]

[0047] Where n is a fixed value, usually 0.01, and [·||·] represents the concatenation operation of features. is the output feature of the l+1th layer GCN, Output features of the l+1th layer of GIN;

[0048] After completing the mapping of the three-dimensional position vector of the trajectory point to the high-dimensional feature vector, the trajectory graph structure data with the feature vector is input into the subsequent feature pooling and fully connected layer modules to output the feature vector for the final association relationship judgment;

[0049] The graph neural network parameters of the graph embedding module and the feature pooling and fully connected layer modules can be obtained by iteratively updating the error back propagation. Due to the uneven distribution of positive and negative samples, different weights are assigned to samples of different categories, and the weighted cross entropy loss function is used as shown below:

[0050]

[0051] Where N is the total number of samples, y i is the true label (0 or 1) of the i-th sample, b i is the probability that the i-th sample is predicted to be 1, i.e., the associated flag bit, w i is the weight of the i-th sample;

[0052] If two trajectories come from the same target, the output association flag of the input interrupt trajectory tracing association model is 1, b = 1;

[0053] If the two trajectories come from different targets, the output association flag of the input interruption trajectory tracing association model is 0, b = 0;

[0054] Step 4: Multi-target interrupted trajectory completion based on graph structure;

[0055] The multi-target interruption trajectory completion model based on graph structure is divided into a graph embedding module and a trajectory restoration module. The graph embedding module inputs trajectory graph data and outputs a high-dimensional feature graph after processing. The trajectory restoration module inputs the high-dimensional feature graph and outputs the completed trajectory graph. Finally, the completed trajectory graph is converted into trajectory data.

[0056] The trajectories before and after the multi-target interruption are mathematically represented as known trajectories, n is the known trajectory length, and the completed trajectory is obtained by mapping f:

[0057] T=[T1,T2,…,T i ]

[0058] f(T)=f({(x0,y0,z0),(x1,y1,z1),L,(x n ,y n ,z n )})

[0059] ={(x n+1 ,y n+1 ,z n+1 ),(x n+2 ,y n+2 ,z n+2 ),L,(x n+m ,y n+m ,z n+m )},

[0060] The graph neural network model plays the role of simulating the mapping f, specifically, the trajectory T before and after the multi-target interruption i +T i+1 The corresponding graph structure G i +G i+1 Mapped to a high-dimensional feature map G M , and then G M Mapped into the combined trajectory T of the trajectory before and after the interruption and the interruption trajectory i +T m +T i+1 The corresponding graph G p , the mapping process is as follows:

[0061]

[0062] Among them, f1 represents the mapping function of the graph embedding module, and f2 represents the mapping function of the trajectory restoration module. First, the historical trajectory is arbitrarily supplemented with m trajectory points to convert it into a complete trajectory of length m+n. This step is to make the input graph data size match the graph convolutional neural network input. The graph convolutional neural network will not change the size of the graph. When the network output predicts that the trajectory graph size is m*n, the network input is also m*n. If the historical trajectory is directly converted to a graph size of only n, m trajectory points need to be supplemented.

[0063] The complete trajectory is converted into graph data. The input graph is passed through the maneuver trajectory prediction graph embedding module to obtain a high-dimensional feature graph, which is more conducive to accurate trajectory graph prediction in the high-dimensional space of the graph. The high-dimensional feature graph is then passed through the trajectory restoration module to obtain a three-dimensional position graph. At this time, the node parameters supplemented on the graph have been updated;

[0064] Finally, the updated three-dimensional position graph is converted into the form of trajectory sequence, and the three-dimensional position parameters of any supplementary interruption points on the trajectory are updated;

[0065] The error between the mapping result and the true value of the trajectory of the multi-target interrupted trajectory completion model based on the graph structure is back-propagated to update the parameters of the graph neural network model. The root mean square error used in training is as follows:

[0066]

[0067] in, Represents the three-dimensional coordinate value of the trajectory point completed by the model, and (x, y, z) represents the true three-dimensional coordinate value of the trajectory. The error of a single prediction is converted into the increment ΔL of each parameter in the model through the gradient descent algorithm. θ1 ,ΔL θ2 ,…,ΔL θn , the gradient descent algorithm is as follows:

[0068]

[0069] Among them, θ n+1 is the updated value of the parameter, θ n is the current value of the parameter, η is the learning rate or step size, which controls the incremental size of the parameter at each iteration to avoid missing the extreme point due to a large value, and to avoid taking too long to converge due to a small value. is the gradient, which is the fastest rising point of the function at the current position; J(θ) is the objective function currently simulated by the model.

[0070] Example

[0071] Step 1: Process the GeoLife GPS dataset, extract the movement trajectory of each target and save it;

[0072] Step 2: Randomly intercept 10-30 consecutive points of the motion trajectory, and the obtained trajectory is used as the interruption trajectory T m , the truncated motion trajectory is used as the trajectory before and after the interruption T i , and mark the associated flag bit label to obtain a multi-target interrupted trajectory dataset. After a complete trajectory sample is intercepted, n interrupted trajectory samples, n-1 associated flag bit labels and n-1 completed trajectory samples can be obtained;

[0073] Step 3: Obtain a multi-target interruption trajectory completion data set;

[0074] Step 4: Combine all the interrupted trajectory samples after being intercepted in pairs. The trajectories before and after the interruption from the same trajectory are positive samples and are labeled with positive labels. The trajectories before and after the interruption from different trajectories are negative samples and are labeled with negative labels.

[0075] Step 5: Obtain a multi-target interruption trajectory association data set;

[0076] Step 6: Roughly correlate the known multi-target interruption trajectory association data set, and calculate the speed matching ρ v and velocity direction ρ d Matching reduces the candidate combinations of fine associations of multi-target interruption trajectories;

[0077] Step 7: Build a multi-target interruption trajectory fine association model based on a graph structure, which is composed of a graph embedding module and a feature pooling and fully connected layer module in series;

[0078] Step 8: The graph embedding module consists of a graph convolutional neural network, a homogeneous graph neural network, and an activation layer. It aggregates the adjacent node information of each node in the trajectory of the graph structure, and then generates multi-layer graph data under the calculation of multiple parameter combinations to finally realize the three-dimensional position vector (x i ,y i ,z i ) to the multidimensional feature vector (a1, a2, K, a n )i The mapping of graph convolutional neural networks uses convolution operations to aggregate node neighbor information, and homomorphic graph neural networks aggregate the features of nodes and their neighbors more flexibly through multi-layer perceptrons and learnable parameters.

[0079] Step 9: Input the result of the coarse association of the multi-target interruption trajectory into the multi-target interruption trajectory fine association model based on the graph structure, convert the trajectory data into a graph structure trajectory, train the model, calculate the error through the weighted cross entropy loss function, and iteratively update the graph neural network parameters of the graph embedding module and the feature pooling and fully connected layer modules through back propagation;

[0080] Step 10: finally output the associated flag bit;

[0081] Step 11: Build a multi-target interruption trajectory completion model based on a graph structure, which is divided into a graph embedding module and a trajectory restoration module;

[0082] Step 12: The graph embedding module inputs the trajectory graph data and outputs a high-dimensional feature graph after processing, and the trajectory restoration module inputs the high-dimensional feature graph and outputs a completed trajectory graph;

[0083] Step 13: Input the trajectory pairs with the correlation flag bit between the trajectories before and after the interruption as 1 into the multi-target interruption trajectory completion model based on the graph structure, calculate the root mean square error and convert it into the increment of each parameter in the model through the gradient descent algorithm to train the model;

[0084] Step 14: Convert the completed trajectory graph output by the multi-target interrupted trajectory completion model based on the graph structure into a complete trajectory.

[0085] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A multi-target interrupted trajectory completion method based on graph structure, characterized in that: Specifically include: Step 1: Construct a multi-target interruption trajectory dataset; Step 2: Coarse association of multi-target interruption trajectories; Step 3: Multi-target interruption trajectory fine association based on graph structure; Step 4: Multi-target interrupted trajectory completion based on graph structure.

2. The method for completing multi-target interrupted trajectories based on graph structure according to claim 1, characterized in that: In step 1, the trajectory obtained by randomly intercepting 10-30 consecutive points of the motion trajectory is used as the interruption trajectory T m , the truncated motion trajectory is used as the trajectory before and after the interruption T i , and mark the associated flag bit label to obtain the multi-target interruption trajectory dataset.

3. The method for completing multi-target interrupted trajectories based on graph structure according to claim 1, characterized in that: In step 2, the known trajectories before and after the interruption are roughly associated with the multi-target interruption trajectory, and the speed size is matched by calculating ρ v and velocity direction ρ d Matching reduces the candidate combinations of multi-target interruption trajectories.

4. The method for completing multi-target interrupted trajectories based on graph structure according to claim 1, characterized in that: In step 3, the multi-target interruption trajectory fine association model based on graph structure is composed of a graph embedding module and a feature pooling and fully connected layer module in series. The graph embedding module is composed of a graph convolutional neural network, a homogeneous graph neural network and an activation layer. The graph embedding module aggregates the adjacent node information of each node in the trajectory of the graph structure, and then generates multi-layer graph data under the calculation of multiple parameter combinations to finally realize the three-dimensional position vector (x i ,y i ,z i ) to the multidimensional feature vector (a1, a2, K, a n ) i Finally, the feature pooling and fully connected layer modules output the associated flag to represent the association relationship between the trajectories before and after the interruption.

5. The method for completing multi-target interrupted trajectories based on graph structure according to claim 1, characterized in that: In step 4, the multi-target interruption trajectory completion model based on graph structure is divided into a graph embedding module and a trajectory restoration module. The graph embedding module inputs the trajectory graph data and outputs a high-dimensional feature graph after processing. The trajectory restoration module inputs the high-dimensional feature graph and outputs the completed trajectory graph. The trajectory pairs with the association flag bit between the trajectories before and after the interruption as 1 are input into the multi-target interruption trajectory completion model. Finally, the completed trajectory graph is converted into trajectory data and the complete trajectory data is output.

Citation Information

Patent Citations

  • Missing POI track completion method based on mask and bidirectional model

    CN114116692A

  • Track backtracking and alarm event tracing method based on interactive model

    CN115034059A

  • Urban scene-oriented pedestrian trajectory prediction method, model and storage medium

    CN115071762A

  • Road network track completion method based on learning interpolation prediction

    CN115905434A

  • Track processing method, device and equipment

    CN117075073A