Method and system for long time series point cloud scene flow estimation

Abstract

The application discloses a method and system for long-time sequence point cloud scene flow estimation, relates to the technical field of three-dimensional point cloud processing and motion estimation, and aims to solve the problem of poor time sequence consistency caused by the limitation of adjacent frame pairing in the prior art, which is easily affected by geometric shape change, occlusion and error accumulation in long-time sequence. The application constructs an end-to-end estimation framework, first divides the long-time sequence point cloud sequence into partially overlapped sliding windows, and completes query point trajectory initialization through feature extraction and adjacent interpolation; then constructs geometric matching relationship from the point level and the voxel level, fuses spatio-temporal information to obtain fused features; and performs multi-round iteration optimization across frames and within frames in the sliding window, and realizes trajectory state transmission between sliding windows. The application enhances geometric perception and global displacement constraint through double-scale matching to suppress drift, reasonably infers the position of the occluded point, suppresses error accumulation, and improves the cross-frame corresponding consistency, trajectory continuity and stability of long-time sequence scene flow estimation.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud processing and motion estimation technology, and in particular to a method and system for long-time-series point cloud scene flow estimation. Background Technology

[0002] 3D point cloud scene flow estimation is a core technology in fields such as autonomous driving, robot navigation, and virtual reality, accurately inferring the 3D motion trajectory of each point in a scene from a long-term point cloud sequence. Existing technologies mostly employ estimation methods that pair adjacent frames, which presents the following key problems in practical applications:

[0003] (1) Insufficient temporal adaptability. In long temporal scenes, object shape deformation, viewpoint movement, and point cloud sampling differences can lead to the failure of local feature matching, making it impossible to maintain point-level temporal consistency.

[0004] (2) Trajectory continuity is impaired. In real-world scenarios, when occlusion and out-of-view phenomena are frequent, relying solely on information from adjacent frames is insufficient to infer the intermediate trajectory of the occluded point, leading to trajectory interruption or deviation.

[0005] (3) The accumulation of errors is serious. The discreteness of point cloud data leads to the lack of a global motion information fusion mechanism in existing methods. The estimation errors of adjacent frames accumulate over time, and the trajectory accuracy drops significantly over long time series.

[0006] The aforementioned problems make it difficult for existing technologies to output continuous, accurate, and stable three-dimensional motion trajectories in real-world scenarios with long time series, high dynamics, and multiple occlusions, thus limiting their application in fields with high reliability requirements such as autonomous driving.

[0007] To address the aforementioned issues, some existing technologies attempt to incorporate multi-frame information to improve robustness, but fail to address the core challenge of long-term point cloud scene flow estimation. For example, CN111476822B proposes a scene flow-based LiDAR target detection and motion tracking method, which uses voxelized multi-frame point clouds for target detection and tracking. However, the scene flow is only used as an auxiliary feature in the fusion process, without modeling the continuity of point-level trajectories over long time. CN109522832B proposes a loop closure detection method based on point cloud segment matching constraints and trajectory drift optimization, focusing on global trajectory optimization in LiDAR mapping and correcting cumulative drift through loop closure detection. However, its core objective is map construction rather than point-level scene flow estimation. These solutions lack segmentation mechanisms suitable for long sequences, a systematic process for initial trajectory generation, cross-frame geometric matching, and state transfer between sliding windows, and robustness in geometric relationship modeling through point-level and voxel-level dual-scale matching. In particular, they lack trajectory memory and inference mechanisms for occluded scenes, making it impossible to achieve stable continuity of occluded point trajectories. Therefore, there is an urgent need to propose a new method to break through the current technical bottleneck of long-term point cloud motion perception. Summary of the Invention

[0008] In view of this, the present invention proposes a method and system for long-time-series point cloud scene flow estimation. By constructing a complete framework specifically for long-time-series point cloud scene flow estimation, the initial trajectory generation, multi-scale geometric verification, and cross-sliding window state transfer are co-optimized, thereby effectively suppressing error accumulation and ensuring trajectory continuity and accuracy.

[0009] This invention provides a method for flow estimation in long-time-series point cloud scenes, comprising the following steps:

[0010] S1. Divide the long-time point cloud sequence into multiple sliding windows with partial temporal overlap. Each sliding window contains point cloud data from multiple consecutive time frames. Extract features from the point cloud data of each time frame within each sliding window to obtain high-dimensional point cloud features. Based on a given query point, estimate the features of the query point in each time frame using nearest neighbor interpolation combined with the high-dimensional point cloud features to obtain the initial trajectory of the query point within the sliding window.

[0011] S2. Within each sliding window, a spatiotemporal fusion feature is constructed. Based on the spatiotemporal fusion feature, the initial trajectory of the query point is iteratively optimized in multiple rounds across the frame temporal dimension and the intra-frame spatial dimension to obtain the optimized trajectory state of the query point in the current sliding window. This optimized trajectory state is then passed to the next sliding window in the time series to guide the trajectory estimation of the next sliding window. All sliding windows are processed sequentially, and the optimized trajectory states output by each sliding window are concatenated to finally obtain the final motion trajectory of the query point in the entire long time series.

[0012] Among them, the spatiotemporal fusion feature is obtained by fusing dual-scale geometric matching relationship, temporal sequence information of query points, motion change information and spatial location information; the dual-scale geometric matching relationship includes point-level scale and voxel-level scale.

[0013] Furthermore, the initialization trajectory of the query point within the sliding window includes the following steps:

[0014] S11. Divide the input long-time point cloud sequence into multiple sliding windows with overlapping time components. , where n is the sliding window index and N is the total number of sliding windows;

[0015] S12, Process the first sliding window :

[0016] right Each time frame For raw point cloud data A point cloud feature extraction network is used to perform layer-by-layer feature encoding, mapping the point cloud to a high-dimensional feature space. The feature extraction network employs a deep neural network with a hierarchical downsampling structure, which outputs the high-dimensional point cloud features corresponding to each point through local neighborhood aggregation and progressive downsampling. ,in Let D be the number of points in the i-th frame, and D be the feature dimension.

[0017] Given a query point q, search for its K nearest neighbors in the first frame and utilize the high-dimensional point cloud features. And the nearest neighbor interpolation algorithm is used to estimate the feature representation of the query point in frame 0. ;

[0018] The spatial coordinates q of the query point in frame 0 are used as its initial position estimate. The initial trajectory segment of the first sliding window is obtained. ;

[0019] Perform multiple rounds of iterative optimization on the initial trajectory segment to obtain the optimized trajectory state of the query point within the first sliding window. ;

[0020] S13, Process subsequent sliding windows And n≥2:

[0021] Take the previous sliding window The optimized trajectory status is output in the overlapping region;

[0022] Determine the time index of the last frame in the overlapping region ;

[0023] For overlapping frames It directly inherits the initialization result of the previous sliding window;

[0024] For non-overlapping frames, initialize the state of the last frame in the overlapping region. Assign the value directly to the current sliding window The initialization state of all non-overlapping frames in the middle;

[0025] The current sliding window is obtained after merging. Complete initial trajectory segment .

[0026] Furthermore, the calculation formula for the nearest neighbor interpolation algorithm is as follows:

[0027] ;

[0028] in, This is the high-dimensional feature estimate for query point q. The high-dimensional feature of the j-th neighboring point Let be the Euclidean distance between the j-th neighboring point and the query point. This represents the number of neighboring points. These are the interpolation weights based on the inverse distance normalization.

[0029] Furthermore, the construction of the spatiotemporal fusion feature includes the following steps:

[0030] S21. In the i-th frame of the current sliding window, with the current trajectory estimation position of the query point as the center, perform point-level scale geometric matching to obtain point-level matching features, and perform voxel-level scale geometric matching to obtain voxel-level matching features.

[0031] S22. Obtain the absolute time index t of the query point in the current frame, as the time order information in the spatial-temporal context;

[0032] S23. Calculate the displacement vector of the query point from the initial frame of the long-term time series to the current frame. , as information about motion changes;

[0033] S24. Obtain the three-dimensional coordinates of the query point in the current frame. , as spatial location information;

[0034] S25. Input the point-level matching features, voxel-level matching features, temporal sequence information, motion change information and spatial location information into the corresponding feature encoding functions for mapping to obtain high-dimensional embedding representations of each information.

[0035] S26. Concatenate all high-dimensional embedding representations to form the spatiotemporal fusion feature of the current frame.

[0036] Furthermore, the point-scale geometric matching step includes:

[0037] Using the estimated location of the query point in the current time frame as the center, search for K nearest neighbors in the local neighborhood;

[0038] For each neighboring point, calculate the geometric correlation score between the high-dimensional features and the historical motion state of the query point. ,in , The high-dimensional feature of the j-th neighboring point The predicted motion vector for the query point. The cosine similarity function;

[0039] Select the M nearest neighbors with the highest relevance scores, where M≤K. Then, extract the high-dimensional features of each selected neighbor. Spatial offset vector relative to the query point By concatenating the features, a joint feature vector is obtained. ;

[0040] The M joint feature vectors are input into a multilayer perceptron network for feature aggregation, and the output is a point-level geometric matching relation representation. .

[0041] Furthermore, the voxel-level scale geometric matching specifically includes:

[0042] Construct a local cubic region of size d×d×d centered on the estimated location of the query point in the current time frame;

[0043] The local cubic region is uniformly divided into a×a×a voxels, where a≥2;

[0044] For each voxel, count the number of its neighboring points and extract the high-dimensional features of all points.

[0045] Calculate the feature mean of all points within each voxel. As a voxel characteristic;

[0046] Concatenate all voxel features into a vector The input is a multilayer perceptron network for feature aggregation, and the output is a voxel-level geometric matching relationship representation. .

[0047] Further, in step S26, the calculation formula for the spatiotemporal fusion feature is:

[0048] ;

[0049] Where k is the number of iterations. To integrate geometric matching relationships, the features are obtained by concatenating point-level and voxel-level matching features and then inputting them into an MLP network for aggregation. The semantic features of the current trajectory, To determine the position of the query point in the initial frame of the long time series, The relative displacement of the query point relative to the initial frame. The feature encoding function for motion changes, For spatial location feature encoding function, For timestamps, It is a feature encoding function for time sequence.

[0050] Furthermore, the multi-round iterative optimization specifically includes:

[0051] In the k-th iteration, the trajectory estimation state is based on the query point within the current sliding window. Reconstruct the spatiotemporal fusion features of each time frame ;

[0052] Based on the spatiotemporal fusion features Jointly optimize the following objectives:

[0053] Intra-frame spatial consistency: bringing the query point location closer to a local structure with a high geometric matching response;

[0054] Cross-frame temporal smoothness: Constrain the velocity or acceleration changes between adjacent frames to not exceed a preset threshold;

[0055] Update the trajectory state of the query point by minimizing the comprehensive loss function. With semantic features The optimization results of the kth round are obtained. ;

[0056] Repeat the above process until the preset number of iterations or trajectory convergence is reached.

[0057] Furthermore, the multi-round iterative optimization of the cross-frame temporal dimension and intra-frame spatial dimension includes:

[0058] In each iteration, the trajectory state of the query point is simultaneously modeled across frames in terms of temporal sequence and within frames in terms of spatial sequence.

[0059] Cross-frame temporal modeling: For each query point, the position and features of its time frame within the current sliding window are used as input, and the inter-frame dependency is constructed through a self-attention mechanism;

[0060] Intra-frame spatial modeling: For multiple query points in the current time frame, using their spatial coordinates and semantic features as nodes, intra-frame spatial relationships are constructed through KNN;

[0061] The inter-frame dependencies and intra-frame spatial relationships are integrated to jointly guide the updating of the trajectory status of the query point.

[0062] Furthermore, the present invention also provides a system for long-time-series point cloud scene flow estimation, comprising: a sliding window segmentation module for dividing the input long-time-series point cloud sequence into multiple partially overlapping sliding windows; an initial trajectory generation module for performing neighbor interpolation on the adjacent high-dimensional point cloud features of a given query point based on the point cloud data of the global initial frame, thereby generating the initial trajectory of the query point on the sliding window; and a feature fusion module for calculating the geometric matching relationship between the query point and the local point cloud at both the point-level and voxel-level scales, centered on the current trajectory estimation position, and fusing the temporal order information of the query point in each iteration of each sliding window. The system constructs a spatiotemporal fusion feature based on the information, motion change information relative to the global initial position, and current spatial position information. An optimization module performs multiple rounds of iterative optimization on the trajectory state of the query point within each sliding window, based on the spatiotemporal fusion feature. Each optimization round simultaneously constrains intra-frame spatial geometric consistency and cross-frame temporal motion smoothness. A state transfer module transfers the optimized trajectory state of the current sliding window to the next sliding window as initial guidance for trajectory estimation in the next sliding window. A trajectory output module splices the optimized trajectory segments output by each sliding window to generate the final motion trajectory of the query point in the entire long-term point cloud sequence.

[0063] The present invention has the following advantages over the prior art:

[0064] (1) Suppress long-term time-series error accumulation. By using partially overlapping sliding windows and cross-sliding window trajectory transfer mechanisms, the limitations of adjacent frame pairing are broken, and errors are avoided from accumulating over time. Combined with the preset number of iterations and the fusion of time and space feature updates, the trajectory state gradually converges, which not only ensures the global consistency of motion estimation under long-term time-series sequences, but also further reduces estimation errors and improves trajectory smoothness.

[0065] (2) To solve the problem of trajectory interruption caused by occlusion, the discriminative power of local features is enhanced by the two-dimensional geometric matching relationship at the point level and voxel level. Combined with the spatiotemporal fusion features of time, motion and space, the structural and motion information across the time range is fully utilized to reasonably infer the position of the occlusion point, maintain the continuity of the trajectory, and make the trajectory inference in the occlusion scene more accurate.

[0066] (3) Improve long-term consistency. Through multi-round iterative optimization across frames and within frames, effectively model the temporal evolution of query points, resist the failure of point-level correspondence caused by object shape deformation and viewpoint changes, ensure the stability of trajectory under long-term conditions, and significantly improve the temporal consistency of point-level motion. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a flowchart illustrating the overall architecture of an embodiment of the present invention;

[0069] Figure 2 This is a schematic diagram of a method according to an embodiment of the present invention;

[0070] Figure 3 This is a system schematic diagram according to an embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0072] like Figure 1 , Figure 2 , Figure 3 As shown, this invention provides a method for flow estimation in long-time-series point cloud scenes, comprising the following steps:

[0073] S1. Divide the long-time point cloud sequence into multiple sliding windows with partial temporal overlap. Each sliding window contains point cloud data from multiple consecutive time frames. Extract features from the point cloud data of each time frame within each sliding window to obtain high-dimensional point cloud features. Based on a given query point, estimate the features of the query point in each time frame using nearest neighbor interpolation combined with the high-dimensional point cloud features to obtain the initial trajectory of the query point within the sliding window.

[0074] In this embodiment, the query point is the basic tracking unit used to characterize the motion state of a 3D scene. Essentially, it is a virtual spatial location with a unique identifier, and it is not required to coincide with any physical sampling point in the original point cloud. In practical applications, query points are typically generated from the first frame of the point cloud in the first sliding window, either pre-defined or uniformly sampled according to task requirements. Once determined, the query point maintains a constant ID throughout the long-term processing and serves as an anchor point for trajectory estimation and state transfer across sliding windows.

[0075] Specifically, this invention addresses the common problems of occlusion, sparse point clouds, and unstable observation in real-world scenarios by optimizing the query point: When the area where the query point is located is not covered by a corresponding point cloud in a certain frame due to occlusion or sensor field of view limitations, its position does not depend on local geometric observations, but is predicted by the trajectory state passed by the previous sliding window, and the continuous trajectory is restored in subsequent visible frames; When the geometric features of the query point's neighborhood are unstable due to low-density sampling or noise interference, the system uses a dual-scale verification mechanism to simultaneously verify local geometric consistency and global motion rationality. The trajectory is only updated when both verifications pass; otherwise, the predicted state is maintained. Therefore, the query point trajectory remains stable and does not diverge during occlusion, and quickly converges to the true position when it becomes visible again.

[0076] In long-term temporal 3D scene understanding, the query point serves as a virtual tracking unit. The core challenge is to endow it with stable and transferable high-dimensional semantic-geometric feature representations under the conditions of sparse, unstructured, and discontinuous point clouds between frames. Traditional methods typically require the query point to coincide with the actual sampling point in a frame, or rely on complex point cloud completion / interpolation networks, which are not only computationally expensive but also prone to failure in occluded or low-density regions.

[0077] In this embodiment, the initialization trajectory of the query point within the sliding window includes the following steps:

[0078] S11. Divide the input long-time point cloud sequence into multiple sliding windows with overlapping time components. , where n is the sliding window index and N is the total number of sliding windows;

[0079] S12, Process the first sliding window :

[0080] right Each time frame For raw point cloud data A point cloud feature extraction network performs layer-by-layer feature encoding to map the point cloud to a high-dimensional feature space. The feature extraction network adopts a hierarchical downsampling architecture based on PointNet++ or KPConv, and outputs the high-dimensional point cloud features corresponding to each point through local neighborhood aggregation and progressive downsampling. ,in Let D be the number of points in the i-th frame, and D be the feature dimension.

[0081] Given a query point q, search for its K nearest neighbors in the first frame and utilize the high-dimensional point cloud features. And the nearest neighbor interpolation algorithm is used to estimate the feature representation of the query point in frame 0. ;

[0082] Preferably, the nearest neighbor interpolation algorithm adopts a distance-weighted average form:

[0083] ;

[0084] in, This is the high-dimensional feature estimate for query point q. The high-dimensional feature of the j-th neighboring point Let be the Euclidean distance between the j-th neighboring point and the query point. This represents the number of neighboring points. These are the interpolation weights based on the inverse distance normalization.

[0085] This formula is not a simple linear interpolation, but a feature field sampling method based on physical spatial proximity. Essentially, it treats the discrete point cloud as an implicit feature field and performs bandwidth-adaptive sampling of this field at the query location. By introducing weights inversely proportional to distance, neighboring points closer to the query point contribute more to feature estimation, thus achieving physically reasonable, computationally efficient, and robust estimation of query point features without the need for a deep learning model.

[0086] The spatial coordinates q of the query point in frame 0 are used as its initial position estimate. The initial trajectory segment of the first sliding window is obtained. ;

[0087] Perform multiple rounds of iterative optimization on the initial trajectory segment to obtain the optimized trajectory state of the query point within the first sliding window. ;

[0088] S13, Process subsequent sliding windows And n≥2:

[0089] Take the previous sliding window The optimized trajectory status is output in the overlapping region;

[0090] Determine the time index of the last frame in the overlapping region ;

[0091] For overlapping frames It directly inherits the initialization result of the previous sliding window;

[0092] For non-overlapping frames, initialize the state of the last frame in the overlapping region. Assign the value directly to the current sliding window The initialization state of all non-overlapping frames in the middle;

[0093] The current sliding window is obtained after merging. Complete initial trajectory segment .

[0094] Specifically, all sliding windows are processed sequentially. Because the sliding windows overlap in time, adjacent sliding windows may output multiple initial position estimates at the same time. To address this, a multi-round iterative optimization is employed, taking the result of the first sliding window and assigning values ​​to the overlapping area, to obtain the optimized trajectory state of the first sliding window. This optimized trajectory serves as the starting point for subsequent iterative optimizations of multiple sliding windows, providing a reliable initial state for the entire scene flow estimation.

[0095] S2. Within each sliding window, a spatiotemporal fusion feature is constructed. Based on the spatiotemporal fusion feature, the initial trajectory of the query point is iteratively optimized in multiple rounds across the frame temporal dimension and the intra-frame spatial dimension to obtain the optimized trajectory state of the query point in the current sliding window. This optimized trajectory state is then passed to the next sliding window in the time series to guide the trajectory estimation of the next sliding window. All sliding windows are processed sequentially, and the optimized trajectory states output by each sliding window are concatenated to finally obtain the final motion trajectory of the query point in the entire long time series.

[0096] Among them, the spatiotemporal fusion feature is obtained by fusing dual-scale geometric matching relationship, temporal sequence information of query points, motion change information and spatial location information; the dual-scale geometric matching relationship includes point-level scale and voxel-level scale.

[0097] In this embodiment, as Figure 2 , Figure 3 As shown, a dual-scale geometric matching mechanism is introduced, and spatiotemporal context information is fused to construct highly discriminative spatiotemporal fusion features. The construction of spatiotemporal fusion features includes the following steps:

[0098] S21. In the i-th frame of the current sliding window, with the current trajectory estimation position of the query point as the center, perform point-level scale geometric matching to obtain point-level matching features, and perform voxel-level scale geometric matching to obtain voxel-level matching features.

[0099] S22. Obtain the absolute time index t of the query point in the current frame, as the time order information in the spatial-temporal context;

[0100] S23. Calculate the displacement vector Δqi=qi-q0 of the query point from the initial frame of the long time series to the current frame, as motion change information;

[0101] S24. Obtain the three-dimensional coordinates qi of the query point in the current frame as spatial location information;

[0102] S25. Input the point-level matching features, voxel-level matching features, temporal sequence information, motion change information and spatial location information into the corresponding feature encoding functions for mapping to obtain high-dimensional embedding representations of each information.

[0103] S26. Concatenate all high-dimensional embedding representations to form the spatiotemporal fusion feature of the current frame.

[0104] Specifically, spatiotemporal fusion features include three steps: point-level geometric matching, voxel-level geometric matching, and feature fusion.

[0105] (1) Point-scale geometric matching specifically includes: taking the estimated position of the query point in the current time frame as the center, searching for K nearest neighbors in the local neighborhood; for each neighbor, calculating the geometric correlation score between the high-dimensional features and the historical motion state of the query point. ,in , The high-dimensional feature of the j-th neighboring point The predicted motion vector for the query point. Using the cosine similarity function; selecting the M nearest neighbors with the highest relevance scores, where M≤K, and then processing the high-dimensional features of each selected neighbor. Spatial offset vector relative to the query point By concatenating the features, a joint feature vector is obtained. The M joint feature vectors are input into a multilayer perceptron network for feature aggregation, and the output is a point-level geometric matching relationship representation. Point-level geometric matching relationships are represented as follows: The calculation formula can be expressed as:

[0106] ;

[0107] Where k is the number of iterations. Let M be the position of the query point in the k-th iteration, and M be the number of highly relevant point cloud neighbors. For highly correlated point cloud neighbors, The relative displacement between the point cloud neighbors and the query point. The correlation features of highly correlated point cloud neighbors and query points.

[0108] (2) Voxel-level scale geometric matching specifically includes: constructing a local cubic region of size d×d×d centered on the estimated position of the query point in the current time frame; uniformly dividing the local cubic region into a×a×a voxels, where a≥2; for each voxel, counting the number of neighboring points contained within it, and extracting the high-dimensional features of all points; and calculating the feature mean of all points within each voxel. As voxel features; concatenate all voxel features into a vector. The input is a multilayer perceptron network (MLP) for feature aggregation, and the output is a voxel-level geometric matching representation. The voxel-level geometric matching relationship is represented by... The calculation formula is:

[0109] ;

[0110] Where k is the number of iterations, r is the neighborhood range in 3D space when the cube is partitioned, and a is the number of partitions along each coordinate axis. The cube's number. For the first A collection of point clouds within a cube For the correlation features of point clouds, For the first The number of point clouds in a cube.

[0111] (3) Constructing spatiotemporal fusion features specifically includes: concatenating point-level and voxel-level geometric matching relationships into geometric matching evidence; encoding the relative displacement, spatial location, and timestamp of the query point, fusing them with the geometric matching evidence and the semantic features of the current trajectory, and outputting spatiotemporal fusion features, the calculation formula of which is:

[0112] ;

[0113] Where k is the number of iterations. The fused geometric matching relationship is obtained by concatenating point-level matching features and voxel-level matching features and then inputting them into an MLP network for aggregation. The semantic features of the current trajectory, To determine the position of the query point in the initial frame of the long time series, The relative displacement of the query point relative to the initial frame. The feature encoding function for motion changes, For spatial location feature encoding function, For timestamps, It is a feature encoding function for time sequence.

[0114] The point-level and voxel-level matching results are concatenated to form a fused geometric matching relationship: Based on this, the movement changes, absolute spatial location, and temporal sequence information of the query point are integrated:

[0115] Relative displacement: , representing the cumulative motion from the initial position to the present;

[0116] Spatial location: Provides a global coordinate context;

[0117] Timestamp: t, which identifies the temporal position of the current frame in the sequence.

[0118] Each through a learnable encoding function , and The function maps the above three types of information into feature vectors. Among them, For the feature encoding function of motion changes, a 1-2 layer multilayer perceptron (MLP) is used, with each layer containing a ReLU activation function and a batch normalization layer; The feature encoding function for spatial location uses a 1-layer MLP with an output dimension of 128. For the temporal feature encoding function, sinusoidal positional encoding or a 1-layer MLP is employed; the network parameters of all encoding functions are jointly optimized during training. Finally, geometric matching evidence is used. Semantic features of the trajectory at the previous time step The three types of spatiotemporal coding features are concatenated to obtain the complete spatiotemporal fusion features. This spatiotemporal fusion feature serves as input to the subsequent trajectory optimization module, used to iteratively update the position and semantic features of the query point, thereby achieving high-precision, occlusion-resistant long-term motion tracking.

[0119] In this embodiment, within each sliding window, the system performs multiple rounds of iterative optimization on the initial trajectory state of the query point based on the aforementioned generated spatiotemporal fusion features. Through state transfer between sliding windows, the system ultimately obtains the complete motion trajectory of the query point throughout the entire long-term point cloud sequence. The multiple rounds of iterative optimization are used to fine-tune the trajectory state of the query point within each sliding window. Its core lies in simultaneously modeling cross-frame temporal dependencies and intra-frame spatial relationships, and achieving collaborative optimization by fusing the two. Specifically, it includes the following steps:

[0120] (1) Initialize the input for the current round

[0121] Let the current processing sliding window be... The query point trajectory state output in the (k-1)th round is: position sequence and characteristic sequences .

[0122] Concatenate the two and use the result as the input for the k-th round:

[0123] .

[0124] (2) Cross-frame temporal modeling based on self-attention mechanism

[0125] For a single query point, construct its dynamic evolution model over time:

[0126] Position coding injection: Add sinusoidal time position coding to each time frame t. , concatenate with input: .

[0127] Self-attention computation: The sequence Input standard Transformer encoder layer:

[0128] ;

[0129] in , It is a learnable projection matrix.

[0130] Output temporal context features: After LayerNorm and MLP, the temporal augmentation features are obtained: .

[0131] (3) Intra-space modeling based on KNN graph convolution

[0132] For the M query points in the current frame t, construct local geometric interactions.

[0133] First, construct a KNN graph and examine all query points in frame t. Using Euclidean distance as the metric, find K=8 nearest neighbors for each point to form an adjacency matrix. .

[0134] Next, message passing is performed, and for each query point mm, its neighbor features are aggregated:

[0135] ;

[0136] in It is a two-layer MLP (64→128 dimensions) with ReLU activation.

[0137] Finally, update the spatial features and fuse the aggregation result with its own features: .

[0138] (4) Spatiotemporal feature fusion and state update

[0139] First, feature alignment and concatenation are performed to combine the temporal features. Split by frame to obtain the global temporal context of each frame. Spatial features and splicing: .

[0140] Secondly, the joint decoding updates the state by synchronously predicting the position offset and feature updates through a shared MLP network: Final Update .

[0141] (5) Iteration termination judgment

[0142] If the preset maximum number of iterations is reached or the position change is less than the threshold for two consecutive rounds, the process terminates; otherwise, it returns to the cross-frame temporal modeling step based on the self-attention mechanism.

[0143] Through the closed-loop mechanism of iterative optimization within the sliding window and consistency calibration between sliding windows, this scheme effectively suppresses trajectory drift and jumps, and can still maintain high-precision and high-robust motion tracking performance in long time series, and finally outputs the final motion trajectory of the query point in the entire sequence.

[0144] In this embodiment, the system for long-time point cloud scene flow estimation can be deployed on autonomous vehicles or mobile robot platforms. Its modules are implemented through a deep learning framework and run on GPU-accelerated devices. The main components include: a sliding window segmentation module, which divides the input long-time point cloud sequence into multiple temporally overlapping sliding windows; an initial trajectory generation module, which extracts high-dimensional features from the point cloud within each sliding window frame by frame, and estimates the features of each frame based on a preset virtual query point through neighbor interpolation to generate the initial trajectory; a feature fusion module, which constructs a point-level and voxel-level dual-scale geometric matching relationship within each sliding window, and fuses the displacement, position, and temporal information of the query point to form a spatiotemporal fusion feature; an optimization module, which uses this fusion feature to perform multiple rounds of iterative optimization of the initial trajectory within a single sliding window, while considering cross-frame motion consistency and intra-frame geometric matching accuracy; a state transfer module, which transfers the optimized trajectory state of the current sliding window to the next sliding window; a consistency check is performed in the overlapping area of ​​the sliding windows, and smoothing correction is performed when necessary to ensure trajectory continuity; and a trajectory output module, which splices the optimized trajectory segments output by each sliding window to generate the final motion trajectory of the query point in the entire long-time point cloud sequence. The modules are connected in series to form a closed-loop processing pipeline, supporting end-to-end training and inference, and can efficiently process long temporal point cloud data of hundreds of frames with limited video memory.

[0145] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for flow estimation in long-time-series point cloud scenes, characterized in that, Includes the following steps: S1. Divide the long time-series point cloud sequence into multiple sliding windows with partial temporal overlap. Each sliding window contains point cloud data from multiple consecutive time frames. Extract features from the point cloud data of each time frame within each sliding window to obtain high-dimensional point cloud features. Based on a given query point, the features of the query point in each time frame are estimated by combining neighbor interpolation with high-dimensional point cloud features, and the initial trajectory of the query point within the sliding window is obtained. S2. Within each sliding window, construct spatiotemporal fusion features, and perform multi-round iterative optimization of the initial trajectory of the query point across frame temporal dimension and intra-frame spatial dimension based on the spatiotemporal fusion features to obtain the optimized trajectory state of the query point in the current sliding window, and pass it to the next sliding window in the time series to guide the trajectory estimation of the next sliding window. All sliding windows are processed sequentially, and the optimized trajectory states output by each sliding window are spliced ​​together to finally obtain the final motion trajectory of the query point in the entire long time series. Among them, the spatiotemporal fusion feature is obtained by fusing dual-scale geometric matching relationship, temporal sequence information of query points, motion change information and spatial location information; the dual-scale geometric matching relationship includes point-level scale and voxel-level scale. The construction of spatiotemporal fusion features includes the following steps: S21. In the i-th frame of the current sliding window, with the current trajectory estimation position of the query point as the center, perform point-level scale geometric matching to obtain point-level matching features, and perform voxel-level scale geometric matching to obtain voxel-level matching features. S22. Obtain the absolute time index t of the query point in the current frame, as the time order information in the spatial-temporal context; S23. Calculate the displacement vector of the query point from the initial frame of the long-term time series to the current frame. , as information about motion changes; S24. Obtain the three-dimensional coordinates of the query point in the current frame. , as spatial location information; S25. Input the point-level matching features, voxel-level matching features, temporal sequence information, motion change information and spatial location information into the corresponding feature encoding functions for mapping to obtain high-dimensional embedding representations of each information. S26. Concatenate all high-dimensional embedding representations to form the spatiotemporal fusion features of the current frame; The multi-round iterative optimization specifically includes: In the k-th iteration, the trajectory estimation state is based on the query point within the current sliding window. Reconstruct the spatiotemporal fusion features of each time frame ; Based on the spatiotemporal fusion features Jointly optimize the following objectives: Intra-frame spatial consistency: bringing the query point location closer to a local structure with a high geometric matching response; Cross-frame temporal smoothness: Constrain the velocity or acceleration changes between adjacent frames to not exceed a preset threshold; Update the trajectory state of the query point by minimizing the comprehensive loss function. With semantic features The optimization results of the kth round are obtained. ; Repeat the above process until the preset number of iterations is reached or the trajectory converges; The multi-round iterative optimization of the cross-frame temporal dimension and intra-frame spatial dimension includes: In each iteration, the trajectory state of the query point is simultaneously modeled across frames in terms of temporal sequence and within frames in terms of spatial sequence. Cross-frame temporal modeling: For each query point, the position and features of its time frame within the current sliding window are used as input, and the inter-frame dependency is constructed through a self-attention mechanism; Intra-frame spatial modeling: For multiple query points in the current time frame, using their spatial coordinates and semantic features as nodes, intra-frame spatial relationships are constructed through KNN; The inter-frame dependencies and intra-frame spatial relationships are integrated to jointly guide the updating of the trajectory status of the query point.

2. The method according to claim 1, characterized in that, The initialization trajectory of the query point within the sliding window includes the following steps: S11. Divide the input long-time point cloud sequence into multiple sliding windows with overlapping time components. , where n is the sliding window index and N is the total number of sliding windows; S12, Process the first sliding window : right Each time frame For raw point cloud data A point cloud feature extraction network is used to perform layer-by-layer feature encoding, mapping the point cloud to a high-dimensional feature space. The feature extraction network employs a deep neural network with a hierarchical downsampling structure, which outputs the high-dimensional point cloud features corresponding to each point through local neighborhood aggregation and progressive downsampling. ,in Let D be the number of points in the i-th frame, and D be the feature dimension. Given a query point q, search for its K nearest neighbors in the first frame and utilize the high-dimensional point cloud features. And the nearest neighbor interpolation algorithm is used to estimate the feature representation of the query point in frame 0. ; The spatial coordinates q of the query point in frame 0 are used as its initial position estimate. The initial trajectory segment of the first sliding window is obtained. ; Perform multiple rounds of iterative optimization on the initial trajectory segment to obtain the optimized trajectory state of the query point within the first sliding window. ; S13, Process subsequent sliding windows And n≥2: Take the previous sliding window The optimized trajectory status is output in the overlapping region; Determine the time index of the last frame in the overlapping region ; For overlapping frames It directly inherits the initialization result of the previous sliding window; For non-overlapping frames, initialize the state of the last frame in the overlapping region. Assign the value directly to the current sliding window The initialization state of all non-overlapping frames in the middle; The current sliding window is obtained after merging. Complete initial trajectory segment .

3. The method according to claim 2, characterized in that, The calculation formula for the nearest neighbor interpolation algorithm is as follows: ； in, For query point High-dimensional feature estimates, The high-dimensional feature of the j-th neighboring point Let be the Euclidean distance between the j-th neighboring point and the query point. W represents the number of neighboring points. j These are the interpolation weights based on the inverse distance normalization.

4. The method according to claim 1, characterized in that, The steps of the point-level scale geometric matching include: Using the estimated location of the query point in the current time frame as the center, search for K nearest neighbors in the local neighborhood; For each neighboring point, calculate the geometric correlation score between the high-dimensional features and the historical motion state of the query point. ,in , The high-dimensional feature of the j-th neighboring point The predicted motion vector for the query point. The cosine similarity function; Select the M nearest neighbors with the highest relevance scores, where M≤K. Then, extract the high-dimensional features of each selected neighbor. Spatial offset vector relative to the query point By concatenating the features, a joint feature vector is obtained. ; The M joint feature vectors are input into a multilayer perceptron network for feature aggregation, and the output is a point-level geometric matching relation representation. .

5. The method according to claim 4, characterized in that, The voxel-level scale geometric matching specifically includes: Construct a local cubic region of size d×d×d centered on the estimated location of the query point in the current time frame; The local cubic region is uniformly divided into a×a×a voxels, where a≥2; For each voxel, count the number of its neighboring points and extract the high-dimensional features of all points. Calculate the feature mean of all points within each voxel. As a voxel characteristic; Concatenate all voxel features into a vector The input is a multilayer perceptron network for feature aggregation, and the output is a voxel-level geometric matching relationship representation. .

6. The method according to claim 5, characterized in that, In step S26, the calculation formula for the spatiotemporal fusion feature is as follows: ； Where k is the number of iterations. To integrate geometric matching relationships, the features are obtained by concatenating point-level and voxel-level matching features and then inputting them into an MLP network for aggregation. The semantic features of the current trajectory, To determine the position of the query point in the initial frame of the long time series, The relative displacement of the query point relative to the initial frame. The feature encoding function for motion changes, For spatial location feature encoding function, For timestamps, It is a feature encoding function for time sequence.

7. A system for flow estimation in long-time-series point cloud scenes, characterized in that, The system is used to implement the method as described in any one of claims 1-6, the system comprising: The sliding window segmentation module is used to divide the input long temporal point cloud sequence into multiple overlapping sliding windows; The initial trajectory generation module is used to perform neighbor interpolation on the adjacent high-dimensional point cloud features of a given query point in each time frame based on the point cloud data of the global initial frame, and generate the initial trajectory of the query point over the entire long time series. The feature fusion module is used to calculate the geometric matching relationship between the query point and the local point cloud at both the point-level and voxel-level scales, centered on the current trajectory estimation position, in each iteration of each sliding window. It also fuses the temporal sequence information of the query point, the motion change information relative to the global initial position, and the current spatial position information to construct spatiotemporal fusion features. The optimization module is used to perform multiple rounds of iterative optimization on the trajectory state of the query point based on the spatiotemporal fusion features within each sliding window, wherein each round of optimization simultaneously constrains the intra-frame spatial geometric consistency and cross-frame temporal motion smoothness. The state transfer module is used to transfer the optimized trajectory state of the current sliding window to the next sliding window as the initial guide for trajectory estimation of the next sliding window; The trajectory output module is used to stitch together the optimized trajectory segments output by each sliding window to generate the final motion trajectory of the query point in the entire long-term point cloud sequence.

Images