Computer-implemented method for estimating a position and / or pose of a vehicle

The method employs a neural network module with graph neural networks and kernel signatures to enhance point cloud registration, addressing the robustness and accuracy issues in vehicle pose estimation by using heat diffusion-based geometric descriptors, achieving superior performance in transformation prediction and environmental adaptability.

WO2024231091A1PCT designated stage expired Publication Date: 2024-11-14CONTINENTAL AUTOMOTIVE TECHNOLOGIES GMBH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/EP2024/061019
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-04-23
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Current methods for estimating the position and pose of a vehicle using point cloud registration are not robust enough, particularly in handling environmental changes and illumination variations, and lack effective geometric descriptors for accurate transformation prediction.

Method used

A computer-implemented method utilizing a neural network module comprising graph neural networks and a kernel signature derived from the Laplace-Beltrami operator, which generates global and self-attention representations to predict transformations and enhance feature extraction robustness, incorporating a heat diffusion process for geometric feature description and perturbation resistance.

Benefits of technology

The method provides a robust and accurate estimation of vehicle position and pose by leveraging isometry-invariant kernel signatures and self-attention mechanisms, outperforming existing methods in terms of root mean square error and mean absolute error, especially in noisy and varying environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure EP2024061019_14112024_PF_FP_ABST
    Figure EP2024061019_14112024_PF_FP_ABST
Patent Text Reader

Abstract

The invention provides a computer-implemented method for estimating a position and / or pose of a vehicle based on sensor data (10) of a sensor of the vehicle and map data (12) of an environment of the vehicle, the sensor data (10) and the map data (12) each including a time-dependent, multi-dimensional frame (14, 16), the frames (14, 16) including a point cloud X = (x 1 , …, x N ) or Y = (y 1 , …,y N ), respectively, of N multi-dimensional points at a time t 0 , wherein the point clouds (X,Y) are configured such that one point cloud (X,Y) can be matched at least with respect to a number of K ≤ N corresponding points with the other point cloud (Y, X) by applying a transformation (TR) to any one of the frames (14, 16), the transformation (TR) including a rotation and / or a translation of said frame (14, 16).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] 202301539 SG SNG 1 DESCRIPTION Computer-implemented method for estimating a position and / or pose of a vehicle TECHNICAL FIELD The invention relates to a computer-implemented method for estimating a position and / or pose of a vehicle. The invention further relates to data processing device, a computer program, and a computer-readable data carrier. BACKGROUND Towards the era of intelligent and smart perception, 3D computer vision techniques are incorporated in many fields including autonomous driving, robotics, scene modeling or reconstruction and so on. As a task of 3D computer vision, point cloud registration has been widely used in numerous applications such as object detection, odometry estimation and Simultaneous Localization And Mapping (SLAM), due to its anti-interference to the changes of environmental season and illumination. The goal of this task is to estimate the relative transformation or pose between two given 3D point cloud frames. Besides, point cloud registration is used to align the point clouds in different coordinate systems into the same coordinate system, which achieves the pose estimation. SUMMARY OF THE INVENTION The object of the invention is to provide an improved method for point cloud registration that can be used for estimating a position and / or pose of a vehicle. The object is achieved by the subject-matter of the independent claims. Advantageous embodiments of the invention are subject-matter of the dependent claims. In one aspect, the invention provides a computer-implemented method for estimating a position and / or pose of a vehicle based on sensor data of a sensor of 202301539 SG SNG 2 the vehicle and map data of an environment of the vehicle, the sensor data and the map data each including a time-dependent, multi-dimensional frame, the frames including a point cloud ^ =(^^, … , ^^)or ^ =(^^, … , ^^), respectively, of ^ multi-dimensional points at a time ^^, wherein the point clouds are configured such that one point cloud can be matched at least with respect to a number of ^ ≤ ^ corresponding points with the other point cloud by applying a transformation to any one of the frames, the transformation including a rotation and / or a translation of said frame, the method comprising: a) For each point cloud ^ ∈{^, ^}with ^^(^^)= ^: a1) Determining a global feature representation ^^at a time ^, ^ > ^^, by feeding a neural network module ^^^^, the neural network module ^^^^including at least a first graph neural network ^^^^ and a second gra ^ ^ ph neural network ^^^^, with an input based on the point cloud ^^(^)at time ^; a2) Determining a global signature representation ^^at time ^, ^ > ^^, by feeding the neural network module ^^^^with an input based on a kernel signature of the point cloud ^^(^)at time ^, the kernel signature being derived from a kernel function ℎ depending on coordinates of a point ^ ∈ ^ and the time ^; and a3) Determining a self-attention representation ^ ^ ^_^^^based on the global feature representation ^^and the signature representation ^^; b) Predicting the transformation based on the determined self-attention representations ^ ^ ^ ^_^^^and ^^_^^^; and c) Estimating the position and / or pose of the vehicle based on the predicted transformation. An advantage of the invention may be that a kernel signature is presented as an intrinsic feature. The kernel feature is a robust local geometric descriptor containing large-scale information. Furthermore, the kernel signature is isometry- invariant, that is, there exists an equivalence relationship between two isometric shapes. Meanwhile, if kernel signatures for two shapes are equal, these two shapes are almost isometric. In this regard, the kernel signature imposes geometric constraints for the point description, providing robustness in some degree. The neural network ^^^^can be used as a filter for the kernel signatures. The global signature representation is introduced into an attention 202301539 SG SNG 3 mechanism as additive weights. By introducing extra information from the kernel signature, the robustness of the point cloud representation is enhanced in the self- attention. Thus, the robustness of the feature extraction is refined. Preferably, the kernel function ℎ depends on eigenvalues ^^(^ = 1, … , and corresponding eigenfunctions ^^(^ = 1, … , ^) of the Laplace-Beltrami operator, with ^ being a predefined parameter. The kernel function ℎ may be based on the heat diffusion process which represent the graph features more concisely and stably. Preferably, the kernel function ℎ is defined as: When the kernel function ℎ is based on the heat diffusion process, an evolution in time may be introduced as given above. From the perspective of the physical meaning, this feature descriptor represents a point with the temperature evolution, in which a heat source is placed and removed immediately, and the temperature diffuses to the neighborhood. This evolution is based on the temperature diffusion speed, which essentially depends on the geometry of the objects projected by the point clouds. According to the property of the kernel function ℎ, it may be used to robustly describe the repeatable features for point clouds. This feature also performs more anti-interference with respect to local perturbations. Preferably, step a1) comprises: a4) Generating the point cloud ^^(^)at time ^, ^ > ^^, by feeding a pre-processing graph neural network ^^^^ ^ with an input based on the point cloud ^^(^^)at time ^^. In order to preprocess the 3D point clouds with potential outliers, a graph neural PDE (= partial differential equation) module can be used first as a learning-based filter. 202301539 SG SNG 4 Preferably, the method further comprises one or both of the following: d1) Constructing a plurality of graphs for one or more of the graph neural networks based on the point cloud ^^(^^)and / or the kernel signature at time ^^, the point cloud ^^(^)and / or the kernel signature at time ^, ^ > ^^, and / or the input for the one or more graph neural networks; and / or d2) Constructing the plurality of graphs for one or more of the graph neural networks based on a neighborhood graph. The neighborhood graph for each point is an undirected complete graph. Then, graph neural PDE layers are used to deal with the neighborhood graph of each point, which can robustly represent the high dimensional feature of the point. Preferably, step a1) comprises: a5) Determining a local feature representation ^ = ^_^^^^^^^(^^)^ at a time ^^, ^^≥ ^^, by feeding an input based on the point cloud time ^^into the first graph neural network ^^^^ ^ of the neural network module ^^^^; and a6) Determining an update ^^(^^)= the local feature representation ^^(^^)at a time ^^, ^^> by feeding an input based on the local feature representation ^^(^^) at time into the second graph neural network ^^^^ ^ of the neural network module ^^^^. The relationship between ^^(^^) at a time ^^, ^^> and ^^(^^) at time ≥ ^^, is an ordinary and / or partial differential equation. In order to deal with the graph- structured data, graph neural PDEs can be designed based on continuous flows. Compared with the conventional graph neural networks, including the Graph ATtention network (GAT) or the Graph Convolutional Network (GCN), the graph neural PDEs have superior performance on some applications such as the node classification for graph-structured data. To approximately solve the graph neural PDE, solvers of neural ODEs can be exploited. As for the robustness for feature representation, the neural ODEs / PDEs have advantages in defending perturbations and even attacks, compared with other deep neural networks without neural diffusion. 202301539 SG SNG 5 Preferably, the method further comprises: Calculating a multi-feature aggregation by feeding an input based on the update ^^(^^)of the local feature representation at time ^^into at least one further graph neural network and concatenating the update ^^(^^)of the local feature representation at time ^^with an output of the at least one further graph neural networks. Calculating a multi-feature aggregation by feeding an input based on the update ^^(^^)of the local feature representation at time ^^into at least one further graph neural network results in better and more robust performance. The number of the further graph neural network may be arbitrary, but computational cost increases with more and more further graph neural networks. In practice, calculating the multi-feature aggregation by feeding an input based on the update ^^(^^)of the local feature representation at time ^^into three further graph neural networks may result in optimal performance with respect to robust results and computation cost. Preferably, step b) comprises: b1) Calculating a joint representation ^ ^ ^^_^^^for each ^ ∈ {^, ^}, the joint representation ^ ^ being based on th ^ ^^_^^^e self-attention representation ^^_^^^and a cross-attention representation depending on the global feature representations ^^and ^^. Based on the previous high dimensional feature, a self-cross or joint attention mechanism is introduced to reinforce the static structure information in each point cloud and the interactive corresponding information between a pair of point clouds, respectively. Thus, an embedding is obtained involving the point-level information interaction within a point cloud frame and between two point cloud frames. Preferably, the method further comprises: e1) Calculating an attention weight matrix ^^^^based on the self-attention representations ^ ^ ^ ^_^^^and ^^_^^^; and e2) Predicting the transformation based on elements of the attention weight matrix ^^^^. 202301539 SG SNG 6 The relationship between the point clouds ^ and ^ can be expressed in a compact format as matrix equation which can be treated computationally efficient. In this regard, the transformed point cloud ^^corresponding to ^ is given as: ^^= Preferably, step e2) comprises: e3) Selecting for each row ^ ∈{1, … , ^}of the attention weight matrix ^^^^an having: e4) Selecting ^ corresponding point pairs{(^^, ^^^): ^ = 1, … , with the ^ greatest elements of the attention matrix ^^^^selected in step e3), where ^ is a predefined parameter; and e5) Predicting the transformation based on the selected ^ corresponding point pairs. Here, for each point in the point cloud ^, a corresponding point in the point cloud ^ is selected, which has the highest similarity with the point in the point cloud ^. Furthermore, based on the Top-^ similarity scores, the corresponding point pairs are selected. Thus, the transformation is predicted based on a best similarity match of ^ corresponding point pairs enabling to calculate the predicted transformation in an optimal way. Preferably, step e5) comprises: e6) Predicting the transformation by minimizing a mean squared error (MSE) defined as: where‖∙‖^denotes the ℒ^norm, ^ and ^ being an estimated translation and rotation with respect to a ground-truth rotation ^ and ^, respectively. 202301539 SG SNG 7 The solution of the optimization problem ^ represents the ^,^ predicted transformation. Thus, the predicted transformation represents a solution of a solvable optimization problem with respect to ^^^^^^, ^^. Preferably, step e6) comprises: e7) Minimizing the mean squared error (MSE) by using a singular value decomposition with trainable weights. The singular value decomposition with learnable weights is an efficient method to solve the optimization problem. Preferably, the method further comprises: f) Training the graph neural networks with a sequence ^^^^(^), ^ ∈ [^^^^^^; ^^^^], of two training point clouds ^^^∈{^^^, ^^^}, with ^^^^^^being a start time and ^^^^being an end time. In another aspect, the invention provides a data processing device comprising means for carrying out the method of any of the preceding claims. In another aspect, the invention provides a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of the preceding embodiments. In another aspect, the invention provides a computer-readable data carrier having stored thereon the computer program. BRIEF DESCRIPTION OF THE DRAWINGS Embodiments of the invention are now explained in more detail with reference to the accompanying drawings of which Fig.1 shows a framework of an embodiment of a computer-implemented method for estimating a position and / or pose of a vehicle; and 202301539 SG SNG 8 Fig.2 shows a framework of an embodiment of a step S11 of the method. DETAILED DESCRIPTION OF EMBODIMENT Fig.1 shows a framework of an embodiment of a computer-implemented method for estimating a position and / or pose of a vehicle. The method is based on sensor data 10 of a sensor of the vehicle and map data 12 of an environment of the vehicle. The vehicle may be for example an autonomous vehicle, such as a car or a robot. The sensor of the vehicle may be any kind of sensor configured for perceiving the environment of the vehicle. For example, the sensor may be a LIDAR or RADAR device. The sensor data 10 and the map data 12 include each a time-dependent, multi-dimensional frame 14, 16. For instance, the frames 14, 16 may be two-dimensional or three-dimensional. When the vehicle is moving, i.e., changing its position and / or pose, the frame 14 of the sensor data 10 is evolving in time. In contrast, the map data 12 is predetermined. The position of the vehicle with respect to the map data 12 is determined and updated, for example, by means of a global positioning device of the vehicle. Thus, the frame 16 of the map data 12 is also evolving in time, as well. Since the global positioning device, however, has some intrinsic inaccuracy, e.g., the global positioning device cannot determine the pose of the vehicle, the frame 14 of the sensor data 10 and the frame 16 of the map data 12 are typically not aligned, but overlap. In other words, the frame 14 of the sensor data 10 includes a point cloud ^ =(^^, … , ^^)of ^ multi-dimensional points at a time ^^. Similarly, the frame 16 of the map data 12 includes a point cloud ^ =(^^, … , ^^)of ^ multi-dimensional points at time ^^. Since the frames 14, 16 are not aligned, but overlap, the points clouds ^, ^ are configured such that one point cloud ^, ^ can be matched at least with respect to a number of ^ ≤ ^ corresponding points with the other point cloud ^, ^ by applying a transformation TR to any one of the frames 14, 16. The transformation TR includes a rotation and / or a translation of said frame. 202301539 SG SNG 9 In this context, the term “match” is understood such that the transformation TR does not need to transform the ^ ≤ ^ corresponding points exactly on top of each other, but such that the transformation TR is a within an acceptable error range a reasonable assumption to a ground-truth transformation. An object of the method is to find said transformation TR. Thereby, neural networks are used. The used neural networks may be trained with one or more sequences of point clouds evolving in time, wherein one of the point clouds is captured by means of the sensor of the vehicle. According to an embodiment of the method, for each point cloud ^ ∈{^, ^}with ^^(^^)= ^, the following steps are performed: In a step S11, a global feature representation ^^at a time ^, ^ > ^^, is determined. This is done by feeding a neural network module ^^^^with an input based on the point cloud ^^(^)at time ^. The output of the neural network module ^^^^is fed into an FC layer 18. The output of the FC layer 18 is the global feature representation ^^having a dimension ^ × 4^, wherein ^ is the number of features. In a step S12, a global signature representation ^^at time ^, ^ > ^^, is determined. This is done by feeding the neural network model ^^^^with an input based on a kernel signature 20 of the point cloud ^^(^)at time ^. The kernel signature 20 is derived from a kernel function ℎ that depends on coordinates of a point ^ ∈ ^ and the time ^. In order to obtain the kernel signature 20, the coordinates of the ^ points of the point cloud ^^(^) and the time ^ are input into the kernel function ℎ. The output of the neural network module ^^^^is fed into the FC layer 18. The output of the FC layer 18 is the global signature representation ^^having a dimension ^ × 4^. 202301539 SG SNG 10 The kernel signature 20 may be derived from a kernel function ℎ based on eigenvalues ^^(^ = 1, … , ^) and corresponding eigenfunctions ^^(^ = 1, … , ^) of the Laplace-Beltrami operator. For example, the kernel function ℎ may be defined as: with ^ being a predefined parameter. Thus, the kernel signature 20 can be regarded as a heat kernel signature and the global signature representation ^^can be regarded as a global heat signature representation. In a step S13, a self-attention representation ^ ^ ^_^^^is determined based on the global feature representation ^^and the signature representation ^^. The self- attention representation ^ ^ ^_^^^can be defined as: where(∙)^denotes the transpose operation and || denotes the concatenation operation. ^ is the number o ^^ ^^ ^^^^f multi-heads for the attention. ^^, ^^, ^^^ ^ ^ , ^ ^ ^ , ^^^ ^ , and ^^are learnable layers, where ^ ^^^^^^^^^^ ^ , ^^^ , ^^^ ∈ ℝ^×^^ ^ , and ^(∙)denotes the row-wise softmax function. Furthermore, in the step S13, a joint representation ^ ^ ^^_^^^for each ^ ∈{^, ^}, is calculated. The joint representation ^ ^ ^^_^^^is based on the self-attention representation ^ ^ ^_^^^and a cross-attention representation ^ ^ ^_^^^depending on the global feature representations ^^and ^^. In a step S14, the joint representation ^ ^ ^^_^^^and the global feature representation ^^are fed into a feed-forward network. The output of the feed-forward network is denoted as ^ ^ ^ ^^_^^and ^^^_^^, respectively, in Fig.1. 202301539 SG SNG 11 In a step S15, an attention weight matrix ^^^^is calculated based on the self- attention representations ^ ^ ^_^^^and ^ ^ ^_^^^. In detail, the attention weight matrix ^^^^is calculated based on ^ ^ ^^_^^, ^ ^ ^^_^^, ^^, and ^^. For instance, the attention weight matrix is defined as: where ^^^^= 4^ and ^^^^^^^^(∙) denotes the row-wise softmax function. In a step S16, ^ corresponding point pairs {(^^, ^^^): ^ = 1, … , are selected based on elements of the attention weight matrix ^^^^, wherein ^ is a predefined parameter. In detail, in a step S16.1, for each row ^ ∈{1, … , ^}of the attention weight matrix ^^^^an element ^ ^^^ ^,^∗ ^ (^, ^ ∈{1, … , ^}) having: is selected. Thereafter, in a step S16.2, the ^ corresponding point pairs{(^^, ^^^): ^ = 1, … , ^} with the ^ greatest elements of the attention matrix ^^^^selected in step S16.1 are selected. In a step S17, the transformation TR is predicted by minimizing a mean squared error (MSE) defined as: where ‖∙‖^denotes the ℒ^norm. The minimizing may be performed by using a singular value decomposition with trainable weights. ^ and ^ are an estimated translation and rotation estimated translation and rotation with respect to a ground- truth rotation ^ and ^, respectively. 202301539 SG SNG 12 The solution of the optimization problem ^ represents the ^,^ predicted transformation TR. Thus, the predicted transformation TR represents an optimal transformation having minimum mean squared error ^^^^^^, ^^. In a step S18, the position and / or the pose of the vehicle is estimated based on the predicted transformation TR. At this point, the frame 14 of the sensor data 10 and the frame 16 of the map data 12 can be aligned by applying the transformation TR to any one of the frames 14, 16. This means, the position and / or pose of the vehicle with respect to the map data 12 can be re-updated. In other words, the intrinsic inaccurary of the global positioning device can be compensated. Fig.2 shows a framework of an embodiment of the step S11. The framework includes the neural network module ^^^^. The neural network module ^^^^at least includes a first graph neural network ^^^^ ^ and a second graph neural network ^^^^ ^ . The neural network module ^^^^preferably also includes further graph neural networks ^^^^ ^ ^ ^^, ^^^^^, ^^^^^as will be discussed in the following: In a step S21, the point cloud ^^(^) at time ^, ^ > ^^, is generated by feeding a pre- processing graph neural network ^^^^ ^ with an input based on the point cloud ^^(^^)at time ^^. The point cloud ^^(^)has a dimension of ^ × ^, wherein ^ is the dimension of the frames 14, 16. The pre-processing graph neural network neural ^^^^ ^ and network module ^^^^together may be considered as a graph neural PDE network 22 (PDE = partial differential equation). In a step S22, the first graph neural network ^^^^ ^ is fed with an input based on the point cloud at a t ^ ime ≥ ^^. The first graph neural network ^^^^thus determines a local feature representation ^^(^^) at time 202301539 SG SNG 13 ^^. The local feature representation ^^(^^)has a dimension of ^ × ^, wherein ^ is the number of features and a predefined parameter. In a step S23, the second graph neural network ^^^^ ^ is fed with an input based on the local feature representation ^^(^^)at time . The second graph neural network ^^^^ ^ thus determines an update ^^(^^) = ^_^^^^ ^ ^^^(^^)^ of the local feature representation ^^(^^) at a time ^^, ^^> ^^. The update ^^(^^) has a dimension of ^ × ^. The embodiment of the neural network ^^^^shown in Fig.2 also includes a first further graph neural network ^^^^ ^^, a second further graph neural network ^^^^ , and a thir ^ ^^d further graph neural network ^^^^^. In a step S24, the first further graph neural network ^^^^ ^^, the second further graph neural network ^^^^ , and the third further graph n ^ ^^eural network ^^^^^are each fed with an input based on the update ^^(^^)of the local feature representation at time ^^. The graph neural networks shown in Fig.2 each use a plurality of graphs. The plurality of graphs are constructing based on the point cloud ^^(^^)at time ^^, the point cloud ^^(^) at time ^, ^ > ^^, and / or the input for the respective graph neural networks. Here, the plurality of graphs are neighborhood graphs that are constructed by means of the K-Nearest Neighbor (KNN) method within a rewiring 20. In detail, the graphs used for the pre-processing graph neural network ^^^^ ^ are constructed by means of the KNN method based on the points of the point cloud ^. The graphs used for the first graph neural network ^^^^ ^ are constructed by means of the KNN method based on the points of the point cloud time ^^. The graphs used for the second graph neural network ^^^^ ^ are constructed by means of the KNN method based on the points of the local feature representation ^ ^ at time ^ . The graphs ^ ^(^)^used for the first further graph neural network ^^^^^ 202301539 SG SNG 14 are constructed by means of the KNN method based on the points of the update ^^(^^) of the local feature representation ^^(^^) at time ^^. The graphs used for the second further graph neural network ^^^^ ^^are constructed by means of the KNN method based on the points of the point cloud ^^(^^)at time ^^, while the graphs used for the third further graph neural network ^^^^ ^^are constructed by means of the KNN method based on the points of the point cloud ^. In a step S25, the update ^^(^^)of the local feature representation at time ^^is concatenated with the outputs of the further graph neural networks ^^^^ ^ ^^, ^^^^^, ^^^^ ^^building the output of the neural network module ^^^^. The output of the neural network module ^^^^is fed into the FC layer 18 as described with respect to step S11. The framework of the embodiment of step S11, i.e., the graph neural PDE module 22 may be in a similar way applied to the kernel signature 20 in order to obtain the global signature representation ^^according to step S12. In this context, the plurality of graphs are constructing based on kernel signature 20 at time ^^, the kernel signature 20 at time ^, ^ > ^^, and / or the input for the respective graph neural networks, preferably by means of the KNN method. In an experiment, the method has been applied to the KITTI dataset that contains LIDAR point clouds captured within an urban environment by means of a laser scanner in three dimensions. The dataset contains several sequences of frames, each frame containing a plurality of points. One of the frames can be used as training, another as testing dataset. In this setup, ^ can be set to 128 while ^ = 20. The graph neural networks can be standard graph neural networks. To evaluate the method, statistics including the root mean square error (RMSE) and the mean absolute error (MAE) for the predicted transformation TR have been used. Compared to other methods, such as ICP, DCP, LDP-Pose, and VCR-Net- Pose, the herein described method yields smaller RMSE and MAE. Thus, the described method can predict the transformation TR and estimate the position and / or pose of the vehicle more accurately and reliably as compared to other methods. 202301539 SG SNG 15 The invention also provides a data processing device (not shown) comprising means for carrying out the herein described method. The invention further provides a computer program (not shown) comprising instructions which, when the program is executed by a computer, cause the computer to carry out the herein described method. The invention further provides a computer-readable data carrier (not shown) having stored thereon the computer program. Preferred embodiments of the invention can be summarized as follows: 1) Within a 3D point cloud frame, the neighborhood graph of each point consisting of its K nearest neighbors is constructed, in which the points are regarded as the vertices. The ℒ^distance with respect to the features of points is used for the neighbor acquisition, where the initial features of points are the 3D coordinates. The neighborhood graph for each point is an undirected complete graph. Then, graph neural PDE layers are used to deal with the neighborhood graph of each point, which can robustly represent the high-dimensional feature of the point. 2) Based on the robust feature representations, a self-cross attention module is applied to obtain the embedding involved with point-level information interaction within a point cloud frame and between two point cloud frames. Besides, as a robust feature descriptor for the point cloud, the heat kernel signature is introduced into the attention module as the weights. 3) Using the above embeddings, an attention module is established, from which the weighted points are obtained for one frame and regarded to correspond to the points in the other frame. 4) Through the correspondence among points in the two point cloud frames, the optimal transformation (including the translation and the rotation) can be estimated using the optimization solution methods like the weighted SVD. The model consists of GNN modules and graph neural PDE modules with different rewiring, which is shown in Fig.1 and 2. 202301539 SG SNG 16 In order to preprocess the 3D point clouds ^, ^ with potential outliers, we first use a graph neural PDE module as a learning-based filter. As for a 3D point cloud denoted by ^ ∈ ℝ^×^, the three-dimensional coordinates are regarded as its feature map, where ^ is the number of points in each 3D point cloud. The neighborhood graphs for the points are constructed by means of the K-Nearest Neighbor (KNN) method. Furthermore, a graph neural PDE module is exploited for the feature updating. High dimensional features for the preprocessed 3D points output from the neural PDE module, are extracted based on a GNN module, such as the GAT, GCN, and DGCNN. In this regard, three-dimensional coordinates of points are extended into ^-dimensional features. In particular, the K-Nearest Neighbor (KNN) method is used to construct a graph for each point, for which the GNN module is used to achieve a feature representation. Based on the previous high dimensional feature, a self-cross attention mechanism is introduced to reinforce the static structure information in each point cloud and the interactive corresponding information between a pair of point clouds, respectively. Besides, to emphasize the robustness of the feature extraction, a heat kernel signature is also introduced into the attention mechanism as additive weights. The self-cross attention or joint representation with respect to the point cloud ^ can be defined as: with ^^_^^^being a self-attention representation and ^^^_^^^(^^, ^^)being a cross-attention representation with respect to the point cloud ^ . The self-cross attention or joint representation with respect to the point cloud ^ is defined correspondingly. 202301539 SG SNG 17 The self-attention representation ^^_^^^(^^)with respect to the point cloud ^ may be defined as described hereinbefore. The cross-attention representation with respect to the point cloud ^ may be defined as: where ^ ^^ , ^^^ , ^^^ , and ^^are learnable l ^ ^^ ^ayers and ^^is the feature dimension for the ^-th attention head. To refine the robustness, the heat kernel signature is introduced into the self- attention module. The heat kernel signature is computed based on the 3D point cloud. According to the property of this feature, it is reasonable to use it to robustly describe the repeatable features for point clouds. This feature also performs more anti-interference with respect to local perturbations. The heat kernel signatures are processed with the graph neural PDE module and the FC layer to obtain the embedding. Similar to the point cloud, the graph neural PDE is used as a filter for the heat kernel signatures. The graph construction in the graph neural PDE is based on the KNN for the heat kernel signatures. The method described hereinbefore can be applied to LIDAR odometry. The method may be used for the local pose estimation based on the adjacent LIDAR frames, which can be regarded as a kind of learning-based LIDAR odometry. Then, the relative poses can be accumulated to achieve the global pose prediction. The accumulated error may eventually harm the predicted global poses. To mitigate this issue, renewing the starting points with accurate global poses is a possible solution. Extensive experiments have demonstrated that compared with other methods, the preferred embodiments of the invention have advantages not only in the case of raw point clouds, but also in the case of additive noise perturbations. Preferred 202301539 SG SNG 18 embodiments of the invention indicate the neural network model benefits the point cloud registration task.

[0002] 202301539 SG SNG 19 REFERENCE SIGNS 10 sensor data 12 map data 14 frame 16 frame 18 FC layer 20 kernel signature 22 graph neural PDE module ^^global feature representation ^^global signature representation ℎ kernel function TR transformation ^^^^neural network model ^ ^ ^_^^^self-attention representation ^ ^ ^^_^^^joint representation ^^^^attention weight matrix ^^^^^^, ^ ^ mean squared error (MSE) ^ estimated rotation ^ estimated translation ^^(^)point cloud at time ^ ^^(^) local feature representation at time ^ / update of the local feature representation ^^^^ ^ first graph neural network ^^^^ ^ second graph neural network ^^^^ ^^first further graph neural network ^^^^ ^^second further graph neural network ^^^^ ^^third further graph neural network

Claims

202301539 SG SNG 20 CLAIMS 1. A computer-implemented method for estimating a position and / or pose of a vehicle based on sensor data (10) of a sensor of the vehicle and map data (12) of an environment of the vehicle, the sensor data (10) and the map data (12) each including a time-dependent, multi-dimensional frame (14, 16), the frames (14, 16) including a point cloud ^ =(^^, … , ^^)or ^ =(^^, … , ^^), respectively, of ^ multi- dimensional points at a time ^^, wherein the point clouds (^, ^) are configured such that one point cloud (^, ^) can be matched at least with respect to a number of ^ ≤ ^ corresponding points with the other point cloud (^, ^) by applying a transformation (TR) to any one of the frames (14, 16), the transformation (TR) including a rotation and / or a translation of said frame (14, 16), the method comprising: a) For each point clouda1) Determining a global feature representation ^^at a time ^, ^ > ^^, by feeding a neural network module ^^^^, the neural network module ^^^^including at least a first graph neural network ^^^^ ^ ^ and a second graph neural network ^^^^, with an input based on the point cloud ^^(^)at time ^; a2) Determining a global signature representation ^^at time ^, ^ > ^^, by feeding the neural network module ^^^^with an input based on a kernel signature (20) of the point cloud ^^(^) at time ^, the kernel signature (20) being derived from a kernel function ℎ depending on coordinates of a point ^ ∈ ^ and the time ^; and a3) Determining a self-attention representation ^ ^ ^_^^^based on the global feature representation ^^and the signature representation ^^; b) Predicting the transformation (TR) based on the determined self-attention representations ^ ^ ^ ^_^^^and ^^_^^^; and c) Estimating the position and / or pose of the vehicle based on the predicted transformation (TR).

2. The method according claim 1, c h a r a c t e r i z e d i n t h a t the kernel function ℎ depends on eigenvalues ^^(^ = 1, and corresponding eigenfunctions ^^(^ = 1, … , ^) of the Laplace-Beltrami operator, with ^ being a predefined parameter.202301539 SG SNG 21 3. The method according to claim 2, c h a r a c t e r i z e d i n t h a t the kernel function ℎ is defined as: exp(−^^^)^^^(^) .

4. The method any of the preceding claims, c h a r a c t e r i z e d i n t h a t step a1) comprises: a4) Generating the point cloud ^^(^)at time ^, ^ > ^^, by feeding a pre-processing graph neural network ^^^^ ^ with an input based on the point cloud ^^(^^)at time ^^.

5. The method according to any of the preceding claims, further comprising one or both of the following: d1) Constructing a plurality of graphs for one or more of the graph neural networks (^^^^ ^ ^ ^ ^ ^ , ^^^^, ^^^^^, ^^^^^, ^^^^^) based on the point cloud ^^(^^)and / or the kernel signature (20) at time ^^, the point cloud ^^(^)and / or the kernel signature (20) at time ^, ^ > ^^, and / or the input for the one or more graph neural networks (^^^^ ^ ^ ^ ^ ^ , ^^^^, ^^^^^, ^^^^^, ^^^^^); and / or d2) Constructing the plurality of graphs for one or more of the graph neural networks (^^^^ , ^^^^ , ^^ ^ ^ ^ ^^^^^, ^^^^^, ^^^^^) based on a neighborhood graph.

6. The method according to any of the preceding claims, c h a r a c t e r i z e d i n t h a t step a1) comprises: a5) Determining a local feature representationat a ^ , ^^≥ ^^, by feeding an input based on the point cloud ^^(^^) at timeinto first graph neural network ^^^^ ^ of the neural network module ^^^^; and a6) Determining an update ^ (^^) =the local representation ^ (^^) at a time ^^, ^^>by feeding an input based on the feature representation ^^(^^)at time into the second graph neural network ^^^^ ^ of the neural network module ^^^^.202301539 SG SNG 22 7. The method according to claim 5, c h a r a c t e r i z e d b y calculating a multi- feature aggregation by feeding an input based on the update ^^(^^) of the local feature representation at time ^^into at least one further graph neural network (^^^^ ^^, ^^^^ ^^, ^^^^ ^^) and concatenating the update ^^(^^)of the local feature representation at time ^^with an output of the at least one further graph neural networks (^^^^ ^ ^ ^^, ^^^^^, ^^^^^).

8. The method according to any of the preceding claims, c h a r a c t e r i z e d i n t h a t step b) comprises: b1) Calculating a joint representation ^ ^ ^^_^^^for each ^ ∈{^, ^}, the joint representation ^ ^ being based on the s ^ ^^_^^^elf-attention representation ^^_^^^and a cross-attention representationdepending on the global representations ^^and ^^.

9. The method according to any of the preceding claims, further comprising: e1) Calculating an attention weight matrix ^^^^based on the self-attention representations ^ ^ ^ ^_^^^and ^^_^^^; and e2) Predicting the transformation (TR) based on elements of the attention weight matrix ^^^^.

10. The method according to claim 9, c h a r a c t e r i z e d i n t h a t step e2) comprises: e3) Selecting for each row ^ ∈ {1, … , ^} of the attention weight matrix ^^^^ane4) Selecting ^ corresponding point pairs{(^^, ^^^): ^ = 1, … , ^}with the ^ greatest elements of the attention matrix ^^^^selected in step e3), where ^ is a predefined parameter; and e5) Predicting the transformation (TR) based on the selected ^ corresponding point pairs.202301539 SG SNG 23 11. The method according to claim 10, c h a r a c t e r i z e d i n t h a t step e5) comprises: e6) Predicting the transformation (TR) by minimizing a mean squared error (MSE) defined as:where‖∙‖^denotes the ℒ^norm.

12. The method according to claim 11, c h a r a c t e r i z e d i n t h a t step e6) comprises: e7) Minimizing the mean squared error (MSE) by using a singular value decomposition with trainable weights.

13. A data processing device comprising means for carrying out the method of any of the preceding claims.

14. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of the claims 1 to 12.

15. A computer-readable data carrier having stored thereon the computer program of claim 14.