A computer-implemented method for camera pose regression in a challenging traffic environment

Abstract

A method of pose regression comprising: receiving images 12, generating a feature map 16 from the images, generating a graph 20 wherein the nodes 24 correspond to features of the feature map, generati

Description

TECHNICAL FIELD The invention relates to a computer-implemented method for camera pose regression, in particular for the purpose of localizing a vehicle such as an autonomous vehicle in a traffic environment. BACKGROUND Camera relocalization is a key issue for autonomous vehicles. Previous camera pose regression models usually achieve better results in rather ideal scenarios, e.g., where there is little environment disturbance. Localization is an important problem especially for autonomous vehicles, as localization may form the foundation for multiple succeeding tasks, including but not limited to navigation, environmental perception and / or map construction. A typical way to solve the problem is to treat it as an image retrieval task. Before inference, a database or a map that stores the prior information (e.g., images, descriptors, poses, and etc.) may be constructed. Then, when given query images, the image retrieval model can find top matched images by computing the similarity. NetVLAD is an example for a deep-learning-based model that does not intuitively compare the photometric similarity of images. Instead, it first uses a convolutional neural network (CNN) to extract features from the images, and then a similarity comparison is carried out between the features from the database images and the query images. Finally, the camera pose of the top matched image is treated as the output. However, since the image retrieval requires a database as a prerequisite, both the time and space complexity are in general proportional to the number of database priors, which is an inherent issue. In order to detach localization from the requirement of a database, research interests turn has recently turned towards camera pose regression. Camera pose regression allows to reduce or even remove the requirement for priors and thus shows stronger potential for automotive driving tasks. PoseNet, for example, uses a CNN to extract image features as vector embeddings. The embeddings can be directly regressed to poses having six degrees of freedom (6-DoF poses). In order to mitigate the weight unbalancing between translation and rotation, PoseNet proposed a learnable parameter to help the loss function converge. To further improve the regression performance and model robustness in challenging driving scenarios, multiview-based models were used that extend the input from a single image to an image sequence that includes multiple nearby frames. LSTM-Pose proposes to combine CNN and long short-term memory (LSTM) approaches to effectively capture spatial and temporal information. MapNet focuses on the post-processing stage where it incorporates the precomputed visual odometry to optimize the regressed trajectory. Local supports global (LsG) integrates graph neural networks (GNN) into CNN to make image nodes better interact with neighbours. CoordiNet enables the network to output regression uncertainty and uses the CoordConv technique to learn richer spatial representations. The above-mentioned multi-view-based models have shown promising performance in the challenging driving environment. Provided there is a database stored with images and the camera poses, when given a query image, the image retrieval models aim at finding top matches of the query image. And the poses of the matched images can be regarded as the pose of the query image. Usually, the database stores not only images but also the global and local descriptors of these images, which can be computed by either a CNN or a classic feature extractor. During inference, the descriptors of the query image are computed in the same way, and matching can be achieved by further computing the similarity between the query descriptor and database descriptors. Traditionally, the descriptors can be computed by SIFT, VLAD, or Fisher vector, all of which are based on human knowledge prior and are not strong enough to represent an image. In the last decade, deep learning has emerged and developed. The descriptors computed from a deep neural network are demonstrated to be more powerful than classic methods. NetVLAD is a pioneer to integrate CNN into image retrieval, where a trainable VLAD layer is proposed. Gradient episodic memory (GeM) proposed to formulate the histogram binning approximation to directly optimize the matching. Patch-NetVLAD extends the vanilla NetVLAD, which splits the whole image into discretized patches and the local features can be aggregated better. The above-mentioned methods all rely on a pre-constructed database to store candidate information, and the storage requirement is linearly proportional to the number of candidate priors with complexity in both the time and space dimension. To reduce the complexity, camera pose regression methods (CPRs) were proposed. Given the query images, CPRs directly regress the camera poses of these images without the need of a database. Thus, it does not depend on the scale of the database and the complexity is considerably smaller, e.g., 0(1). PoseNet proposes the simultaneous learning for location and orientation by integrating a balance parameter. AD PoseNet leverages the semantic information to dropout dynamic area in the image. AtLoc introduces the global attention to guide the network to learn better representation. And GNNMap-Net expand the feature exploring from single image to multiview images by leveraging Graph Neural Networks (GNNs). Coordinet uses the CoordConv and weighted average pooling to capture spatial relations. In control theory and physics, the dynamics of a system is usually described by differential equations which relates one or more unknown functions and their derivatives. An ODE is the equation that defines the derivative of a function y(t): while a PDE contains more than one variable: dy(t,s) SUMMARY OF THE INVENTION It is the object of the invention to improve camera pose regression, preferably in the field of autonomous driving and / or in challenging driving environments. The object is achieved by the subject-matter of the dependent claims. Preferred embodiments are subject-matter of the dependent claims. The invention provides a computer-implemented method for camera pose regression for the purpose of localizing a vehicle, preferably an autonomous vehicle, that moves in a traffic environment, the method comprising: a) with a feature extraction means: receiving a number N of images that are indicative of the traffic environment, wherein the feature extraction means determines a four-dimensional feature map from the images, wherein a first dimension is the number N, a second dimension is a predetermined height H in pixels, a third dimension is a predetermined width W in pixels and a fourth dimension corresponds to a predetermined number C of channels of the feature extraction means; b) with a graph structure generation means: generating a feature graph structure that includes a feature node set and a feature edge set, wherein the feature node set is generated as having a plurality of feature nodes and each feature node includes a pixel-wise feature based on the feature map obtained in step a), wherein the feature edge set is generated as fully connected between any two feature nodes; c) with a feature graph partial differential equation (PDE) layer: receiving the feature graph structure of step b) and determining an updated feature graph structure by solving a PDE based on a graph updating function; d) with a node ordinary differential equation (ODE) layer: receiving the updated feature graph structure of step c) and determining an updated node feature graph structure by solving an ODE based on a node feature updating function; e) with a data conversion means: converting the updated node feature graph of step d) into a plurality of decomposed feature maps; f) with a vector embedding means: for each decomposed feature map, determining a vector embedding by pooling; g) with a vector embedding graph structure generation means: generating a vector embedding structure that includes a vector embedding node set and a vector embedding edge set, wherein the vector embedding node set is generated as having a plurality of vector embedding nodes and each vector embedding node includes one vector embedding obtained in step f), wherein the vector embedding edge set is generated as fully connected between any two vector embedding nodes; h) with a vector embedding differential equation (DE) layer: receiving the vector embedding graph structure of step g) and determining an updated vector embedding graph structure by sequentially solving a PDE and an ODE based on at least one vector embedding updating function; i) with a pose decoder means: decoding the updated vector embedding graph structure of step h) into a set of camera pose coordinates that are indicative of a pose of a camera relative to the traffic environment. Given a number N of nearby images I = { / }, the disclosed camera pose regression pipeline has two different stages. In a first stage (Fig. 1), the network / extracts feature maps as m = / ( / ), which describe the image local patterns. A subsequent pooling module further aggregates the feature maps into vector embeddings h = Pooling(m), which describe the image global patterns. By applying a fully-connected (fc) layer, the camera translation / e IR3 and rotation qt e IR3 of the input images can be regressed from the vector embeddings. In order to deal with more challenging scenarios, a method that is designated RobustLoc is disclosed. The method aims to leverage both graph-wise partial differential equation (PDE) dynamics as well as node-wise ordinary differential equation (ODE) dynamics. Experiments show that the method can surpass known models and can achieve a more robust operation in various environment conditions for nearly all scenarios. The model enables a higher robustness against environmental disturbance (e.g., changing seasons, weather, illumination, and unstable objects.) and allows effectively leveraging neighboring information. ODEs enable better robustness against adversarial disturbances. The graph structure allows for an inherently effective neighborhood information interaction. The disclosed method also use a graph PDE model that not only allows exploring the relation between neighbors, but also enables regularizing feature dynamics. The disclosed method is tested on three challenging autonomous driving datasets and achieves significant improvements as will be discussed later. To meet the two insights mentioned before to achieve robust camera pose regression in a driving environment, the disclosure proposes to capture neighboring and robust representations. Preferably, in step a) the feature extraction means includes a convolutional neural network that determines the four-dimensional feature map based on the set of input images. The use of a CNN allows a more suitable and accurate feature extraction compared to other methods. Preferably, in step b) the graph structure generation means restructures the four-dimensional feature map into a one-dimensional feature map, wherein a first dimension is the number C of channels, wherein pixel-wise features are uniquely identified by a pixel-wise index i based on N, H and W. With the restructuring of the data, the graph structure is generated to be particularly easy to process. Preferably, in step c) the graph updating function is determined for each feature node using a multi-head attention fully-connected layer. The use of multi-head attention focusses the model on more important features. Preferably, in step d) the node feature updating function is determined using a plurality of fully-connected layers that are configured as a multi-layer perceptron (MLP). Preferably, in step e) the data conversion means generates each decomposed feature map as a three-dimensional decomposed feature map, wherein a first dimension corresponds to the height H, a second dimension corresponds to a width W and a third-dimension corresponds to the number C of channels, wherein each decomposed feature map is uniquely identified by an image index i based on the number N of images. Preferably, in step f) the vector embedding means embeds the decomposed feature maps by maxpooling or average pooling. Preferably, in step h) the updated vector embedding graph structure is determined by solving the PDE based on a first vector embedding updating function that is determined for each vector embedding node using a multi-head attention fully-connected layer. Preferably, in step h) the updated vector embedding graph structure is determined by solving the ODE based on a second vector embedding updating function that is determined using a plurality of fully-connected layers that are configured as a multilayer perceptron. Preferably, in step i) the pose decoder means is configured as a branch decoder. Preferably, in step i) the pose decoder means has a common first layer that receives the output of step h), and the common first layer is configured to decouple the vector embeddings into location related vector embeddings and rotation related vector embeddings. Preferably, in step i) the pose decoder means has a branched second layer that receives the output of step i2), and the branched second layer is configured to decode the location related vector embeddings and the rotation related vector embeddings into the camera pose coordinates. Preferably, in step i) the camera pose coordinates include three-dimensional camera location coordinates that describe the position of the camera that recorded the set of images and / or four-dimensional camera rotation coordinates that describe the orientation of the camera that recorded the set of images relative to the traffic environment, respectively. The invention provides a computer-implemented method for navigating a vehicle, preferably an autonomous vehicle, through a traffic environment, the method comprising: a) recording a set of images with an imaging sensor of the vehicle and performing a preferred method on the recorded images in order to obtain camera pose coordinates within the traffic environment; b) determining a current location of the vehicle based on the camera pose coordinates obtained in step a) and planning a path between the current location and a destination for the vehicle to follow; c) generating a control signal that informs a driver of the vehicle of the planned path of step b) or generating a control signal that causes the vehicle to follow the planned path. The invention provides a computer-implemented method for constructing a map of a traffic environment, the method comprising: a) driving a vehicle through the traffic environment and recording a set of images along a driving path; b) performing a preferred method on the set of images of step a) in order to obtain camera pose coordinates; c) generating a map model of the traffic environment based on the camera pose coordinates and storing the map model for use in a navigation task. The invention provides a data processing system comprising means for carrying out a previously described method. The invention provides a vehicle, preferably an autonomous vehicle, comprising the data processing system. The invention provides a computer program or computer-readable medium comprising instructions which, when executed by a data processing system, cause the data processing system to carry out a previously described method. BRIEF DESCRIPTION OF THE DRAWINGS Embodiments of the invention are described in more detail with reference to the accompanying schematic drawings. Fig. 1 and Fig. 2 depict an embodiment of a method for camera pose regression; Fig. 3 depicts experimental results; and Fig. 4 depicts experimental results. DETAILED DESCRIPTION OF EMBODIMENT Referring to Fig. 1 and 2, a camera pose regression pipeline 10 is depicted. The camera pose regression pipeline 10 is used to perform a method for camera pose regression that outputs camera pose coordinates. The camera pose regression 10 operates according to the method described below. The camera pose regression pipeline 10 is fed with a set of images 12, e.g., a number N = 3 images 12. Preferably, the number N of images 12 is odd, e.g. N = 3, 5, 7, 9, 11, ... Each image 12 is fed to a feature extraction means 14. The feature extraction means 14 is configured for extracting features from the images 12 and output a four-dimensional feature map 16. The feature extraction means 14 preferably includes a convolutional neural network that is trained for feature extraction. The four-dimensional feature map 16 has a first dimension according to the number of images N. A second and third dimension of the feature map 16 are the predetermined height H and width W measured in pixels, e.g. H = 4 and W = 4. The height H and width W are typically the same and preferably are chosen from H, W e {4, 5, 6, 7, 8}. Usually, the height H and width W of the feature map 16 are significantly smaller than the height and width of the images 12 that are processed. A fourth dimension of the feature map 16 corresponds to a predetermined number C of channels, e.g. C = 512. The number C of channels is preferably a power of 2, i.e. 2, 4, 8, 16, 32, 64, 128, 256, 512, ... The number C of channels is typically limited by the available computational resources, e.g. memory and vector processing power. The camera pose regression pipeline 10 further comprises a graph structure generation means 18. Based on the feature map 16, a feature graph structure 20 is generated. The feature map 16 that has a structure m e js reformatted by the graph structure generation means 18 into a one-dimensional feature map that has a structure m e in the one-dimensional feature map each pixel-wise feature in that feature map can be denoted as mt e ^.c,i NHW, where i is a feature index. The graph structure generation means 18 generates the feature graph structure 20 that can be described by the formula pfeat = (vfeat,£feat). The feature graph structure 20 includes a feature node set 22 having a plurality of feature nodes 24. The feature node set 22 can be described by the formula Vfeat = {mJ. Each feature node 24 contains a pixel-wise feature map mt. The pixel-wise feature maps mt collectively form the one-dimensional feature map. The edge set £feat is defined to be fully connected between any two feature nodes 24. The output of the graph structure generation means 18 is fed to a feature graph partial differential equation (PDE) layer 26. The idea behind the feature graph PDE layer is that PDE has demonstrated its robustness against both graph architecture and feature perturbation. The camera pose regression in the driving scenario is similar to graph node classification, where the environment variations shown on the image can be regarded as a perturbation to the graph node features. The feature graph PDE layer 26 models the feature node updating as a dynamic flow in two directions: from layer to layer and from feature node 24 to feature node 24. The feature nodes 24 and the layers are processed as variables such that the graph feature flow dynamics can be modeled as: where gfeat denotes the graph feature updating function that is described below. Then, the initial state at the start of the layer Z = 0 is taken to be the feature graph structure 20. The above PDE can be solved for an updated feature graph structure 28 at the end of the layer I = 1. The updated feature graph structure 28 is obtained by a PDE solver such as torchdiffeq that is available from https: / / github.com / rtqichen / torchdiffeq, which supports Pytorch implementation. This can be described as: m(i, 1) = ml = PDESolve(m°,^feat), where ml designates the updated feature graph structure 28 with updated feature nodes 30, PDESolve denotes the PDE solver, m- denotes the feature graph structure 20 and ^feat the feature updating function described below. The feature updating function allows effective node feature interaction. Preferably, the feature graph PDE layer 26 uses an attention-style mechanism to construct the feature updating function, preferably by first generating the message of each feature node 24 using multi-head fully-connected layers with learnable parameters Wfc and bfc at attention head k. The typical number k of attention heads is a power of 2, preferably 4 or 8: rfk = + bfc, aij,k = Softmax(m^ ■ m^), weighted \ ’ ft mi,k = / , ai,j,k^k, g{eat(mt) = m[out = Concatenate(m^fclghted)^ kcK           ' In other words, each updated feature node 30 results from its neighboring feature nodes 24 and itself, as indicated by the dashed arrows in Fig. 1. With this feature updating function the updated feature graph structure 28 is determined. The camera pose regression pipeline 10 includes a node ordinary differential equation (ODE) layer 32. The node ODE layer 32 receives the updated feature graph structure 28. With this node-wise feature dynamics can be described by a node ODE: d?7l(Z) z-featz —=f where / cat denotes the node feature updating function that is described below. Then, the initial state at the start of the layer Z = 0 is taken to be the updated feature graph structure 30. The above ODE can be solved for an updated node feature graph structure 34 at the end of the layer I = 1. The updated node feature graph structure 34 is obtained by an ODE solver. The node ODE layer 32 determines the basic node feature updating function using a multi-layer perceptron (MLP): mf°ut = MLP(m;) And the updated node feature 36 can be obtained as: m;(l) = ODESolve(mf, / feat). where the node feature flow is only applied within the node itself, without interaction from neighbors. As a result, the node ODE layer 32 determines and outputs an updated node feature graph structure 36. The camera pose regression pipeline 10 includes a data conversion means 38. The data conversion means 38 restructures the updated node feature graph structure 36 into decomposed feature maps 40. The decomposed feature maps 40 are preferably three-dimensional and have a first and second dimension that corresponds to the height H and width W. The decomposed feature maps 40 preferably have a third dimension that corresponds to the number C of channels. Each decomposed feature map 40 is uniquely identified by an image index i that is chosen from the number N of images 12. The decomposed feature maps 40 can be denoted as mt e g N. The camera pose regression pipeline 10 includes a vector embedding means 42. The decomposed feature maps 40 are fed into a pooling layer 44 to formulate vector embeddings ht e IRcas: h[ = Pooling(m;) where hi contains rich global features for the input image The pooling layer 44 may be an average pooling layer or a maxpooling layer. The vector embedding means 42 outputs vector embeddings 46. Referring to Fig. 2, the camera pose regression pipeline 10 includes a vector embedding graph structure generation means 48. The vector embedding graph structure generation means 48 are fed with the vector embeddings 46 and generate a vector embedding graph structure 50 that can be described by: pvect _ (.pvcct ^vect) where the vector embedding node set 52 is denoted as Vvect = [hi] and each vector embedding node 54 of the vector embedding node set 52 contains image vector embeddings hi. The vector embedding edge set 56 is denoted as £vect and is defined to be fully connected between any two vector embedding nodes 54. In the following the vector embedding graph structure 50 is fed to a vector embedding differential equation (DE) layer 57 that includes a vector embedding graph partial differential equation (PDE) layer 58 and a subsequent vector embedding graph ordinary differential equation (ODE) layer 60. The vector embedding graph PDE layer 58 and the vector embedding graph ODE layer 60 operate according to the same principles as the corresponding feature graph PDE layer 26 and the node ODE layer 32 with the proviso that they operate on vector embeddings 46 in the vector embedding graph structure 50. The vector embedding DE layer 57 outputs an updated vector embedding graph structure 62. The vector embedding graph structure 62 can be restructured into a final vector embedding 64 in a similar manner as the decomposed feature maps 40. The camera pose regression pipeline 10 includes a pose decoder means 66. The pose decoder means 66 is configured to decode the updated vector embedding graph structure 62 or the final vector embedding 64 in order to determine the camera pose coordinates. The camera pose regression pipeline 10 thus is able to predict multiple numbers simultaneously, i.e. translation {tx,ty,tz} and rotation quaternion {Qw’ Qp< Qq, Qr}- The final vector embedding 64 contains both the self and the neighbouring information. Since there is no node feature interaction anymore, the final vector embedding 64 can be denoted h instead of hi. The pose decoder means 66 is configured as a branch decoder. The pose decoder means 66 includes a common first layer 68. The common first layer 68 extracts the feature vectors for translation and rotation separately as: ht = Wth + bt, hq = Wqh + bq, where Wt Wq, bt, and bq are trainable parameters. Thus, the features of translation and rotation are decoupled. Next a branched second layer 70 preferably includes seven fully connected layers to perform the regression for seven targets separately as: td = Wtdht + btd, % = + ’ where td e {tx, ty, tz} and q& e {qw, q , qq, qr}, such that all seven targets can be learned. This approach allows a better convergence since each target has a different converge basin. The camera pose regression pipeline is trained using the weight balance loss: loss = ||t - tllie + p + \\q - q^e y + y where [3 and y are learnable parameters to balance the loss between translation and rotation. The method performed by the camera pose regression pipeline 10 was evaluated by experiments. The method was tested on two large autonomous driving datasets. The experiments were conducted on the Oxford RobotCar dataset (available at https: / / robotcar-dataset.robots.ox.ac.uk / datasets / ), and the 4Seasons dataset (available at https: / / www.4seasons-dataset.com / ). The Oxford RobotCar dataset is a large autonomous driving dataset which was collected by a car driving along a route in Oxford, UK. It consists of two different routes, the Loop with 8.8 x 104 m2 and the Full with 1.2 x 1 o6 m2 The 4Seasons dataset is even a more challenging autonomous driving dataset, which was collected in Munich, Germany, covering varying perceptual conditions. Specifically, it contains different scenes including the city, town, campus, and residential area. In addition, it consists of a wide variety of weather conditions and illuminations, including day and night. During experiment, the maximum sequence length was set as 11, which is compatible with previous multi-view models and allows comparison. The shorter side of the input images was resized to 128 pixels, and the batch size was 64. The Adam optimizer with learning rate 2e-4 and weight decay 5e-4 was used to train the camera pose regression pipeline 10. The data augmentation techniques include random cropping and color jittering (brightness, saturation, contrast, and hue). All of the experiments are conducted on an NVIDIA A5000. Referring to Fig. 3, the comparison between the invention and known regression models is shown for the Oxford RobotCar dataset. The numbers are given in meters and measure the deviation from the ground truth, i.e., the lower the number the more accurate the results. The column “median” refers to the meadian error, whereas the column “mean” refers to the mean error. The disclosed model achieves the best performance in ten out of a total of twelve metrics. Especially for the mean error, which is easily influenced by outlier predictions, the disclosed method outperforms the counterparts by a large margin. In the most challenging scene “Full,” the disclosed method is the first to achieve less than 10 m of mean error. Referring to Fig. 4, the comparison between the invention and known regressions models is shown for the 4Seasons dataset. The 4Seasons dataset is a more challenging dataset, where the disclosed method is shown to be even more powerful under environment disturbance. As seen in the table the disclosed model outperforms all counterparts in all metrics. Subsequently a method for navigating an autonomous vehicle through a traffic environment is described. The autonomous vehicle records a sequence of images and performs the previously described method using the camera pose regression pipeline 10 in order to obtain the camera pose coordinates, especially the camera location td e {tx,ty,tz}. The camera location is used as the vehicle’s location. A path planner - that is known per se - is used to plan a path from the current vehicle location to a destination. After the path was planned, a control signal is generated by the vehicle that informs the driver of the path to be taken. Alternatively or additionally, the control signal can be generated to cause the vehicle to follow the planned path. Subsequently a method for constructing a map of a traffic environment is described. An autonomous vehicle drives or is driven by a driver through the traffic environment. The vehicle keeps recording images along the driving path. The camera pose regression pipeline 10 is used to obtain the camera pose coordinates based on the recorded images. The camera pose coordinates within the traffic environment are collated and a graph of the driving path is generated therefrom. The graph of the driving path can be stored as a map for future use in other vehicles. REFERENCE SIGNS 10 camera pose regression pipeline 12 image 14 feature extraction means 16 four-dimensional feature map 18 graph structure generation means 20 feature graph structure 22 feature node set 24 feature nodes 26 feature graph partial differential equation (PDE) layer 28 updated feature graph structure 30 updated feature node 32 node ordinary differential equation (ODE) layer 34 updated node feature graph structure 36 updated node feature 38 data conversion means 40 decomposed feature maps 42 vector embedding means 44 pooling layer 46 vector embeddings 48 vector embedding graph structure generation means 50 vector embedding graph structure 52 vector embedding node set 54 vector embedding node 56 vector embedding edge set 57 vector embedding differential equation (DE) layer 58 vector embedding graph partial differential equation (PDE) layer 60 vector embedding graph ordinary differential equation (ODE) layer 62 updated vector embedding graph structure 64 final vector embedding 66 pose decoder means 68 common first layer 70 branched second layer

Claims

1. A computer-implemented method for camera pose regression for the purpose of localizing a vehicle, preferably an autonomous vehicle, that moves in a traffic environment, the method comprising:a) with a feature extraction means: receiving a number N of images (12) that are indicative of the traffic environment, wherein the feature extraction means (14) determines a four-dimensional feature map (16) from the images (12), wherein a first dimension is the number N, a second dimension is a predetermined height H in pixels, a third dimension is a predetermined width W in pixels and a fourth dimension corresponds to a predetermined number C of channels of the feature extraction means (14);b) with a graph structure generation means (18): generating a feature graph structure (20) that includes a feature node set (22) and a feature edge set, wherein the feature node set (22) is generated as having a plurality of feature nodes (24) and each feature node (24) includes a pixel-wise feature based on the feature map obtained in step a), wherein the feature edge set is generated as fully connected between any two feature nodes (22);c) with a feature graph partial differential equation (PDE) layer (26): receiving the feature graph structure (20) of step b) and determining an updated feature graph structure (28) by solving a PDE (26) based on a graph updating function;d) with a node ordinary differential equation (ODE) layer (32): receiving the updated feature graph structure (28) of step c) and determining an updated node feature graph structure (34) by solving an ODE (32, 60) based on a node feature updating function;e) with a data conversion means (38): converting the updated node feature graph of step d) into a plurality of decomposed feature maps (40);f) with a vector embedding means (42): for each decomposed feature map (40), determining at least one vector embedding (46) by pooling;g) with a vector embedding graph structure generation means (48): generating a vector embedding structure that includes a vector embedding node set (52) and a vector embedding edge set (56), wherein the vector embedding node set (52) is generated as having a plurality of vector embedding nodes (54) and each vector embedding node (54) includes one vector embedding (46) obtained in step f),wherein the vector embedding edge set (56) is generated as fully connected between any two vector embedding nodes (54);h) with a vector embedding differential equation (DE) layer (57): receiving the vector embedding graph structure (50) of step g) and determining an updated vector embedding graph structure (62) by sequentially solving a PDE (26, 58) and an ODE (32, 60) based on at least one vector embedding updating function;i) with a pose decoder means (66): decoding the updated vector embedding graph structure (62) of step h) into a set of camera pose coordinates that are indicative of a pose of a camera relative to the traffic environment.

2. The method according to claim 1, wherein in step a) the feature extraction means (14) includes a convolutional neural network that determines the four-dimensional feature map (16) based on the set of input images (12).

3. The method according to any of the preceding claims, wherein in step b) the graph structure generation means restructures the four-dimensional feature map (16) into a one-dimensional feature map, wherein a first dimension is the number C of channels, wherein pixel-wise features are uniquely identified by a pixel-wise index i based on N, H and W.

4. The method according to any of the preceding claims, wherein in step c) the graph updating function is determined for each feature node (24) using a multi-head attention fully-connected layer.

5. The method according to any of the preceding claims, wherein in step d) the node feature updating function is determined using a plurality of fully-connected layers that are configured as a multi-layer perceptron.

6. The method according to any of the preceding claims, wherein in step e) the data conversion means (38) generates each decomposed feature map (40) as a three-dimensional decomposed feature map (40), wherein a first dimension corresponds to the height H, a second dimension corresponds to a width W and a third-dimension corresponds to the number C of channels, wherein each decomposed feature map (40) is uniquely identified by an image index i based on the number N of images (12).

7. The method according to any of the preceding claims, wherein in step f) the vector embedding means (42) embeds the decomposed feature maps (40) by maxpooling or average pooling.

8. The method according to any of the preceding claims, wherein in step h) the updated vector embedding graph structure (62) is determined by:hi) solving the PDE (26, 58) based on a first vector embedding updating function that is determined for each vector embedding node using a multi-head attention fully-connected layer;h2) solving the ODE (32, 60) based on a second vector embedding updating function that is determined using a plurality of fully-connected layers that are configured as a multi-layer perceptron.

9. The method according to any of the preceding claims, wherein in step i) i1) the pose decoder means (66) is configured as a branch decoder; and / or i2) the pose decoder means (66) has a common first layer (68) that receives the output of step h), and the common first layer (68) is configured to decouple the vector embeddings (46) into location related vector embeddings and rotation related vector embeddings; and / ori3) the pose decoder means (66) has a branched second layer (70) that receives the output of step i2), and the branched second layer (70) is configured to decode the location related vector embeddings and the rotation related vector embeddings into the camera pose coordinates.

10. The method according to any of the preceding claims, wherein in step i) the camera pose coordinates include three-dimensional camera location coordinates that describe the position of the camera that recorded the set of images (12) and / or four-dimensional camera rotation coordinates that describe the orientation of the camera that recorded the set of images (12) relative to the traffic environment, respectively.

11. A computer-implemented method for navigating a vehicle, preferably an autonomous vehicle, through a traffic environment, the method comprising:a) recording a set of images (12) with an imaging sensor of the vehicle and performing a method according to any of the previous claims on the recorded images (12) in order to obtain camera pose coordinates within the traffic environment;b) determining a current location of the vehicle based on the camera pose coordinates obtained in step a) and planning a path between the current location and a destination for the vehicle to follow;c) generating a control signal that informs a driver of the vehicle of the planned path of step b) or generating a control signal that causes the vehicle to follow the planned path.

12. A computer-implemented method for constructing a map of a traffic environment, the method comprising:a) driving a vehicle through the traffic environment and recording a set of images (12) along a driving path;b) performing a method according to any of the preceding claims on the set of images (12) of step a) in order to obtain camera pose coordinates;c) generating a map model of the traffic environment based on the camera pose coordinates and storing the map model for use in a navigation task.

13. A data processing system comprising means for carrying out the method according to any of the claims 1 to 10 and / or means for carrying out step a) of the method according to claim 11 and / or means for carrying out step b) of the method according to claim 12.

14. A vehicle, preferably an autonomous vehicle comprising a data processing system according to claim 13.

15. A computer program or computer-readable medium comprising instructions which, when executed by a data processing system, cause the data processing system to carry out the method according to any of the claims 1 to 12.

Images