Monocular visual odometry method and system based on bird's-eye view representation and differentiable weighted Procrustes solver

Abstract

The present invention discloses a monocular visual odometry method and system based on bird's-eye view representation and a differentiable weighted Procrustes solver. The method and system include: constructing a pose estimation model, which can extract and match key points in a bird's-eye view image and perform relative pose estimation through an interpretable weighted Procrustes solver; constructing a loss function based on pose information, optimizing the pose estimation model parameters using the loss function, and then using the pose estimation model with the optimized parameters to estimate the pose information between two frames of monocular images to realize monocular visual odometry. This method can excel in reducing scale drift and exhibit stable performance under different weather and lighting conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of the combination of computer vision and autonomous driving, and specifically relates to a monocular visual odometry method and system based on bird's-eye view representation and a differentiable weighted Procrustes solver. Background Art

[0002] Monocular visual odometry (MVO) is a positioning technology widely used on ground vehicles, such as logistics robots and autonomous vehicles, due to its low hardware cost and easy deployment. As a key complement to the Global Positioning System (GPS) and inertial navigation systems, MVO ensures the stable operation of unmanned systems while providing a more economical alternative to LiDAR (Light Detection and Ranging). However, a key issue with MVO technology is that, in long-range applications, the lack of depth information leads to scale drift, which limits its application.

[0003] Traditional MVO methods, such as direct, feature-based, and semi-direct methods, typically address depth uncertainty by using the first few frames as a global reference during the initialization phase to establish a relative scale. However, the global scale of these methods depends on the initial motion state, so their performance degrades significantly when the initial motion differs significantly from subsequent motion. Furthermore, due to calibration errors, feature matching errors, and motion blur, these traditional methods suffer from severe scale drift after long runs, limiting their application to long-distance tasks.

[0004] To address these issues, several deep learning-based MVO methods have emerged in recent years. These methods utilize deep learning networks to reduce scale drift. Early deep learning methods directly extracted features from perspective images and regressed poses, but their performance was relatively poor. Later methods combined the modeling capabilities of deep learning with the interpretability of traditional methods. They achieved better performance by training optical flow prediction and depth estimation networks through supervised learning, combined with traditional multi-view geometry and back-end optimization. However, while self-supervised learning methods reduce scale drift to some extent, their performance remains inferior to supervised learning methods due to scale ambiguity.

[0005] Recently, with the prevalence of the ground plane assumption in autonomous driving scenarios, bird's-eye view (BEV) representation methods have begun to gain widespread attention in the fields of computer vision and autonomous driving. Compared with perspective images, BEV representation methods have shown better performance in 3D detection and segmentation tasks. However, existing BEV odometry methods often rely on semantic segmentation tasks, which not only increases the annotation cost but also limits the possibility of end-to-end pose supervision optimization. Therefore, it is unclear whether the scale drift reduction effect of existing BEV representation methods in odometry tasks is due to the uniform scale structure of the BEV representation itself or the auxiliary information introduced by the segmentation task. Summary of the Invention

[0006] In light of the above, the present invention aims to provide a monocular visual odometry method based on a bird's-eye view representation and a differentiable weighted Procrustes solver. This method utilizes the bird's-eye view representation and a differentiable weighted Procrustes solver, enabling end-to-end training using only pose supervision. Experimental validation on multiple datasets demonstrates that the proposed method excels in reducing scale drift and exhibits stable performance across diverse weather and lighting conditions.

[0007] To achieve the above-mentioned object of the invention, the embodiment provides a monocular visual odometry method based on bird's-eye view representation and a differentiable weighted Procrustes solver, comprising the following steps:

[0008] Construct a pose estimation model, which includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective features from two adjacent monocular image frames and map them into a bird's-eye view feature map, namely a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map and obtain the position weights, validity weights, and feature point descriptors of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weights, validity weights, and descriptors to obtain feature point pairs with matching confidence; the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and use the BEV feature map to calculate the relative pose information between the two monocular image frames;

[0009] A loss function is constructed based on the true pose value and the predicted pose information of the pose estimation model. After using the loss function to optimize the pose estimation model parameters, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images, realizing a high-precision, low-scale drift monocular visual odometry.

[0010] Preferably, the visual BEV encoder includes a feature extraction submodule, a camera parameter auxiliary encoding submodule, a depth distribution prediction submodule, and a feature mapping submodule;

[0011] The feature extraction submodule is used to extract a multi-scale perspective feature map from a monocular image;

[0012] The camera parameter auxiliary encoding submodule is used to encode the camera parameters and weight the camera parameter encoding and the perspective view feature map through channel attention to obtain a weighted feature map;

[0013] The depth distribution prediction submodule is used to predict the depth distribution characteristics of the features through a deep network based on the weighted feature map;

[0014] The feature mapping submodule is used to perform cone mapping on the features fused with the weighted feature map and the depth distribution feature to the space around the vehicle coordinate system, and then obtain the BEV feature map based on the voxel pooling layer.

[0015] Preferably, the feature point extraction module adopts a U-Net style encoder-decoder network structure;

[0016] The encoder downsamples the BEV feature map to a low resolution to capture multi-scale feature information through its multi-layer convolution and downsampling operations;

[0017] The decoder contains two parallel branches, which are used to predict the position weight of the feature points and the validity weight of the feature points respectively. Each branch contains the same number of upsampling stages as the encoder layers, restoring the feature map from low resolution to the original resolution; at the same time, the feature maps of each downsampling stage of the encoder are upsampled to the original size and spliced ​​in the channel dimension to form a multi-dimensional feature point descriptor map with the same resolution as the original.

[0018] Preferably, the feature matching module divides the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and performs feature point matching in each block based on position weight, validity weight and descriptor to obtain feature point pairs with matching confidence, including:

[0019] The BEV feature map of the previous frame is divided into multiple non-overlapping blocks, and the position of the corresponding candidate feature point in each block is calculated according to the corresponding position weight, which is expressed as:

[0020]

[0021] Among them, i and j represent the index of the point, block n represents the nth block, represents the coordinates of point i in this block in the original BEV feature map, Represents the position weights of the feature points of the i-th point and the j-th point, Indicates the nth block n The position of the candidate feature points in ;

[0022] The candidate feature point descriptors of each block in the previous frame and the dense descriptors of the next frame are normalized, and the similarity matrix is ​​calculated by tensor multiplication. The similarity matrix is ​​scaled using the temperature-weighted softmax function to calculate the candidate matching score of any two feature points:

[0023]

[0024] in, Represents the candidate feature point p in the previous frame c and feature point p in the next frame m The candidate matching score between and Represents the feature point descriptor D from the previous frame key1 The feature point p c The descriptor and the feature point descriptor D from the next frame key2 The feature point p m descriptor, τ represents the temperature parameter, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the feature point p k Descriptor of;

[0025] Calculate the matching coordinates of the candidate feature points in the previous frame in the next frame based on the candidate matching scores:

[0026]

[0027] Among them, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set, Represents the kth feature point p in the next frame k Coordinates in the original BEV feature map, Represents block n The candidate feature point p in c With the kth feature point p k The candidate matching score between Represents the calculated matching point p in the next frame m Coordinates on the BEV characteristic map;

[0028] Calculate the feature point pair (p c , p m) represents the final confidence score:

[0029]

[0030] in, and Respectively represent p c and p m The validity weights on the respective BEV characteristic maps, Represents the final score of the feature point pair.

[0031] Preferably, the differentiable weighted Procrustes solver uses the matched feature point pairs and their confidences to calculate the relative pose information between the two monocular images using the BEV feature map through differentiable weighted singular value decomposition, including:

[0032] First, the position coordinates of the matching point pair are converted to the position coordinates of the actual size, and the weighted centroid of each feature point in the position coordinates of the actual size is calculated based on the matching score;

[0033] Based on the centralized matching point pairs determined by the weighted centroid, the weighted covariance matrix is ​​calculated, and the covariance matrix is ​​subjected to singular value decomposition to calculate the rotation matrix and translation vector to form the relative pose information between the two monocular images.

[0034] Preferably, a loss function is constructed based on the true pose and the predicted pose information of the pose estimation model, including: constructing a loss function based on the translation error and rotation error between the predicted pose and the true pose, expressed as:

[0035] L pose =|t pred,x -t gt,x |+|t pred,y -t gt,y |+α·|θ pred -θ gt |

[0036] Among them, θ pred and θ gt Represents the predicted yaw angle in the rotation matrix and the true value of the yaw angle, t pred,x and t pred,y Represents the components of the predicted translation vector in the x and y directions, t gt,x and t gt,y They represent the components of the x and y directions in the true value of the translation vector, and α represents the weighting factor.

[0037] Preferably, when optimizing the pose estimation model parameters using the loss function, a staged training strategy is used to optimize the model parameters: first, global feature point pre-training is performed to stabilize the optimization process, and then a guided translation convergence strategy is used to improve the pose estimation accuracy, specifically:

[0038] During the initial training phase, the validity weights of all feature points are set to 1. The predicted values ​​of the rotation matrix calculated using singular value decomposition are replaced with the true values ​​of the rotation matrix. The weighted centroid of the subsequent frame is transformed into the coordinate system of the previous frame to calculate the translation vector. This guides the translation vector to converge in the correct direction and prevents the translation vector network from receiving incorrect gradient directions during optimization due to incorrect rotation matrix predictions.

[0039] To achieve the above-mentioned object of the invention, an embodiment of the present invention further provides a monocular visual odometry system based on bird's-eye view representation and a differentiable weighted Procrustes solver, comprising:

[0040] a pose estimation model construction unit, which is used to construct a pose estimation model, comprising a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective view features from two adjacent monocular image frames and map them into a bird's-eye view feature map, i.e., a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map and obtain the position weights, validity weights, and feature point descriptors of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weights, validity weights, and descriptors to obtain feature point pairs with matching confidence; the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and use the BEV feature map to calculate the relative pose information between the two monocular image frames;

[0041] The parameter optimization unit is used to construct a loss function based on the true pose value and the predicted pose information of the pose estimation model. After using the loss function to optimize the parameters of the pose estimation model, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images to realize monocular visual odometry.

[0042] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver.

[0043] To achieve the above-mentioned purpose of the invention, the embodiment further provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-mentioned monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This method leverages the advantages of BEV representation to simplify pose estimation from 6 degrees of freedom to 3 degrees of freedom, and reduces scale drift by utilizing the unified metric scale characteristics of BEV feature maps. Compared to traditional methods that only obtain relative scales or require complex learning methods and multi-task supervision, this method relies solely on pose supervision to achieve end-to-end training, significantly simplifying system design while ensuring accurate motion estimation.

[0046] This paper uses a unique phased training strategy to improve network performance. In the first two training cycles, a global feature point pre-training strategy sets the validity weights of all feature points to 1, ensuring feature learning within the global BEV feature map. In the first N training cycles, a guided translation convergence strategy uses the true rotation matrix to align the center of mass of the second frame's feature map, effectively reducing translation error. This training strategy significantly improves network convergence performance and pose estimation accuracy. The loss function uses a weighted L1 loss for translation and rotation errors, balancing the contributions of the two types of errors through weight coefficients.

[0047] Experimental results on datasets such as NCLT, Oxford, and KITTI demonstrate significant advantages over existing methods: achieving average relative translation errors of 8.67% and 6.92% on the NCLT and Oxford datasets, and average relative rotation errors of 3.53° / 100m and 1.33° / 100m, respectively. Competitive performance is also maintained on the KITTI dataset, which has significant elevation variations. In terms of real-time performance, the proposed method runs at 16 FPS on an RTX 4090 GPU, with a GPU utilization rate of 45% and a memory footprint of only 7.5GB, demonstrating excellent computational efficiency.

[0048] Furthermore, the interpretable backend design proposed in this paper, including modules such as feature point extraction and matching based on the BEV feature map, weight calculation, and a differentiable weighted Procrustes solver, not only provides a clear pose estimation process but also guides the network to learn better feature representations through structured feedback. This design not only improves accuracy but also enhances the system's generalization capabilities across different scenarios.

[0049] In summary, the method of the present invention achieves high-precision motion estimation and robust pose prediction by introducing feature point matching, weight calculation, differentiable weighted Procrustes solver, and carefully designed pose supervision and optimization strategies. Compared with the existing technology, the present invention not only effectively reduces the scale drift problem in traditional monocular visual odometry, but also improves the real-time performance and computational efficiency of the system by simplifying the calculation process. These innovations enable the method of the present invention to show significant advantages in application scenarios such as autonomous driving and robot autonomous navigation that require high precision and real-time performance, providing a technical solution that is efficient, accurate, and widely applicable for related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 Flowchart of a monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver provided in an embodiment;

[0052] Figure 2 Schematic diagram of the structure of the pose estimation model provided in the embodiment;

[0053] Figure 3 is a visualization diagram of trajectory and key point matching provided by an embodiment;

[0054] Figure 4 is a comparison of the absolute trajectory error (ATE) indicator of different methods provided in the embodiment on two data sets;

[0055] Figure 5 1 is a structural diagram of a monocular visual odometry system based on a bird's-eye view representation and a differentiable weighted Procrustes solver provided in an embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0057] The present invention aims to address several key challenges faced by existing monocular visual odometry technology in practical applications, thereby meeting the demand for high-precision, low-cost positioning systems for ground vehicles such as autonomous vehicles and logistics robots. Specifically, the present invention provides a monocular visual odometry method based on a bird's-eye view representation and a differentiable weighted Procrustes solver, which improves and optimizes the following aspects:

[0058] 1. Reduce scale drift: Due to the lack of depth information, MVO technology is prone to scale drift in long-distance applications, which limits its widespread practical application. This invention introduces a unified metric scale Bird's Eye View (BEV) representation, enabling the system to more effectively handle scale drift in long-distance motion, significantly improving positioning accuracy and reliability.

[0059] 2. Enhanced robustness in complex environments: Traditional MVO methods often experience significant performance degradation in complex environments such as lighting changes, low-texture areas, and dynamic scenes. To address this issue, this paper designs a new pose estimation model based on BEV representation. Combined with a weighted Procrustes solver, this model effectively improves the robustness of the system in complex environments by extracting and matching key points on the BEV feature map, enabling it to operate stably in a variety of complex scenarios.

[0060] 3. Improved generalization: Existing deep learning-based MVO methods have limited generalization capabilities in new scenarios, primarily due to their lack of interpretability. This paper, by employing an interpretable weighted Procrustes solver and end-to-end training methods, enables the system to not only perform well on the training data but also generalize better to unseen scenarios, ensuring its broad applicability across diverse application environments.

[0061] 4. Simplify implementation and reduce costs: Existing BEV representation odometry methods rely on additional tasks such as semantic segmentation, resulting in high annotation costs and complex implementation. This invention avoids reliance on additional tasks by directly using BEV representation for pose estimation, reducing system implementation complexity and cost, and simplifying the entire development and deployment process.

[0062] 5. Improved Overall Performance: This invention not only innovates in reducing scale drift and enhancing robustness, but also achieves higher overall system performance through optimized algorithmic framework and training strategies, including higher positioning accuracy, faster computation speed, and more stable long-distance tracking capabilities. Validation on multiple datasets demonstrates the superiority of this invention in terms of scale consistency and pose accuracy.

[0063] In summary, the present invention aims to comprehensively solve the problems existing in the existing technology by introducing a unified metric scale BEV representation and a weighted Procrustes solver, providing a more efficient, reliable and easy-to-implement solution to meet the needs of autonomous driving and other ground vehicles for high-precision and low-cost positioning systems.

[0064] Based on the above invention concept, Figure 1 As shown, the embodiment provides a monocular visual odometry method based on bird's-eye view representation and a differentiable weighted Procrustes solver, comprising the following steps:

[0065] S1 builds a pose estimation model that can extract and match key points in the bird's-eye view (BEV) and perform relative pose estimation through an interpretable weighted Procrustes solver.

[0066] like Figure 2 As shown, the pose estimation model provided by the embodiment includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver.

[0067] In this embodiment, a visual BEV encoder (i.e., a PV-BEV encoder) is used to extract multi-scale perspective features from two adjacent monocular image frames and map them into a bird's-eye view feature map. Specifically, the LSS (Lift, Splat, Shoot) architecture is used to extract three-dimensional spatial features from the perspective images and map them into BEV space. The LSS architecture predicts the depth distribution to achieve a scale-consistent BEV projection. Combined with camera parameters (i.e., camera intrinsics and extrinsics), it enhances interpretability and supports multi-camera setups with accurate extrinsics.

[0068] Specifically, the visual BEV encoder includes a feature extraction submodule, a camera parameter auxiliary encoding submodule, a depth distribution prediction submodule, and a feature mapping submodule. The feature extraction submodule is used to extract multi-scale perspective view feature maps from monocular images. By using ResNet-50 as the backbone network, multi-scale features can be extracted from the input monocular image. The extracted features are fused through a feature pyramid network (FPN) to generate a multi-scale perspective view (PV) feature map, i.e., a PV feature map.

[0069] The camera parameter auxiliary encoding submodule is used to encode the camera parameters and then weight the camera parameter encoding with the perspective view feature map through channel attention weighting (SE) to obtain a weighted feature map. Specifically, the camera parameters are encoded through a multi-layer perceptron (MLP). The encoded camera parameters are then element-wise multiplied with the PV feature map to generate an enhanced weighted feature map to ensure spatial consistency of image features. The weighted feature map is then passed through a convolutional layer to generate the final weighted feature map.

[0070] The depth distribution prediction submodule is used to predict the depth distribution characteristics of features based on the weighted feature map through a deep network. The deep network includes a deformable convolutional network (DCN), atrous spatial pyramid pooling (ASPP), and convolutional layer stacking residual blocks (Res Blocks). The depth distribution map is optimized through pose supervision, allowing the depth estimation process to be closely integrated with pose estimation. The generation of depth distribution features involves weighting the distribution information of each pixel in the feature map at different depths to form a 4D depth distribution feature map.

[0071] The feature mapping submodule is used to perform frustum mapping of the weighted feature map fused with the depth distribution features to the space surrounding the vehicle coordinate system. This is then used in a voxel pooling layer to generate a BEV feature map. Specifically, frustum projection is used to map the 4D depth distribution features to the BEV space, generating a BEV feature map. Voxel pooling is then used to compress the information along the Z axis, ultimately forming a BEV representation with multidimensional features. This operation ensures that image features are accurately mapped to the BEV space of the vehicle coordinate system.

[0072] In one embodiment, the feature point extraction module is used to extract representative feature points from the BEV feature map and obtain the position weights, validity weights, and feature point descriptor maps of the feature points. Specifically, a U-Net-style encoder-decoder network structure is adopted; wherein the encoder downsamples the BEV feature map to a low resolution through its multi-layer convolution and downsampling operations to capture multi-scale feature information. Specifically, the encoder path includes four stages of progressive downsampling, reducing the feature map from its original resolution to a more compact representation.

[0073] The decoder consists of two parallel branches, one for predicting the position weight W of the feature point pos and the validity weight W of the feature points valid Each branch contains the same number of upsampling stages as the encoder layers, restoring the feature map from low resolution to the original resolution; at the same time, the feature maps of each downsampling stage of the encoder are upsampled to the original size and spliced ​​in the channel dimension to form a multi-dimensional feature point descriptor map D with the same resolution as the original key .

[0074] In this embodiment, the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks and perform feature point matching within each block based on position weights, validity weights, and descriptors to obtain feature point pairs with matching confidence levels to estimate the relative motion of the vehicle. Specifically, the BEV feature map of the previous frame is first divided into multiple non-overlapping blocks, and the position of the corresponding candidate feature point is calculated for each feature point in each block based on the corresponding position weight, expressed as:

[0075]

[0076] Among them, i and j represent the index of the point, block n represents the nth block, represents the coordinates of point i in this block in the original BEV feature map, Represents the position weights of the feature points of the i-th point and the j-th point, Indicates the nth block n The position of the candidate feature points in ;

[0077] The candidate feature point descriptors of each block in the previous frame and the dense descriptors of the subsequent frame are normalized. It should be noted that since the subsequent frame is full-size, the corresponding descriptor composed of all feature points is called a dense descriptor. The similarity matrix is ​​calculated by tensor multiplication. In order to avoid interference from non-matching feature points at a distance, the system sets a mask range based on the typical limitations of vehicle motion, and returns the similarity values ​​outside the range to zero. The similarity matrix is ​​scaled using a temperature-weighted softmax function to calculate the candidate matching score of the two feature points:

[0078]

[0079] in, Represents the candidate feature point p in the previous frame c and feature point p in the next frame m The candidate matching score between and Represents the feature point descriptor D from the previous frame key1 The feature point p c The descriptor and the feature point descriptor D from the next frame key2 The feature point p m descriptor, τ represents the temperature parameter, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the feature point p k Descriptor of;

[0080] The matching coordinates of the candidate feature points in the previous frame in the subsequent frame are calculated based on the candidate matching scores, that is, the matching coordinates are obtained by weighted summation with the pixel coordinates in the feature point descriptor of the subsequent frame:

[0081]

[0082] Among them, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set, Represents the kth feature point p in the next frame k Coordinates in the original BEV feature map, Represents block n The candidate feature point p in c With the kth feature point p k The candidate matching score between Represents the calculated matching point p in the next frame m Coordinates on the BEV characteristic map;

[0083] Calculate the matching point pair (p c , p m ) represents the final score of confidence. The final score comprehensively considers the similarity and validity weight of the feature point descriptors to ensure the accuracy and reliability of the matching results. The matching weight reflects the similarity of the feature point descriptors and their confidence at the matching position:

[0084]

[0085] in, and Respectively represent p c and p m The validity weights on the respective BEV characteristic maps, Represents the final score of the feature point pair.

[0086] In this embodiment, a differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and the BEV feature map to calculate the relative pose information between the two monocular images. Specifically, it calculates the optimal rigid transformation between the two sets of key point pairs, including the rotation matrix and translation vector. This solver is based on the weighted singular value decomposition (Weighted SVD) algorithm, supports gradient backpropagation, is suitable for deep learning training, and maintains interpretability. The specific process is:

[0087] First, the matching point pair (p c , p m )'s position coordinates Converted to actual size position coordinates The weighted centroid of each feature point in the actual size position coordinates will be calculated based on the matching score The coordinates of the centralized matching point pairs are obtained by subtracting the centroid of each feature point pair from their respective coordinates. and The formula is:

[0088]

[0089] in, is the final score of the matching point pair, n represents a value in the matching score, and hw is the number of partition blocks;

[0090] Then, based on the coordinates of the centered matching point pairs, the weighted covariance matrix W is calculated cov , and perform singular value decomposition on the covariance matrix to calculate the rotation matrix R and translation vector t to form the pose information.

[0091]

[0092] R=U·diag(1,1,det(UV T ))·V T

[0093]

[0094] The entire process supports gradient backpropagation, allowing the weighted Procrustes solver to be effectively optimized during deep learning training.

[0095] S2, constructs a loss function based on the true pose value and the predicted pose information of the pose estimation model, and uses the loss function to optimize the pose estimation model parameters. Then, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images to realize monocular visual odometry.

[0096] To optimize the model's pose estimation, this paper designs a loss function based on the true pose value and the predicted pose information of the pose estimation model. This loss function includes the L1 loss of the rotation matrix and the translation vector, which is used to measure the difference between the predicted pose parameters and the true value. To balance the errors of rotation and translation, a weighting factor α is introduced into the loss function, allowing the network to effectively reduce the errors between the two during the optimization process. The loss function is expressed as:

[0097] L pose =|t pred,x -t gt,x |+|t pred,y -t gt,y |+α·|θ pred -θgt |

[0098] Among them, θ pred and θ gt Represents the predicted yaw angle in the rotation matrix and the true value of the yaw angle, t pred,x and t pred,y Represents the components of the predicted translation vector in the x and y directions, t gt,x and t gt,y They represent the components of the x and y directions in the true value of the translation vector respectively. In the embodiment, α can be set to 10 to balance the errors of rotation and translation, because the rotation error will affect the calculation of translation.

[0099] When optimizing the pose estimation model parameters using the loss function, a phased training strategy is used to optimize the model parameters: first, global feature point pre-training is performed to stabilize the optimization process, and then a guided translation convergence strategy is used to improve the pose estimation accuracy, thereby accelerating convergence and improving network performance. Specifically:

[0100] Strategy 1: Global feature point pre-training: In the initial stage of training, the validity weights of all feature points are set to 1 to ensure that the network pays attention to every position in the BEV feature map evenly. This method helps stabilize the optimization process and enables the network to learn the differences between key points evenly across the entire feature map.

[0101] Strategy 2: Guided Translation Convergence: During the first N (e.g., 10) training cycles, inaccurate rotation matrix predictions can lead to large errors in the translation vector, hindering network convergence. Therefore, during the initial N training cycles, the true value of the rotation matrix is ​​used to replace the predicted value calculated using singular value decomposition. The weighted center of mass of the subsequent frame is transformed into the coordinate system of the previous frame to calculate the translation direction. Subsequently, the network gradually transitions to using the predicted rotation matrix for calculations, improving the accuracy of translation estimation.

[0102] The embodiment also provides an experimental example to verify the effect of the above method, specifically:

[0103] This paper conducts comprehensive experimental validation on multiple public datasets to evaluate its performance in different environments and conditions. These datasets include:

[0104] NCLT dataset: This dataset features high jitter, significant illumination variations, and motion blur, making it suitable for evaluating algorithms under complex visual conditions. The training sequences (2012-01-08, 2012-02-04, and 2013-04-05) cover different weather conditions during midday and afternoon hours, while the test sequences (2012-02-02, 2012-02-19, 2012-03-17, and 2012-08-20) incorporate unseen time-of-day, weather, and seasonal characteristics to assess the generalization capabilities of algorithms.

[0105] Oxford Radar RobotCar Dataset: This dataset contains complex urban driving scenes with a dynamic background and a varying number and size of traffic participants, making it suitable as a challenging urban testbed. The training sequences (19-01-11-13, 19-01-14-14, and 19-01-15-14) represent typical conditions, while the test sequences (19-01-11-12, 19-01-15-13, 19-01-16-14, and 19-01-17-12) include strong lighting conditions not seen during training.

[0106] KITTIOdometry Dataset: This dataset is widely used for odometry comparisons. Many methods provide pretrained models and published results, facilitating consistent benchmarking. The training sequences (00-08) and test sequences (09-10) contain significant vertical variation, reaching 38 meters in sequence 09 and 24 meters in sequence 10. This variation poses a challenge to 3-DOF odometry methods that assume a flat ground surface.

[0107] We apply the same hyperparameter settings across all datasets to avoid dataset-specific tuning. The BEV grid is set to 256×256 with a resolution of 0.4 meters to accommodate the data characteristics of NCLT (low speed and high jitter), the urban complexity of Oxford, and the high-speed, open scenes of KITTI.

[0108] In terms of baseline setup, our method is compared with three types of baseline methods: traditional methods from a PV perspective, learning-based methods from a PV perspective, and comparative methods from a BEV perspective. Each type of baseline method evaluates specific aspects of our method design:

[0109] Perspective View (PV)-based baselines: These include the traditional method ORB-SLAM3 and the learning-based methods DeepVO, TartanVO, DF-VO, DROID-SLAM, and DPVO. These methods perform odometry estimation under perspective view.

[0110] Bird’s-Eye View (BEV)-based baselines: We design three BEV baseline methods to study the impact of interpretable components (e.g., feature point matching module, weighted Procrustes solver):

[0111] (a) BEV+CNNs+MLPs: uses convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) for direct feature extraction and pose regression without keypoint-based matching;

[0112] (b) BEV+global correlation+MLPs: global correlation is calculated on the BEV feature map for motion estimation, but lacks spatial constraints;

[0113] (c) BEV+local correlation+MLPs: By focusing on local areas and increasing spatial constraints, the accuracy of inter-frame estimation is improved.

[0114] The goals of the comparison are twofold: first, to verify the advantage of the unified metric-scaled BEV representation in simplifying pose estimation (from 6 DOF to 3 DOF); second, to evaluate whether the interpretable design can guide the intermediate layers to better anchor the scale and capture key features, thereby further improving the overall performance and reducing scale drift.

[0115] Experimental parameter settings: To simulate different speeds and motions, we randomly sample frame pairs and oversample frames with significant rotation. Test trajectories are calculated by accumulating relative poses over a fixed interval without any post-processing. Training is performed on an NVIDIA RTX 4090 GPU using the Adam optimizer with an initial learning rate of 1×10 -4 , decaying by 0.95 per epoch. The entire training process takes about 72 hours.

[0116] Evaluation strategy: The accuracy of the odometry is evaluated by RTE (mean translation RMSE), RRE (mean rotation RMSE), and ATE (mean translation error). RTE and RRE are calculated in the range of 100 to 800 meters, respectively, while ATE measures the average translation error between the predicted pose and the ground truth pose. In addition, a composite metric (log2(SE(3) / Sim(3))) is designed to quantify the error change before and after scale alignment and evaluate the performance of different methods in terms of scale consistency. To ensure fairness, for MVO methods that lack absolute scale or rely on pre-trained models, the first 10 meters of ground truth data are used for scale alignment, which is consistent with the actual MVO usage scenario.

[0117] Ablation experiment results show that the design of each module has a significant impact on the overall performance:

[0118] Keypoint validity weights: Adding validity weights to keypoints to capture their global importance, combined with position weights, significantly improved keypoint matching accuracy. Removing these weights significantly degraded performance, demonstrating the importance of combining local and global weights.

[0119] Training strategy: The use of global feature point pre-training and guided translation convergence strategy facilitates convergence in the early stages and improves the RTE and RRE indicators.

[0120] BEV hyperparameters: The resolution and size of the BEV grid are adjusted for different datasets. The grid is refined to accommodate the low speed and high jitter of NCLT, and the grid is coarsened to accommodate the high speed and stable motion of Oxford. This further verifies the importance of selecting appropriate BEV hyperparameters based on the environmental conditions.

[0121] Real-time performance: Evaluated on the KITTI dataset using an NVIDIA RTX 4090 GPU with a batch size of 1 and an input size of (3, 1216, 384). This method achieved a processing speed of 16 frames per second (FPS), a GPU utilization of 45%, and a memory usage of 7.5 GB (including data transfer latency). This result demonstrates the potential of this method for real-time visual odometry applications.

[0122] In summary, extensive experiments on multiple datasets demonstrate that our method demonstrates superior performance across all odometry estimation metrics, particularly in scale consistency and complex scene handling, significantly outperforming existing baseline methods. Ablation experiments further validate the effectiveness of each module design, and runtime performance demonstrates its potential for practical application. These results fully demonstrate the innovation and practicality of our method in the field of visual odometry.

[0123] Attachment Figure 3 This is a visualization of the trajectory and feature point matching of the monocular visual odometry method based on bird's-eye view representation on the NCLT and Oxford datasets. The figure shows the comparison of the method with the ground truth trajectory, and different scenarios such as straight, right turn and left turn are evaluated on the NCLT and Oxford datasets respectively. The results show that the method can achieve high path accuracy in long-distance and complex turning scenarios without the need for any post-processing steps. In addition, through key point matching and weighted Procrustes solver, the method can effectively identify and utilize geometric patterns in BEV feature maps, thereby improving the structure and interpretability of pose estimation. It demonstrates the robustness and accuracy of the present invention in long-distance and complex turning scenarios, and verifies the system's ability to handle complex trajectories in different environments. Specifically, it includes the following key parts:

[0124] 1. Trajectory comparison chart (upper part):

[0125] The left side shows the trajectory comparison results in the NCLT dataset, showing the comparison between the trajectory predicted by BEV-DWPVO and the actual reference trajectory (GT). Figure 3 The positions A, B, and C marked in the figure correspond to the straight section, right turn scenario, and left turn scenario, respectively. These positions are used to further demonstrate the operating results of the system in different situations.

[0126] The right side shows the trajectory comparison results in the Oxford dataset, which also shows the comparison between the system's predicted trajectory and the actual reference trajectory. Figure 3 The positions D, E, and F marked in the figure correspond to the straight-ahead, right-turn, and left-turn scenarios, respectively.

[0127] 2. Feature point matching visualization (lower part):

[0128] The image below shows the feature point matching results for the selected locations in the trajectory comparison image above. The background is the BEV feature map of the previous frames. Red dots represent key points in the previous frames, green dots represent key points matched in the next frames, and white lines represent the matching relationships between key points.

[0129] Figure 3 A and D in the figure show the matching of feature points in the straight-ahead scene. Figure 3 Figures B, E, C, and F show the matching results in right-turn and left-turn scenarios. It can be observed that in complex turning scenarios, the matching relationship of feature points is more complicated, but overall the method of the present invention can still accurately perform key point matching and motion estimation.

[0130] Attachment Figure 4 This is a comparison chart of the absolute trajectory error (ATE) metric of different methods on the NCLT and Oxford datasets. It shows the ATE performance of each method under SE(3) and Sim(3) alignment, as well as a comparison of the combined metric Log2(SE(3) / Sim(3)). These results illustrate the differences in scale drift among different methods, further verify the superiority of the present invention in different complex scenarios, and demonstrate its innovative contribution in reducing scale drift and improving motion estimation accuracy.

[0131] 1. ATE Comparison (left panel): This section shows the ATE values ​​of various methods on the NCLT and Oxford datasets. The horizontal axis represents the ATE value, while the vertical axis lists the methods being compared. Figure 4 The blue and orange bars represent the ATE results of SE(3) and Sim(3) alignment on the NCLT and Oxford datasets, respectively. The proposed method shows the smallest ATE value, indicating that its accuracy is better than other methods under different alignment modes.

[0132] 2. Scale Drift Comparison (right side of the figure): This section shows a boxplot of the combined metric Log2(SE(3) / Sim(3)), which is used to quantify the performance of different methods on scale drift. The horizontal axis represents the Log2(SE(3) / Sim(3)) value, where the closer it is to 0, the better the scale consistency. Figure 4 It can be seen from the figure that the method of the present invention has the smallest scale drift on the two data sets and shows the best scale consistency.

[0133] 3. Illustration: The data shown in the figure show that the present invention can maintain highly stable pose estimation accuracy under different environmental conditions by combining BEV representation and differentiable weighted Procrustes solver. Compared with other methods (such as ORB-SLAM3, DeepVO, TartanVO, DF-VO, DROID-SLAM, DPVO, etc.), the robustness and generalization ability of the present invention in complex environments have been significantly improved, especially in dealing with scale drift problems. Due to the use of a solver with better interpretability, this method has better generalization than other methods that also use BEV representation, and the average performance on multiple test sequences shows better scale consistency and accuracy.

[0134] The monocular visual odometry method described above, based on bird's-eye view representation and a differentiable weighted Procrustes solver, is primarily used for real-time positioning and navigation tasks for ground vehicles such as autonomous vehicles and logistics robots. It is particularly well-suited for high-precision, low-drift 3-degree-of-freedom (3DoF) motion estimation in complex environments. The core advantage of this method lies in its high-precision motion estimation achieved through feature point matching and pose supervision, along with excellent generalization and robustness, making it adaptable to a variety of complex scenarios.

[0135] In autonomous vehicle applications, vehicles require accurate real-time positioning and navigation in complex environments such as highways and urban roads. Traditional visual odometry systems can accumulate errors over time, affecting vehicle safety and path planning accuracy. The present invention offers the following advantages in such scenarios:

[0136] 1. Accurate positioning during long driving periods: Autonomous vehicles require continuous acquisition of accurate position and posture information during extended driving. This invention effectively mitigates the scale drift caused by monocular vision by combining bird's-eye-view feature point matching with position and posture supervision, thereby maintaining highly accurate positioning data over long time series.

[0137] 2. Real-time responsiveness in dynamic environments: This system uses highly accurate pose estimation to adjust the vehicle's path in real time to accommodate changing traffic conditions (such as other vehicles merging into other lanes or the sudden appearance of pedestrians). Compared to traditional systems, this system makes fast and accurate decisions in dynamic environments without requiring additional depth information, ensuring safe driving and reliable path planning for autonomous vehicles.

[0138] In the application of logistics robots, when logistics robots perform cargo handling in warehouses or automation tasks in factories, accurate motion estimation is the key to the successful completion of the tasks. The present invention has the following advantages in such application scenarios:

[0139] 1. High-precision positioning in complex environments: Logistics robots operating in large warehouses often need to maintain a stable path and positioning accuracy in narrow aisles and complex environments. By optimizing feature point matching and depth distribution estimation, this invention can provide continuous high-precision pose data in such complex planar environments, effectively reducing the drift problem common in traditional monocular vision systems.

[0140] 2. Low-cost, efficient monocular solution: Due to cost constraints, many logistics robots are equipped with only a monocular camera. This invention fully utilizes the monocular camera's field of view, achieving efficient motion estimation and path planning through precise feature point matching and pose supervision. This enables logistics robots to operate stably in complex logistics environments even with only a monocular camera.

[0141] In applications such as airport guide vehicles and indoor delivery vehicles, the vehicle needs to operate in a flat environment for a long time while maintaining high-precision motion estimation. This invention is particularly suitable for such application scenarios due to its excellent scale consistency and robustness:

[0142] 1. Maintaining Path Accuracy During Long-Term Operation: Plane navigation vehicles operating in environments such as airports and shopping malls must maintain the accuracy of their intended paths over extended periods. The method proposed in this invention allows the vehicle to maintain path accuracy over extended periods, avoiding deviations caused by accumulated errors and enabling stable navigation even in complex planar environments.

[0143] 2. Ability to cope with dynamically changing environments: In busy public spaces, planar navigation vehicles must respond in real time to the sudden appearance of pedestrians and obstacles. This method, through real-time, high-precision pose estimation, enables the vehicle to rapidly adjust its path, avoid dynamic obstacles, and continue its mission, ensuring efficient completion of the mission.

[0144] In summary, the proposed method achieves high-precision, low-drift motion estimation over long periods of time and in complex environments by combining bird's-eye-view feature point matching with pose supervision. It demonstrates excellent performance in applications such as autonomous driving, logistics robotics, and surface navigation vehicles. Its strong generalization and robustness enable it to effectively meet the real-time positioning and navigation needs of various ground vehicles.

[0145] like Figure 5 As shown, the embodiment also provides a monocular visual odometry system based on bird's-eye view representation and differentiable weighted Procrustes solver, including a pose estimation model construction unit 51 and a parameter optimization and application unit 52, wherein the pose estimation model construction unit 51 is used to construct a pose estimation model, which can use key point extraction and matching in the bird's-eye view and perform relative pose estimation through an interpretable weighted Procrustes solver; the parameter optimization and application unit 52 is used to construct a loss function based on pose information, and after using the loss function to optimize the pose estimation model parameters, use the pose estimation model after parameter optimization to estimate the pose information between two frames of monocular images to realize monocular visual odometry.

[0146] Based on the same inventive concept, an embodiment further provides a computing device including a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, the device is used to implement the above-mentioned monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver, which specifically includes the following steps:

[0147] S1, build a pose estimation model that can extract and match key points in the bird's-eye view and perform relative pose estimation through an interpretable weighted Procrustes solver;

[0148] S2, constructs a loss function based on the pose information, and uses the loss function to optimize the pose estimation model parameters. Then, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images to realize monocular visual odometry.

[0149] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver, characterized by: The following steps are involved: Construct a pose estimation model, which includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective features from two adjacent monocular image frames and map them into a bird's-eye view feature map, namely a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map and obtain the position weights, validity weights, and feature point descriptors of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weights, validity weights, and descriptors to obtain feature point pairs with matching confidence; the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and use the BEV feature map to calculate the relative pose information between the two monocular image frames; The specific implementation process of the feature matching module includes: dividing the BEV feature map of the previous frame into multiple non-overlapping blocks, and calculating the position of the corresponding candidate feature points in each block according to the corresponding position weight, which is expressed as: Among them, i and j represent the index of the point, block n represents the nth block, represents the coordinates of point i in this block in the original BEV feature map, Represents the position weights of the feature points of the i-th point and the j-th point, Indicates the nth block n The position of the candidate feature points in ; The candidate feature point descriptors of each block in the previous frame and the dense descriptors of the next frame are normalized, and the similarity matrix is ​​calculated by tensor multiplication. The similarity matrix is ​​scaled using the temperature-weighted softmax function to calculate the candidate matching score of any two feature points: in, Represents the candidate feature point p in the previous frame c and feature point p in the next frame m The candidate matching score between and Represents the feature point descriptor D from the previous frame key1 The characteristic point p c The descriptor and the feature point descriptor D from the next frame key2 The characteristic point p m descriptor, τ represents the temperature parameter, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the feature point p k Descriptor of; Calculate the matching coordinates of the candidate feature points in the previous frame in the next frame based on the candidate matching scores: Among them, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set, Represents the kth feature point p in the next frame k Coordinates in the original BEV feature map, Represents block n The candidate feature point p in c With the kth feature point p k The candidate matching score between Represents the calculated matching point p in the next frame m Coordinates on the BEV characteristic map; Calculate the feature point pair (p c , p m ) represents the final confidence score: in, and Respectively represent p c and p m The validity weights on the respective BEV characteristic maps, Represents the final score of the feature point pair; A loss function is constructed based on the true pose value and the predicted pose information of the pose estimation model. After using the loss function to optimize the pose estimation model parameters, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images to realize monocular visual odometry.

2. The monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The visual BEV encoder includes a feature extraction submodule, a camera parameter auxiliary encoding submodule, a depth distribution prediction submodule, and a feature mapping submodule; The feature extraction submodule is used to extract a multi-scale perspective feature map from a monocular image; The camera parameter auxiliary encoding submodule is used to encode the camera parameters and weight the camera parameter encoding and the perspective view feature map through channel attention to obtain a weighted feature map; The depth distribution prediction submodule is used to predict the depth distribution characteristics of the features through a deep network based on the weighted feature map; The feature mapping submodule is used to perform cone mapping on the features fused with the weighted feature map and the depth distribution feature to the space around the vehicle coordinate system, and then obtain the BEV feature map based on the voxel pooling layer.

3. The monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The feature point extraction module adopts a U-Net style encoder-decoder network structure; The encoder downsamples the BEV feature map to a low resolution to capture multi-scale feature information through its multi-layer convolution and downsampling operations; The decoder contains two parallel branches, which are used to predict the position weight of the feature points and the validity weight of the feature points respectively. Each branch contains the same number of upsampling stages as the encoder layers, restoring the feature map from low resolution to the original resolution; at the same time, the feature maps of each downsampling stage of the encoder are upsampled to the original size and spliced ​​in the channel dimension to form a multi-dimensional feature point descriptor map with the same resolution as the original.

4. The monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The differentiable weighted Procrustes solver uses the matched feature point pairs and their confidence scores, through differentiable weighted singular value decomposition, and uses the BEV feature map to calculate the relative pose information between two monocular images, including: First, the position coordinates of the matching point pair are converted to the position coordinates of the actual size, and the weighted centroid of each feature point in the position coordinates of the actual size is calculated based on the matching score; Based on the centralized matching point pairs determined by the weighted centroid, the weighted covariance matrix is ​​calculated, and the covariance matrix is ​​subjected to singular value decomposition to calculate the rotation matrix and translation vector to form the relative pose information between the two monocular images.

5. The monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The loss function is constructed based on the pose truth and the predicted pose information of the pose estimation model, including: constructing a loss function based on the translation error and rotation error between the predicted pose and the true pose, expressed as: L pose =|t pred,x -t gt,x |+|t pred,y -t gt,y |+α·|θ pred -θ gt | Among them, θ pred and θ gt Represents the predicted yaw angle in the rotation matrix and the true value of the yaw angle, t pred,x and t pred,y Represents the components of the predicted translation vector in the x and y directions, t gt,x and t gt,y They represent the components of the x and y directions in the true value of the translation vector, and α represents the weighting factor.

6. The monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: When optimizing the pose estimation model parameters using the loss function, a phased training strategy is used to optimize the model parameters: first, global feature point pre-training is performed to stabilize the optimization process, and then a guided translation convergence strategy is used to improve the pose estimation accuracy. Specifically: In the initial stage of training, the validity weights of all feature points are set to 1, the true value of the rotation matrix is ​​used to replace the predicted value of the rotation matrix calculated using singular value decomposition, and the weighted center of mass of the subsequent frame is converted to the coordinate system of the previous frame to calculate the translation vector.

7. A monocular visual odometry system based on bird's-eye view representation and differentiable weighted Procrustes solver, characterized by: include: a pose estimation model construction unit, which is used to construct a pose estimation model, comprising a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective view features from two adjacent monocular image frames and map them into a bird's-eye view feature map, i.e., a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map and obtain the position weights, validity weights, and feature point descriptors of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weights, validity weights, and descriptors to obtain feature point pairs with matching confidence; the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and use the BEV feature map to calculate the relative pose information between the two monocular image frames; The specific implementation process of the feature matching module includes: dividing the BEV feature map of the previous frame into multiple non-overlapping blocks, and calculating the position of the corresponding candidate feature points in each block according to the corresponding position weight, which is expressed as: Among them, i and j represent the index of the point, block n represents the nth block, represents the coordinates of point i in this block in the original BEV feature map, Represents the position weights of the feature points of the i-th point and the j-th point, Indicates the nth block n The position of the candidate feature points in ; The candidate feature point descriptors of each block in the previous frame and the dense descriptors of the next frame are normalized, and the similarity matrix is ​​calculated by tensor multiplication. The similarity matrix is ​​scaled using the temperature-weighted softmax function to calculate the candidate matching score of any two feature points: in, Represents the candidate feature point p in the previous frame c and feature point p in the next frame m The candidate matching score between and Represents the feature point descriptor D from the previous frame key1 The characteristic point p c The descriptor and the feature point descriptor D from the next frame key2 The characteristic point p m descriptor, τ represents the temperature parameter, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the feature point p k Descriptor of; Calculate the matching coordinates of the candidate feature points in the previous frame in the next frame based on the candidate matching scores: Among them, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set, Represents the kth feature point p in the next frame k Coordinates in the original BEV feature map, Represents block n The candidate feature point p in c With the kth feature point p k The candidate matching score between Represents the calculated matching point p in the next frame m Coordinates on the BEV characteristic map; Calculate the feature point pair (p c , p m ) represents the final confidence score: in, and Respectively represent p c and p m The validity weights on the respective BEV characteristic maps, Represents the final score of the feature point pair; The parameter optimization unit is used to construct a loss function based on the true pose value and the predicted pose information of the pose estimation model. After using the loss function to optimize the parameters of the pose estimation model, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images to realize monocular visual odometry.

8. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the one or more processors execute the executable code, they are used to implement the monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, the monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver according to any one of claims 1 to 6 is implemented.

Images