Monocular vision odometer method and system based on bird's-eye view representation and differentiable weighted Procrustes solver

Abstract

The invention discloses a monocular visual odometer method and system based on aerial view representation and differentiable weighted Procrustes solver, and the method comprises the steps: constructing a pose estimation model which can carry out the extraction and matching of key points in an aerial view, and carries out the relative pose estimation through an interpretable weighted Procrustes solver; the method comprises the following steps: constructing a loss function based on pose information, optimizing parameters of a pose estimation model by using the loss function, and performing pose information estimation between two frames of monocular images by using the parameter-optimized pose estimation model to realize the monocular visual odometer, so that the monocular visual odometer is superior in the aspect of reducing scale drift; and stable performance is shown under different weather and illumination conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of the combination of computer vision and autonomous driving, and specifically relates to a monocular visual odometer method and system based on bird's-eye view representation and a differentiable weighted Procrustes solver. Background Art

[0002] Monocular Visual Odometry (MVO) is a positioning technology widely used on ground vehicles, such as logistics robots and self-driving cars, due to its low hardware cost and convenient deployment. As an important supplement to the Global Positioning System (GPS) and inertial navigation system, MVO ensures the stable operation of unmanned systems and provides a more economical alternative to LiDAR. However, there is a key problem with MVO technology, that is, in long-distance applications, it is prone to scale drift due to the lack of depth information, which limits its scope of application.

[0003] Traditional MVO methods, such as direct methods, feature-based methods, and semi-direct methods, usually deal with depth uncertainty by using the first few frames as a global reference to establish relative scales during the initialization phase. However, the global scale of this method depends on the initial motion state, so its performance will drop significantly when the initial motion is very different from the subsequent motion. In addition, due to calibration errors, feature matching errors, and motion blur, these traditional methods will suffer from severe scale drift after long-term operation, limiting their application in long-distance tasks.

[0004] In order to solve the above problems, some MVO methods based on deep learning have emerged in recent years. These methods use deep learning networks to reduce scale drift. Early deep learning methods directly extract features from perspective images and regress poses, but their performance is relatively poor. Later methods combine the modeling ability of deep learning with the interpretability of traditional methods, train optical flow prediction and depth estimation networks through supervised learning, and combine traditional multi-view geometry and back-end optimization to achieve better performance. However, although self-supervised learning methods reduce scale drift to a certain extent, their performance is still inferior to supervised learning methods due to scale ambiguity.

[0005] Recently, with the prevalence of the ground plane assumption in autonomous driving scenarios, Bird's-Eye View (BEV) representation methods have begun to gain widespread attention in the fields of computer vision and autonomous driving. Compared with perspective images, BEV representation methods have shown better performance in 3D detection and segmentation tasks. However, existing BEV odometry methods usually rely on semantic segmentation tasks, which not only increases the annotation cost but also limits the possibility of end-to-end pose supervision optimization. Therefore, it is not clear whether the scale drift reduction effect of existing BEV representation methods in odometry tasks is due to the unified scale structure of the BEV representation itself or the auxiliary information introduced by the segmentation task. Summary of the invention

[0006] In view of the above, the purpose of the present invention is to provide a monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver, which uses bird's-eye view representation and differentiable weighted Procrustes solver to perform end-to-end training using only pose supervision. Through experimental verification on multiple data sets, the method of the present invention performs well in reducing scale drift and exhibits stable performance under different weather and lighting conditions.

[0007] To achieve the above-mentioned object of the invention, the embodiment provides a monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver, comprising the following steps:

[0008] Constructing a pose estimation model, which includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective view features from two monocular image frames at adjacent moments and map them into a bird's-eye view feature map, namely, a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map, and obtain the position weights, validity weights, and feature point descriptors of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weights, validity weights, and descriptors to obtain feature point pairs with matching confidence; the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, and use the BEV feature map to calculate the relative pose information between the two monocular image frames through differentiable weighted singular value decomposition;

[0009] A loss function is constructed based on the true pose value and the predicted pose information of the pose estimation model. After the loss function is used to optimize the parameters of the pose estimation model, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images, realizing a high-precision, low-scale drift monocular visual odometer.

[0010] Preferably, the visual BEV encoder includes a feature extraction submodule, a camera parameter auxiliary encoding submodule, a depth distribution prediction submodule, and a feature mapping submodule;

[0011] The feature extraction submodule is used to extract a multi-scale perspective feature map from a monocular image;

[0012] The camera parameter auxiliary encoding submodule is used to encode the camera parameters and obtain a weighted feature map by weighting the camera parameter encoding and the perspective view feature map through channel attention;

[0013] The depth distribution prediction submodule is used to predict the depth distribution characteristics of the features through a deep network based on the weighted feature map;

[0014] The feature mapping submodule is used to perform cone mapping on the features fused from the weighted feature map and the depth distribution feature to the space around the vehicle coordinate system, and then obtain the BEV feature map based on the voxel pooling layer.

[0015] Preferably, the feature point extraction module adopts a U-Net style encoder-decoder network structure;

[0016] The encoder downsamples the BEV feature map to a low resolution to capture multi-scale feature information through its multi-layer convolution and downsampling operations;

[0017] The decoder comprises two parallel branches, which are used to predict the position weight of feature points and the validity weight of feature points respectively. Each branch comprises an upsampling stage with the same number of encoder layers to restore the feature map from low resolution to the original resolution. At the same time, the feature maps of each downsampling stage of the encoder are upsampled to the original size and spliced ​​in the channel dimension to form a multi-dimensional feature point descriptor map with the same resolution as the original resolution.

[0018] Preferably, the feature matching module divides the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and performs feature point matching in each block based on the position weight, validity weight and descriptor to obtain feature point pairs with matching confidence, including:

[0019] The BEV feature map of the previous frame is divided into multiple non-overlapping blocks, and the position of the corresponding candidate feature point in each block is calculated according to the corresponding position weight, which is expressed as:

[0020]

[0021] Among them, i and j represent the index of the point, block n represents the nth block, represents the coordinates of point i in this block in the original BEV feature map, Represents the position weights of the feature points of the i-th point and the j-th point, Indicates the nth block n The position of the candidate feature points in ;

[0022] The candidate feature point descriptors of each block in the previous frame and the dense descriptors of the next frame are normalized, and the similarity matrix is ​​calculated by tensor multiplication. The similarity matrix is ​​scaled using the temperature-weighted softmax function to calculate the candidate matching scores of any two feature points:

[0023]

[0024] in, Represents the candidate feature point p in the previous frame c and feature point p in the next frame m The candidate matching score between and Represent the feature point descriptors D from the previous frame respectively key1 The characteristic point p c The descriptor of the feature point in the next frame is key2 The characteristic point p m descriptor, τ represents the temperature parameter, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the feature point p k Descriptor of;

[0025] Calculate the matching coordinates of the candidate feature points in the previous frame in the next frame based on the candidate matching scores:

[0026]

[0027] Among them, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the kth feature point p in the next frame k The coordinates in the original BEV feature map, Represents block n The candidate feature point p in c With the kth feature point p k The candidate matching score between Represents the calculated matching point p in the next frame m Coordinates on the BEV characteristic map;

[0028] Calculate the feature point pair (p c , p m) represents the final confidence score:

[0029]

[0030] in, and Respectively represent p c and p m The validity weights on the respective BEV feature maps, Represents the final score of the feature point pair.

[0031] Preferably, the differentiable weighted Procrustes solver uses the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and uses the BEV feature map to calculate the relative pose information between the two monocular images, including:

[0032] First, the position coordinates of the matching point pair are converted into the position coordinates of the actual size, and the weighted centroid of each feature point in the position coordinates of the actual size is calculated based on the matching score;

[0033] Based on the centralized matching point pairs determined by the weighted centroid, the weighted covariance matrix is ​​calculated, and the covariance matrix is ​​subjected to singular value decomposition to calculate the rotation matrix and translation vector to form the relative pose information between the two monocular images.

[0034] Preferably, constructing a loss function based on the true pose value and the predicted pose information of the pose estimation model includes: constructing a loss function based on the translation error and rotation error between the predicted pose and the true pose, expressed as:

[0035] L pose =|t pred,x -t gt,x |+|t pred,y -t gt,y |+α·|θ pred -θ gt |

[0036] Among them, θ pred and θ gt Respectively represent the yaw angle in the predicted rotation matrix and the true value of the yaw angle, t pred,x and t pred,y Respectively represent the components of the predicted translation vector in the x and y directions, t gt,x and t gt,y They represent the components of the x and y directions in the true value of the translation vector respectively, and α represents the weighting factor.

[0037] Preferably, when optimizing the pose estimation model parameters using the loss function, a phased training strategy is used to optimize the model parameters: first, global feature point pre-training is performed to stabilize the optimization process, and then a guided translation convergence strategy is used to improve the pose estimation accuracy, specifically:

[0038] In the initial stage of training, the validity weights of all feature points are set to 1, the predicted values ​​of the rotation matrix calculated using singular value decomposition are replaced by the true values ​​of the rotation matrix, and the weighted centroid of the subsequent frame is converted to the coordinate system of the previous frame to calculate the translation vector. This can guide the convergence direction of the translation vector to the correct direction and avoid the partial network of the translation vector receiving the wrong gradient direction during optimization due to the wrong prediction of the rotation matrix.

[0039] To achieve the above-mentioned object of the invention, an embodiment of the present invention further provides a monocular visual odometer system based on bird's-eye view representation and a differentiable weighted Procrustes solver, comprising:

[0040] A pose estimation model construction unit, which is used to construct a pose estimation model, and includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective view features from two frames of monocular images at adjacent moments and map them into a bird's-eye view feature map, namely, a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map, and obtain the position weight, validity weight, and feature point descriptor of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weight, validity weight and descriptor to obtain feature point pairs with matching confidence; the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and use the BEV feature map to calculate the relative pose information between the two frames of monocular images;

[0041] The parameter optimization unit is used to construct a loss function based on the true value of the pose and the predicted pose information of the pose estimation model, and after optimizing the parameters of the pose estimation model using the loss function, the pose estimation model after parameter optimization is used to estimate the pose information between two frames of monocular images to realize the monocular visual odometer.

[0042] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver.

[0043] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-mentioned monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention simplifies the pose estimation from 6 degrees of freedom to 3 degrees of freedom by taking advantage of the BEV representation, and reduces the scale drift by using the unified metric scale characteristics of the BEV feature map. Compared with traditional methods that can only obtain relative scales or learning methods that require complex design and multi-task supervision, the present invention can achieve end-to-end training by relying only on pose supervision, which significantly simplifies the system design while ensuring the accuracy of motion estimation.

[0046] The present invention adopts a unique phased training strategy to improve network performance. In the first two training cycles, the validity weights of all feature points are set to 1 through the global feature point pre-training strategy to ensure feature learning in the global range of the BEV feature map; in the first N training cycles, a guided translation convergence strategy is used to align the center of mass of the second frame feature map using the true value rotation matrix, effectively reducing the translation error. This training strategy significantly improves the network convergence performance and pose estimation accuracy. The loss function uses the weighted L1 loss of translation and rotation errors, and balances the contribution of the two types of errors through the weight coefficient.

[0047] The experimental results of the present invention on NCLT, Oxford and KITTI datasets show that it has significant advantages over existing methods: it achieves an average relative translation error of 8.67% and 6.92% on NCLT and Oxford datasets, and an average relative rotation error of 3.53° / 100m and 1.33° / 100m respectively; it also maintains competitive performance on the KITTI dataset with significant elevation changes. In terms of real-time performance, the present invention runs at 16FPS on the RTX 4090 GPU, with a GPU utilization rate of 45% and a memory usage of only 7.5GB, showing good computing efficiency.

[0048] In addition, the interpretable backend design proposed in this invention, including modules such as feature point extraction and matching based on BEV feature graph, weight calculation, and differentiable weighted Procrustes solution, not only provides a clear pose estimation process, but also guides the network to learn better feature representation through structured feedback. This design improves accuracy while also enhancing the system's generalization ability in different scenarios.

[0049] In summary, the method of the present invention achieves high-precision motion estimation and robust pose prediction by introducing feature point matching, weight calculation, differentiable weighted Procrustes solver, and carefully designed pose supervision and optimization strategies. Compared with the prior art, the present invention not only effectively reduces the scale drift problem in traditional monocular visual odometers, but also improves the real-time performance and computational efficiency of the system by simplifying the calculation process. These innovations enable the method of the present invention to show significant advantages in application scenarios such as autonomous driving and robot autonomous navigation that require high accuracy and real-time performance, and provide a highly efficient, accurate and widely applicable technical solution for related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 It is a flow chart of a monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver provided in an embodiment;

[0052] Figure 2 is a structural schematic diagram of a posture estimation model provided in an embodiment;

[0053] Figure 3 is a visualization diagram of trajectory and key point matching provided by an embodiment;

[0054] Figure 4 is a comparison of the absolute trajectory error (ATE) index of different methods provided in the embodiment on two data sets;

[0055] Figure 5 It is a structural schematic diagram of a monocular visual odometer system based on a bird's-eye view representation and a differentiable weighted Procrustes solver provided in an embodiment. DETAILED DESCRIPTION

[0056] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0057] The invention concept of the present invention is to solve the various key problems faced by the existing monocular visual odometer technology in practical applications, so as to meet the needs of ground vehicles such as autonomous driving and logistics robots for high-precision and low-cost positioning systems. Specifically, the present invention provides a monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver, which is improved and optimized in the following aspects:

[0058] 1. Reduce the scale drift problem: MVO technology generally has the problem of scale drift in long-distance applications due to the lack of depth information, which limits its extensiveness in practical applications. This invention introduces a unified metric scale bird's-eye view (BEV) representation, which enables the system to more effectively handle scale drift in long-distance movement and significantly improve the accuracy and reliability of positioning;

[0059] 2. Enhanced robustness in complex environments: The performance of traditional MVO methods often drops significantly when facing complex environments such as lighting changes, low-texture areas, and dynamic scenes. To address this problem, this paper designs a new pose estimation model based on BEV representation, combined with a weighted Procrustes solver, and effectively improves the robustness of the system in complex environments by extracting and matching key points on the BEV feature map, enabling it to work stably in a variety of complex scenarios;

[0060] 3. Improve generalization ability: Existing deep learning-based MVO methods have limited generalization ability in new scenarios, mainly because these methods usually lack interpretability. This invention adopts an interpretable weighted Procrustes solver and an end-to-end training method, so that the system can not only perform well in the training data, but also better generalize to unseen scenarios, ensuring the wide applicability of the system in different application environments;

[0061] 4. Simplify implementation and reduce costs: The existing BEV representation odometer method relies on additional tasks such as semantic segmentation, resulting in high annotation costs and complex implementation processes. This invention avoids the reliance on additional tasks by directly using BEV representation for pose estimation, reduces the complexity and cost of system implementation, and simplifies the entire development and deployment process;

[0062] 5. Improve overall performance: This invention not only innovates in reducing scale drift and enhancing robustness, but also achieves higher overall system performance, including higher positioning accuracy, faster computing speed, and more stable long-distance tracking capabilities, by optimizing the algorithm framework and training strategy. The superiority of this invention in scale consistency and pose accuracy has been demonstrated through verification on multiple data sets.

[0063] In summary, the present invention aims to comprehensively solve the problems existing in the prior art by introducing a BEV representation with a unified metric scale and a weighted Procrustes solver, and to provide a more efficient, reliable and easy-to-implement solution to meet the needs of autonomous driving and other ground vehicles for high-precision, low-cost positioning systems.

[0064] Based on the above invention concept, Figure 1 As shown, the embodiment provides a monocular visual odometer method based on bird's-eye view representation and a differentiable weighted Procrustes solver, comprising the following steps:

[0065] S1 builds a pose estimation model that can extract and match key points in the bird's-eye view (BEV) and perform relative pose estimation through an interpretable weighted Procrustes solver.

[0066] like Figure 2 As shown, the pose estimation model provided by the embodiment includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver.

[0067] In the embodiment, the visual BEV encoder (i.e., PV-BEV encoder) is used to extract multi-scale perspective features from two monocular images at adjacent moments and map them into feature maps from a bird's-eye view. Specifically, the LSS (Lift, Splat, Shoot) architecture is used to extract three-dimensional spatial features from perspective images and map them to BEV space. The LSS architecture predicts the depth distribution to achieve scale-consistent BEV projection, combines camera parameters (i.e., camera intrinsics and extrinsics), enhances interpretability, and supports multi-camera settings with accurate extrinsics.

[0068] Specifically, the visual BEV encoder includes a feature extraction submodule, a camera parameter auxiliary encoding submodule, a depth distribution prediction submodule, and a feature mapping submodule. The feature extraction submodule is used to extract multi-scale perspective feature maps from monocular images. By using ResNet-50 as the backbone network, multi-scale features can be extracted from the input monocular image. The extracted features are fused through the Feature Pyramid Network (FPN) to generate a multi-scale perspective (PV) feature map, namely the PV feature map.

[0069] The camera parameter auxiliary encoding submodule is used to encode the camera parameters and obtain the weighted feature map by weighting the camera parameter encoding and the perspective feature map through channel attention (SE). Specifically, the camera parameters are encoded through a multi-layer perceptron (MLP), and the encoded camera parameters are multiplied element-wise with the PV feature map to generate an enhanced weighted feature map to ensure the spatial consistency of the image features. The weighted feature map is then passed through a convolutional layer to generate the final weighted feature map.

[0070] The depth distribution prediction submodule is used to predict the depth distribution features of the features through a deep network based on the weighted feature map, where the deep network includes a deformable convolutional network DCN, a void space pyramid pooling ASPP, and a convolutional layer stacked residual block Res Block. The depth distribution map is optimized through pose supervision, so that the depth estimation process is closely integrated with the pose estimation. The generation of the depth distribution feature includes weighting the distribution information of each pixel of the feature map at different depths to form a 4D depth distribution feature map.

[0071] The feature mapping submodule is used to perform frustum mapping of the features fused with the weighted feature map and the depth distribution features to the space around the vehicle coordinate system, and then obtain the BEV feature map based on the voxel pooling layer. Specifically, the 4D depth distribution features are mapped to the BEV space through the frustum projection technology to generate the BEV feature map. Then, the information in the Z-axis direction is compressed through the voxel pooling technology, and finally a BEV representation with multi-dimensional features is formed. This operation ensures that the image features can be accurately mapped to the BEV space of the vehicle coordinate system.

[0072] In the embodiment, the feature point extraction module is used to extract representative feature points from the BEV feature map, and obtain the position weight, validity weight, and feature point descriptor map of the feature points. Specifically, a U-Net-style encoder-decoder network structure is adopted; wherein the encoder downsamples the BEV feature map to a low resolution to capture multi-scale feature information through its multi-layer convolution and downsampling operations. Specifically, the encoder path includes four stages of progressive downsampling to reduce the feature map from the original resolution to a more compact representation.

[0073] The decoder consists of two parallel branches, which are used to predict the position weights W of the feature points. pos and the validity weight W of the feature point valid , each branch contains the same number of upsampling stages as the encoder layers, restoring the feature map from low resolution to the original resolution; at the same time, the feature maps of each downsampling stage of the encoder are upsampled to the original size and concatenated in the channel dimension to form a multi-dimensional feature point descriptor map D with the same resolution as the original key .

[0074] In the embodiment, the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weight, validity weight and descriptor to obtain feature point pairs with matching confidence to estimate the relative motion of the vehicle. Specifically, the BEV feature map of the previous frame is first divided into multiple non-overlapping blocks, and the position of the corresponding candidate feature point in each block is calculated according to the corresponding position weight, which is expressed as:

[0075]

[0076] Among them, i and j represent the index of the point, block n represents the nth block, represents the coordinates of point i in this block in the original BEV feature map, Represents the position weights of the feature points of the i-th point and the j-th point, Indicates the nth block n The position of the candidate feature points in ;

[0077] The candidate feature point descriptors of each block in the previous frame and the dense descriptors of the next frame are normalized. It should be noted that since the next frame is full-size, the corresponding descriptors composed of all feature points are called dense descriptors. The similarity matrix is ​​calculated by tensor multiplication. In order to avoid interference from long-distance non-matching feature points, the system sets a mask range according to the typical restrictions of vehicle movement, and returns the similarity values ​​beyond the range to zero. The similarity matrix is ​​scaled using the temperature-weighted softmax function to calculate the candidate matching scores of the two feature points:

[0078]

[0079] in, Represents the candidate feature point p in the previous frame c and feature point p in the next frame m The candidate matching score between and Respectively represent the feature point descriptors D from the previous frame key1 The characteristic point p c The descriptor of the feature point descriptor D from the next frame key2 The characteristic point p m descriptor, τ represents the temperature parameter, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the feature point p k Descriptor of;

[0080] The matching coordinates of the candidate feature points in the previous frame in the subsequent frame are calculated based on the candidate matching scores, that is, the matching coordinates are obtained by weighted summation of the pixel coordinates in the feature point descriptor of the subsequent frame:

[0081]

[0082] Among them, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the kth feature point p in the next frame k The coordinates in the original BEV feature map, Represents block n The candidate feature point p in c With the kth feature point p k The candidate matching score between Represents the calculated matching point p in the next frame m Coordinates on the BEV characteristic map;

[0083] Calculate the matching point pair (p c , p m ) represents the final score of confidence. The final score comprehensively considers the similarity and validity weight of the feature point descriptors to ensure the accuracy and reliability of the matching results. The matching weight reflects the similarity of the feature point descriptors and their confidence at the matching position:

[0084]

[0085] in, and Respectively represent p c and p m The validity weights on the respective BEV feature maps, Represents the final score of the feature point pair.

[0086] In the embodiment, the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, and to calculate the relative pose information between two frames of monocular images using the differentiable weighted singular value decomposition and the BEV feature map, specifically calculating the optimal rigid transformation between the two sets of key point pairs, including the rotation matrix and the translation vector. The solver is based on the weighted singular value decomposition (Weighted SVD) algorithm, supports gradient back propagation, is suitable for deep learning training, and maintains interpretability. The specific process is:

[0087] First, the matching point pair (p c , p m )'s position coordinates Convert to actual size position coordinates The weighted centroid of each feature point in the actual size position coordinates will be calculated based on the matching score The coordinates of the centralized matching point pairs are obtained by subtracting the centroid of each feature point pair from their respective coordinates. and The formula is:

[0088]

[0089] in, is the final score of the matching point pair, n represents a value in the matching score, and hw is the number of partition blocks;

[0090] Then, based on the coordinates of the centered matching point pairs, the weighted covariance matrix W is calculated cov , and perform singular value decomposition on the covariance matrix to calculate the rotation matrix R and translation vector t to form the pose information.

[0091]

[0092] R=U·diag(1,1,det(UV T ))·V T

[0093]

[0094] The whole process supports gradient back propagation, allowing the weighted Procrustes solver to be effectively optimized during deep learning training.

[0095] S2, constructs a loss function based on the true value of the pose and the predicted pose information of the pose estimation model, and uses the loss function to optimize the parameters of the pose estimation model. Then, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images to realize monocular visual odometer.

[0096] In order to optimize the pose estimation of the model, the present invention designs a loss function based on the pose truth value and the predicted pose information of the pose estimation model. The loss function includes the L1 loss of the rotation matrix and the translation vector, which is used to measure the gap between the predicted pose parameters and the true value. In order to balance the errors of rotation and translation, a weighting factor α is introduced into the loss function, so that the network can effectively reduce the errors of the two during the optimization process. The loss function is expressed as:

[0097] L pose =|t pred,x -t gt,x |+|t pred,y -t gt,y |+α·|θ pred -θgt |

[0098] Among them, θ pred and θ gt Respectively represent the yaw angle in the predicted rotation matrix and the true value of the yaw angle, t pred,x and t pred,y Respectively represent the components of the predicted translation vector in the x and y directions, t gt,x and t gt,y They respectively represent the components of the x and y directions in the true value of the translation vector. In the embodiment, α can be set to 10 to balance the errors of rotation and translation, because the rotation error will affect the calculation of translation.

[0099] When using the loss function to optimize the parameters of the pose estimation model, a phased training strategy is used to optimize the model parameters: first, global feature point pre-training is performed to stabilize the optimization process, and then a guided translation convergence strategy is used to improve the pose estimation accuracy, so as to accelerate convergence and improve network performance. Specifically:

[0100] Strategy 1, global feature point pre-training: In the initial stage of training, the validity weights of all feature points are set to 1 to ensure that the network pays attention to each position in the BEV feature map evenly. This method helps the stability of the optimization process, allowing the network to learn the differences between key points evenly across the entire feature map.

[0101] Strategy 2: Guided translation convergence: In the first N (e.g., 10) cycles of training, the inaccurate prediction of the rotation matrix may lead to large errors in the translation vector, thus hindering the convergence of the network. To this end, in the first N training cycles, the real value of the rotation matrix is ​​used to replace the predicted value of the rotation matrix calculated using singular value decomposition, and the weighted centroid of the subsequent frame is converted to the coordinate system of the previous frame to calculate the translation direction; subsequently, the network gradually transitions to using the predicted rotation matrix for calculation to improve the accuracy of translation estimation.

[0102] The embodiment also provides an experimental example to verify the effect of the above method, specifically:

[0103] The present invention has been comprehensively experimentally verified on multiple public datasets to evaluate its performance in different environments and conditions. These datasets include:

[0104] NCLT dataset: This dataset has high jitter, significant illumination changes and motion blur, and is suitable for evaluating the performance of algorithms under complex visual conditions. The training sequences (2012-01-08, 2012-02-04, 2013-04-05) cover different weather conditions at noon and in the afternoon, and the test sequences (2012-02-02, 2012-02-19, 2012-03-17, 2012-08-20) contain unseen time, weather and season features to evaluate the generalization ability of the algorithm.

[0105] Oxford Radar RobotCar Dataset: This dataset contains complex urban driving scenes with dynamic backgrounds and a large number and size of traffic participants, making it suitable as a challenging urban testbed. The training sequences (19-01-11-13, 19-01-14-14, 19-01-15-14) represent typical conditions, while the test sequences (19-01-11-12, 19-01-15-13, 19-01-16-14, 19-01-17-12) contain strong lighting conditions not seen during training.

[0106] KITTIOdometry dataset: This dataset is widely used for odometry comparison, and many methods provide pre-trained models and published results to facilitate consistency benchmarking. The training sequences (00-08) and test sequences (09-10) contain significant vertical variations, reaching 38 meters in sequence 09 and 24 meters in sequence 10. These variations pose a challenge to 3DOF odometry methods that assume a flat ground plane.

[0107] In all datasets, the present invention uniformly applies the same hyperparameter settings to avoid tuning for specific datasets. The BEV grid is set to 256×256 with a resolution of 0.4 meters to adapt to the data characteristics of NCLT (low speed and high jitter), the urban complexity of Oxford, and the high-speed open scenes of KITTI.

[0108] In terms of baseline setting, the proposed method is compared with three types of baseline methods (traditional methods from a PV perspective, learning-based methods from a PV perspective, and comparative methods from a BEV perspective). Each type of baseline method evaluates specific aspects of the proposed method design:

[0109] Perspective View (PV)-based baselines: including the traditional method ORB-SLAM3 and the learning-based methods DeepVO, TartanVO, DF-VO, DROID-SLAM, and DPVO. These methods perform odometer estimation under perspective view.

[0110] Bird's-Eye View (BEV)-based baselines: We designed three BEV baseline methods to study the impact of interpretable components (e.g., feature point matching module, weighted Procrustes solver):

[0111] (a) BEV+CNNs+MLPs: uses convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) for direct feature extraction and pose regression without keypoint-based matching;

[0112] (b) BEV+global correlation+MLPs: global correlation is calculated on the BEV feature map for motion estimation, but lacks spatial constraints;

[0113] (c) BEV+local correlation+MLPs: By focusing on local areas and adding spatial constraints, the accuracy of inter-frame estimation is improved.

[0114] The goals of the comparison are twofold: first, to verify the advantage of the unified metric-scaling BEV representation in simplifying pose estimation (from 6 DOF to 3 DOF); second, to evaluate whether the interpretable design can guide the intermediate layers to better anchor the scale and capture key features, thereby further improving the overall performance and reducing scale drift.

[0115] Experimental parameter settings: To simulate different speeds and motions, we randomly sample frame pairs and oversample frames with significant rotations. The test trajectory is calculated by accumulating relative poses in a fixed interval without any post-processing. Training is performed on an NVIDIA RTX 4090 GPU using the Adam optimizer with an initial learning rate of 1×10 -4 , decaying by 0.95 per epoch. The entire training process takes about 72 hours.

[0116] Evaluation strategy: The accuracy of the odometer is evaluated by RTE (mean translation RMSE), RRE (mean rotation RMSE) and ATE (mean translation error). RTE and RRE are calculated in the range of 100 to 800 meters, respectively, while ATE measures the average translation error between the predicted pose and the ground truth pose. In addition, a composite indicator (log2(SE(3) / Sim(3))) is designed to quantify the error change before and after scale alignment and evaluate the performance of different methods in terms of scale consistency. To ensure fairness, for MVO methods that lack absolute scale or rely on pre-trained models, the first 10 meters of ground truth data are used for scale alignment, which is in line with the actual MVO usage scenario.

[0117] The ablation experiment results show that the design of each module has a significant impact on the overall performance:

[0118] Feature point validity weights: Adding validity weights to key points to capture their global importance, combined with position weights, significantly improved the accuracy of key point matching. Removing these weights significantly degraded the performance, demonstrating the importance of combining local and global weights.

[0119] Training strategy: The use of global feature point pre-training and guided translation convergence strategy helps convergence in the early stages and improves the RTE and RRE indicators.

[0120] BEV hyperparameters: The resolution and size of the BEV grid are adjusted for different datasets, with the grid refined to accommodate the low speed and high jitter of NCLT and the grid coarsened to accommodate the high speed and stable motion of Oxford, further verifying the importance of selecting appropriate BEV hyperparameters according to the environmental conditions.

[0121] Real-time performance: On the KITTI dataset, the NVIDIA RTX 4090 GPU was used for evaluation, with a batch size of 1 and an input size of (3, 1216, 384). This method achieved a processing speed of 16 frames per second (FPS), a GPU utilization of 45%, and a memory usage of 7.5GB (including data transmission delay). This result shows that this method has the potential for real-time visual odometer applications.

[0122] In summary, through extensive experiments on multiple data sets, the method of the present invention has shown superior performance in various indicators of odometry estimation, especially in terms of scale consistency and complex scene processing capabilities, which are significantly better than existing baseline methods. Ablation experiments further verify the effectiveness of the design of each module, and the runtime performance shows that it has the potential for practical application. These results fully demonstrate the innovation and practicality of the present invention in the field of visual odometry.

[0123] Attached Figure 3 It is a visualization result of trajectory and feature point matching of the monocular visual odometer method based on bird's-eye view representation on the NCLT and Oxford datasets. This figure shows the comparison between the method and the ground truth trajectory, and different scenarios such as straight, right turn and left turn are evaluated on the NCLT and Oxford datasets respectively. The results show that the method can achieve high path accuracy in long-distance and complex turning scenarios without any post-processing steps. In addition, through key point matching and weighted Procrustes solver, the method can effectively identify and utilize geometric patterns in BEV feature maps, which improves the structure and interpretability of pose estimation. It demonstrates the robustness and accuracy of the present invention in long-distance and complex turning scenarios, and verifies the ability of the system to handle complex trajectories in different environments. Specifically includes the following key parts:

[0124] 1. Trajectory comparison chart (upper part):

[0125] The left side shows the trajectory comparison results in the NCLT dataset, showing the comparison between the trajectory predicted by BEV-DWPVO and the actual reference trajectory (GT). Figure 3 The A, B, and C positions marked in the figure correspond to the straight section, right turn scenario, and left turn scenario, respectively. These positions are used to further demonstrate the operating results of the system under different situations.

[0126] The right side shows the trajectory comparison results in the Oxford dataset, which also shows the comparison between the trajectory predicted by the system and the actual reference trajectory. Figure 3 The D, E, and F positions marked in the figure correspond to the straight-ahead, right-turn, and left-turn scenarios, respectively.

[0127] 2. Feature point matching visualization (lower part):

[0128] The image below shows the feature point matching results of the selected position in the above trajectory comparison image. The background is the BEV feature map of the previous multiple frames, the red points represent the key points of the previous multiple frames, the green points represent the key points matched in the next multiple frames, and the white lines represent the matching relationship between the key points.

[0129] Figure 3 A and D in the figure show the matching of feature points in the straight-ahead scene. Figure 3 Figures B, E, C, and F in the figure show the matching conditions in the right turn and left turn scenes. It can be observed that in the complex turning scene, the matching relationship of the feature points is more complicated, but overall the method of the present invention can still accurately perform key point matching and motion estimation.

[0130] Attached Figure 4 This is a comparison chart of the absolute trajectory error (ATE) index of different methods on the NCLT and Oxford datasets. It shows the ATE performance of each method under SE(3) and Sim(3) alignment, as well as the comparison of the combined index Log2(SE(3) / Sim(3)). These results illustrate the differences in scale drift among different methods, further verify the superiority of the present invention in different complex scenarios, and demonstrate its innovative contribution in reducing scale drift and improving motion estimation accuracy.

[0131] 1. ATE index comparison (left part of the figure): This part shows the ATE values ​​of various methods on the NCLT and Oxford datasets. The horizontal axis represents the ATE value, and the vertical axis lists the various methods compared. Figure 4 The blue and orange bars represent the ATE results of SE (3) and Sim (3) alignment on the NCLT and Oxford datasets, respectively. The method of the present invention shows the smallest ATE value, indicating that its accuracy is better than other methods under different alignment modes.

[0132] 2. Scale drift comparison (right part of the figure): This part shows the box plot of the combined indicator Log2(SE(3) / Sim(3)), which is used to quantify the performance of different methods on scale drift. The horizontal axis represents the Log2(SE(3) / Sim(3)) value. The closer it is to 0, the better the scale consistency. Figure 4 It can be seen that the method of the present invention has the smallest scale drift on the two data sets and shows the best scale consistency.

[0133] 3. Illustration: The data shown in the figure show that the present invention can maintain highly stable pose estimation accuracy under different environmental conditions by combining BEV representation and differentiable weighted Procrustes solver. Compared with other methods (such as ORB-SLAM3, DeepVO, TartanVO, DF-VO, DROID-SLAM, DPVO, etc.), the robustness and generalization ability of the present invention in complex environments have been significantly improved, especially in dealing with scale drift problems. Due to the use of a solver with better interpretability, this method has better generalization than other methods that also use BEV representation, and the average performance on multiple test sequences shows better scale consistency and accuracy.

[0134] The monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver provided above is mainly used for real-time positioning and navigation tasks of ground vehicles such as self-driving cars and logistics robots, and is particularly suitable for 3-degree-of-freedom (3DoF) motion estimation scenarios that require high precision and low drift in complex environments. The core advantage of the present invention is that it achieves high-precision motion estimation through feature point matching and posture supervision, has excellent generalization and robustness, and can adapt to various complex scenarios.

[0135] In the application of self-driving cars, vehicles need to perform accurate real-time positioning and navigation in complex environments such as highways and urban roads. Traditional visual odometer systems may cause error accumulation due to long-term operation, thus affecting the driving safety and path planning accuracy of the vehicle. The present invention has the following advantages in such scenarios:

[0136] 1. Accurate positioning during long-term driving: Autonomous vehicles need to continuously obtain accurate posture information during long-term driving. This invention effectively reduces the scale drift problem caused by monocular vision by combining feature point matching from a bird's-eye view with posture supervision, thereby maintaining highly accurate positioning data in a long time series.

[0137] 2. Real-time response in dynamic environments: In the face of ever-changing traffic conditions (such as other vehicles merging into other lanes, the sudden appearance of pedestrians, etc.), the present invention can adjust the vehicle's driving path in real time through high-precision posture estimation. Compared with traditional systems, the present invention can make fast and accurate decisions in dynamic environments without the need for additional depth information, ensuring the driving safety of autonomous vehicles and the reliability of path planning.

[0138] In the application of logistics robots, when logistics robots perform cargo handling in warehouses or automation tasks in factories, accurate motion estimation is the key to successfully completing the tasks. The present invention exhibits the following advantages in such application scenarios:

[0139] 1. High-precision positioning in complex environments: When logistics robots are operating in large warehouses, they often need to maintain stable paths and positioning accuracy in narrow passages and complex environments. By optimizing feature point matching and depth distribution estimation, the present invention can provide continuous high-precision pose data in such complex planar environments, effectively reducing the drift problem common in traditional monocular vision systems.

[0140] 2. Low-cost, efficient monocular solution: Due to the need for cost control, many logistics robots can only be equipped with a monocular camera. The present invention can make full use of the field of view of the monocular camera, and achieve efficient motion estimation and path planning through accurate feature point matching and posture supervision. This allows the logistics robot to operate stably in a complex logistics environment when using only a monocular camera.

[0141] In the application of plane navigation vehicles, such as airport guide vehicles and indoor delivery vehicles, the vehicles need to run for a long time in a plane environment while maintaining high-precision motion estimation. The present invention is particularly suitable for such application scenarios due to its excellent scale consistency and robustness:

[0142] 1. Maintaining path accuracy during long-term operation: When a plane navigation vehicle is operating in an environment such as an airport or a shopping mall, it is necessary to maintain the accuracy of the predetermined path for a long time. Through the method of the present invention, the vehicle can maintain path accuracy during long-term operation, avoid deviation caused by error accumulation, and can navigate stably even in a complex plane environment.

[0143] 2. Ability to cope with dynamically changing environments: In busy public places, planar navigation vehicles need to cope with the sudden appearance of pedestrians and obstacles in real time. The method of the present invention enables the vehicle to quickly adjust the driving path, avoid dynamic obstacles and continue to perform the task through real-time high-precision posture estimation, ensuring efficient completion of the task.

[0144] In summary, the method of the present invention achieves high-precision, low-drift motion estimation in a long time and complex environment by combining feature point matching from a bird's-eye view and posture supervision. It has excellent performance in applications such as autonomous driving, logistics robots, and planar navigation vehicles, has strong generalization and robustness, and can effectively meet the real-time positioning and navigation needs of various ground vehicles.

[0145] like Figure 5 As shown, the embodiment also provides a monocular visual odometer system based on bird's-eye view representation and differentiable weighted Procrustes solver, including a pose estimation model construction unit 51 and a parameter optimization and application unit 52, wherein the pose estimation model construction unit 51 is used to construct a pose estimation model, which can utilize key point extraction and matching in the bird's-eye view, and perform relative pose estimation through an interpretable weighted Procrustes solver; the parameter optimization and application unit 52 is used to construct a loss function based on pose information, and after optimizing the pose estimation model parameters using the loss function, use the pose estimation model after parameter optimization to estimate the pose information between two frames of monocular images, so as to realize monocular visual odometer.

[0146] Based on the same inventive concept, an embodiment further provides a computing device, including a memory and one or more processors, wherein executable code is stored in the memory, and when the one or more processors execute the executable code, the method for implementing the monocular visual odometer based on bird's-eye view representation and differentiable weighted Procrustes solver is specifically provided, which includes the following steps:

[0147] S1, build a pose estimation model, which can extract and match key points in the bird's-eye view and perform relative pose estimation through an interpretable weighted Procrustes solver;

[0148] S2, constructing a loss function based on the pose information, and using the loss function to optimize the pose estimation model parameters, and then using the pose estimation model with optimized parameters to estimate the pose information between two frames of monocular images to realize monocular visual odometer.

[0149] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A monocular visual odometry method based on bird's-eye view representation and differentiable weighted Procrustes solver, characterized in that: The following steps are involved: Constructing a pose estimation model, which includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective view features from two monocular image frames at adjacent moments and map them into a bird's-eye view feature map, namely, a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map, and obtain the position weights, validity weights, and feature point descriptors of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and perform feature point matching in each block based on the position weights, validity weights, and descriptors to obtain feature point pairs with matching confidence; the differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences, and use the BEV feature map to calculate the relative pose information between the two monocular image frames through differentiable weighted singular value decomposition; A loss function is constructed based on the true pose value and the predicted pose information of the pose estimation model. After the loss function is used to optimize the parameters of the pose estimation model, the pose estimation model with optimized parameters is used to estimate the pose information between two frames of monocular images to realize monocular visual odometer.

2. The monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The visual BEV encoder includes a feature extraction submodule, a camera parameter auxiliary encoding submodule, a depth distribution prediction submodule, and a feature mapping submodule; The feature extraction submodule is used to extract a multi-scale perspective feature map from a monocular image; The camera parameter auxiliary encoding submodule is used to encode the camera parameters and obtain a weighted feature map by weighting the camera parameter encoding and the perspective view feature map through channel attention; The depth distribution prediction submodule is used to predict the depth distribution characteristics of the features through a deep network based on the weighted feature map; The feature mapping submodule is used to perform cone mapping on the features fused from the weighted feature map and the depth distribution feature to the space around the vehicle coordinate system, and then obtain the BEV feature map based on the voxel pooling layer.

3. The monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The feature point extraction module adopts a U-Net style encoder-decoder network structure; The encoder downsamples the BEV feature map to a low resolution to capture multi-scale feature information through its multi-layer convolution and downsampling operations; The decoder comprises two parallel branches, which are used to predict the position weight of feature points and the validity weight of feature points respectively. Each branch comprises an upsampling stage with the same number of encoder layers to restore the feature map from low resolution to the original resolution. At the same time, the feature maps of each downsampling stage of the encoder are upsampled to the original size and spliced ​​in the channel dimension to form a multi-dimensional feature point descriptor map with the same resolution as the original resolution.

4. The monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The feature matching module divides the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and performs feature point matching in each block based on the position weight, validity weight and descriptor to obtain feature point pairs with matching confidence, including: The BEV feature map of the previous frame is divided into multiple non-overlapping blocks, and the position of the corresponding candidate feature point in each block is calculated according to the corresponding position weight, which is expressed as: Among them, i and j represent the index of the point, block n represents the nth block, represents the coordinates of point i in this block in the original BEV feature map, Represents the position weights of the feature points of the i-th point and the j-th point, Indicates the nth block n The position of the candidate feature points in ; The candidate feature point descriptors of each block in the previous frame and the dense descriptors of the next frame are normalized, and the similarity matrix is ​​calculated by tensor multiplication. The similarity matrix is ​​scaled using the temperature-weighted softmax function to calculate the candidate matching scores of any two feature points: in, Represents the candidate feature point p in the previous frame c and feature point p in the next frame m The candidate matching score between and Respectively represent the feature point descriptors D from the previous frame key1 The characteristic point p c The descriptor of the feature point descriptor D from the next frame key2 The characteristic point p m descriptor, τ represents the temperature parameter, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set of Represents the feature point p k Descriptor of; Calculate the matching coordinates of the candidate feature points in the previous frame in the next frame based on the candidate matching scores: Among them, k∈dense2 represents the feature point p k Feature point descriptor D from the next frame key2 The point set, Represents the kth feature point p in the next frame k The coordinates in the original BEV feature map, Represents block n The candidate feature point p in c With the kth feature point p k The candidate matching score between Represents the calculated matching point p in the next frame m Coordinates on the BEV characteristic map; Calculate the feature point pair (p c , p m ) represents the final confidence score: in, and Respectively represent p c and p m The validity weights on the respective BEV feature maps, Represents the final score of the feature point pair.

5. The monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The differentiable weighted Procrustes solver uses the matched feature point pairs and their confidences, through differentiable weighted singular value decomposition, and uses the BEV feature map to calculate the relative pose information between two monocular images, including: First, the position coordinates of the matching point pair are converted into the position coordinates of the actual size, and the weighted centroid of each feature point in the position coordinates of the actual size is calculated based on the matching score; Based on the centralized matching point pairs determined by the weighted centroid, the weighted covariance matrix is ​​calculated, and the covariance matrix is ​​subjected to singular value decomposition to calculate the rotation matrix and translation vector to form the relative pose information between the two monocular images.

6. The monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: The loss function is constructed based on the true pose value and the predicted pose information of the pose estimation model, including: the loss function is constructed based on the translation error and rotation error between the predicted pose and the true pose, which is expressed as: L pose =|t pred,x -t gt,x |+|t pred,y -t gt,y |+α·|θ pred -θ gt | Among them, θ pred and θ gt Respectively represent the yaw angle in the predicted rotation matrix and the true value of the yaw angle, t pred,x and t pred,y Respectively represent the components of the predicted translation vector in the x and y directions, t gt,x and t gt,y They represent the components of the x and y directions in the true value of the translation vector respectively, and α represents the weighting factor.

7. The monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver according to claim 1, characterized in that: When using the loss function to optimize the parameters of the pose estimation model, a phased training strategy is used to optimize the model parameters: first, global feature point pre-training is performed to stabilize the optimization process, and then a guided translation convergence strategy is used to improve the accuracy of pose estimation, specifically: In the initial stage of training, the validity weights of all feature points are set to 1, the predicted value of the rotation matrix calculated using singular value decomposition is replaced by the true value of the rotation matrix, and the weighted centroid of the subsequent frame is transformed into the coordinate system of the previous frame to calculate the translation vector.

8. A monocular visual odometry system based on bird's-eye view representation and differentiable weighted Procrustes solver, characterized in that: include: The pose estimation model construction unit is used to construct a pose estimation model, which includes a visual BEV encoder, a feature point extraction module, a feature matching module, and a differentiable weighted Procrustes solver, wherein the visual BEV encoder is used to extract multi-scale perspective view features from two monocular images at adjacent moments and map them into a bird's-eye view feature map, namely, a BEV feature map; the feature point extraction module is used to extract feature points from the BEV feature map and obtain the position weights, validity weights, and feature point descriptors of the feature points; the feature matching module is used to divide the BEV feature map corresponding to the previous frame into multiple non-overlapping blocks, and in each Feature point matching is performed within the block based on position weights, validity weights and descriptors to obtain feature point pairs with matching confidence. A differentiable weighted Procrustes solver is used to use the matched feature point pairs and their confidences to calculate the relative pose information between two monocular images using the BEV feature map through differentiable weighted singular value decomposition. A parameter optimization unit is used to construct a loss function based on the true pose value and the predicted pose information of the pose estimation model, and after optimizing the pose estimation model parameters using the loss function, the pose estimation model after parameter optimization is used to estimate the pose information between the two monocular images to realize monocular visual odometer.

9. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the one or more processors execute the executable code, they are used to implement the monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the monocular visual odometer method based on bird's-eye view representation and differentiable weighted Procrustes solver described in any one of claims 1 to 7 is implemented.

Images