Continuous learning visual mileage estimation method for plateau mountain land
By utilizing online continuous learning and a batch image database with diverse strategies, and employing an unsupervised monocular depth estimation network, the problem of poor adaptability and robustness of visual odometry in high-altitude and mountainous environments was solved, achieving efficient visual odometry in complex environments.
Patent Information
- Application Number
- CN202311657098.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-05
- Publication Date
- 2026-03-31
AI Technical Summary
Existing visual odometry methods are difficult to adapt to different scenarios and environmental changes, especially in unknown and variable environments such as high-altitude and mountainous areas. Furthermore, learning-based methods are sensitive to interference from factors such as changes in lighting and occlusion, and have poor robustness.
An unsupervised monocular depth estimation network is used for continuous online learning. By constructing a batch image database with diverse strategies, the network performs self-optimization and learning using image sequences. Combined with a real-time visual odometry inference module, it provides real-time visual odometry estimation.
It improves the adaptability and robustness of visual odometry in high-altitude and mountainous environments, enabling it to self-optimize and learn in complex environments, adapt to environmental changes, reduce computing resource requirements, and possess high efficiency and scalability.
Smart Images
Figure CN121767436A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual SLAM technology, and more specifically, to a continuous learning visual odometry estimation method for high-altitude mountainous terrain. Background Technology
[0002] Simultaneous Localization and Mapping (SLAM) technology is widely used in Augmented Reality (AR), drones, autonomous driving, robotics, smart homes, and other fields. Visual Odometry (VO) is a crucial component of SLAM systems, estimating the motion of a mobile device from a sequence of images captured by a camera and then reconstructing its trajectory. However, most existing visual odometry methods are limited to predefined target domains. For example, feature-matching-based methods require prior feature extraction and modeling of the environment, making them difficult to adapt to different scenes and environmental changes. Furthermore, learning-based visual odometry methods typically require large amounts of labeled data for training and have poor generalization ability to unknown environments, making them susceptible to interference from factors such as lighting changes, occlusion, and dynamic objects. Summary of the Invention
[0003] To overcome the shortcomings of existing technologies, this invention provides a continuous learning visual odometry estimation method for plateau and mountainous areas, which can improve the adaptability of visual odometry estimation to unknown and variable environments such as plateau and mountainous areas.
[0004] The technical solution of this invention is as follows: A continuous learning visual odometry estimation method for plateau and mountainous areas, comprising:
[0005] S1. Acquire image sequences to obtain multiple consecutive frames of images and IMU data for supervision;
[0006] S2. Construct a batch image database and select samples based on a diversity strategy;
[0007] S3, Online Continuous Learning Module, uses an unsupervised monocular depth estimation network as a branch task to continuously learn pose transformations between sequential images online;
[0008] S4, the real-time visual odometry inference application module, continuously calculates visual odometry based on network output, providing odometry input for downstream tasks.
[0009] Furthermore, the image sequences are acquired via vehicle-mounted or robot-mounted cameras, including RGB cameras, infrared cameras, or binocular cameras.
[0010] Furthermore, the image sequence consists of the image at the current moment and the images of the previous and next N frames, where N is 1.
[0011] Furthermore, in step 1, when acquiring image sequences, image sequences are acquired at different speeds, in different directions, and under different environments. When acquiring data, image sequences in which the camera is clearly in motion are selected for training.
[0012] Furthermore, a diversity strategy is employed to screen samples in the batch image database. This strategy uses cosine similarity as a representation of the images. Specifically, an image is taken from the current image sequence and its cosine similarity is calculated with an image from another image sequence in the batch image database. If the maximum cosine similarity is still less than a threshold, the sample is considered to have strong differences, and this sample is included in the batch image database. If the number of images exceeds the threshold, the cosine similarity between an image sampled from each image sequence and other images is calculated. The cosine similarities are accumulated, and the image with the maximum accumulated similarity is identified. This image is considered to have low differences from other images and is excluded from the batch image database.
[0013] Furthermore, the online continuous learning module performs iterative learning of the odometry network through an unsupervised deep estimation network structure.
[0014] Furthermore, the unsupervised depth estimation network architecture includes:
[0015] The monocular depth estimation network structure predicts the depth value of each pixel in the source image at time s using the monocular image as input.
[0016] The pose estimation network structure estimates the camera pose change between two consecutive frames of images taken at time s and time t.
[0017] Furthermore, the online continuous learning module performs iterative learning of the odometry network through an unsupervised deep estimation network structure, including the following steps:
[0018] (1) Using the current image, the inverse matrix of camera intrinsics, and the predicted depth of the current image. It can restore all pixels in the current image to a point cloud in a three-dimensional coordinate system, that is...
[0019] (2) Transform the recovered point cloud in the 3D coordinate system using the camera pose transformation matrix. The coordinates of the point cloud at time t can be obtained. Then, through perspective projection transformation, the 3D point cloud at time t is projected onto the phase plane to reconstruct the image at time t.
[0020] (3) Loss is calculated by reconstructing the image and the original image at time t. The loss includes two parts: reconstruction loss and smoothing loss, which are used to control the similarity of the reconstructed image and the clarity of the image edge, respectively. The network weights can be continuously updated by iterating in reverse according to the loss to achieve the training effect.
[0021] Furthermore, the online continuous learning module performs iterative learning of the odometry network using unsupervised deep estimation methods, specifically including:
[0022] Get the images at the current time and the previous and next times;
[0023] Perform diversity calculations to determine whether the data needs to be included in the batch image database;
[0024] Sampling from a batch image database, the depth and camera motion from time t-2 to time t-1 and from time t-1 to time t are estimated;
[0025] Calculate the loss and update the network through reverse iteration;
[0026] Repeat the above steps until the loss converges.
[0027] Furthermore, the real-time visual odometry inference application module outputs the camera motion from time t-1 to time t in the camera motion estimation network as visual odometry.
[0028] According to the above-described solution, the beneficial effects of this invention are as follows:
[0029] (1) The present invention provides a continuous learning visual odometry estimation method for plateau and mountainous areas. It uses an unsupervised monocular depth estimation network for online continuous learning, which can effectively utilize image sequence data for self-optimization and learning. This makes the method more efficient in handling visual odometry tasks in complex environments such as plateau and mountainous areas. Secondly, the method includes a real-time visual odometry inference application module, which can continuously calculate visual odometry based on network output and provide odometry input for downstream tasks in real time.
[0030] (2) The present invention provides a continuous learning visual odometry estimation method for plateau and mountainous areas. The method uses a diversity-based sample selection strategy to construct a batch image database, which can improve the robustness of the network when dealing with complex environments and lighting changes. Secondly, through the online continuous learning module, the method can continuously learn the pose transformation between sequential images, further improving the adaptability to environmental changes. In addition, the method adopts an unsupervised monocular depth estimation network as a branch task, which can be applied to different types of image sequence data and different pose estimation tasks. This makes the method not only applicable to plateau and mountainous environments, but also applicable to other fields and complex scenarios.
[0031] (3) The present invention provides a continuous learning visual odometry estimation method for plateau and mountainous areas. Through an online continuous learning module, it continuously learns new pose transformation knowledge and information, and continuously updates and optimizes the network model. This makes the method highly scalable and adaptable, and can be continuously improved and expanded as the environment and task change. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0033] Figure 1 This is a flowchart of a continuous learning visual odometry estimation method for plateau and mountainous areas according to an embodiment of the present invention;
[0034] Figure 2 This is a schematic diagram of the unsupervised depth estimation network structure in an embodiment of the present invention;
[0035] Figure 3 This is a schematic diagram of the pose estimation network in an embodiment of the present invention;
[0036] Figure 4 This is a schematic diagram of the structural composition of a continuous learning visual odometry estimation method for plateau and mountainous areas according to an embodiment of the present invention. Detailed Implementation
[0037] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of the present invention by way of example, but should not be used to limit the scope of the present invention, that is, the present invention is not limited to the described embodiments.
[0038] To better understand the present invention, it will be further described below with reference to the accompanying drawings and embodiments:
[0039] See Figures 1-4 As shown, this embodiment of the invention provides a continuous learning visual odometry estimation method for plateau and mountainous terrain, including:
[0040] S1. Acquire image sequences to obtain multiple consecutive frames of images and IMU data for supervision;
[0041] S2. Construct a batch image database and select samples based on a diversity strategy;
[0042] S3, Online Continuous Learning Module, uses an unsupervised monocular depth estimation network as a branch task to continuously learn pose transformations between sequential images online;
[0043] S4, the real-time visual odometry inference application module, continuously calculates visual odometry based on network output, providing odometry input for downstream tasks.
[0044] This invention provides a continuous learning visual odometry estimation method for high-altitude mountainous environments. By acquiring image sequences and obtaining multiple consecutive frames of images, along with supervised IMU data, this method can obtain richer environmental information, thereby improving the accuracy and robustness of visual odometry. Simultaneously, using IMU data for supervision provides more accurate pose estimation results, assisting visual odometry. Especially in high-altitude mountainous environments, this method, through continuous learning and updating of model parameters, can better adapt to changes in terrain and environment, further improving the accuracy and robustness of visual odometry. Secondly, this method employs an unsupervised monocular depth estimation network as a branch task, which can effectively utilize image sequence data for self-optimization and learning, further improving the accuracy and robustness of visual odometry.
[0045] This invention provides a continuous learning visual odometry estimation method for high-altitude mountainous environments. Through an online continuous learning module, it can learn and update in real time using acquired image sequence data, continuously optimizing model parameters and improving the model's adaptability and robustness. Simultaneously, the real-time visual odometry inference application module can quickly calculate visual odometry based on network output and provide real-time odometry input for downstream tasks, meeting real-time requirements. Using an unsupervised monocular depth estimation network as a branch task, it can effectively utilize image sequence data for self-optimization and learning, thus requiring less labeled data and computational resources, resulting in lower cost and computational resource consumption. Furthermore, its efficiency and low cost allow for wide application in various high-altitude mountainous environments. The online continuous learning module can continuously learn new pose transformation knowledge and information, constantly updating and optimizing the network model, making the method highly scalable and adaptable, continuously improving and expanding with changes in environment and task. In addition, the method can further expand its application scope by introducing new branch tasks to meet more practical needs.
[0046] In this embodiment, the image sequence is acquired via a vehicle-mounted or robot-mounted camera, which includes, but is not limited to, RGB cameras, infrared cameras, or binocular cameras. Specifically, the camera can be flexibly configured and optimized according to different camera types and parameters. For example, using an RGB camera can acquire rich color information, helping the method to better understand and utilize environmental features; while using an infrared camera can acquire images under low visible light conditions, thereby expanding the application range of the method; and a binocular camera can perform accurate stereo matching and depth estimation, further improving the motion estimation accuracy of the method in complex terrain.
[0047] In this embodiment, the image sequence is the image at the current moment and the images of the previous and next N frames, where N is 1.
[0048] In this embodiment, in step 1, when acquiring image sequences, image sequences are acquired at different speeds, in different directions, and under different environments. When acquiring data, image sequences in which the camera is obviously in motion are selected for training.
[0049] In this embodiment, a diversity strategy is used to screen samples in the batch image database. The diversity strategy selects the cosine similarity of images as a representation. Specifically, an image is taken from the current image sequence and its cosine similarity is calculated with an image from another image sequence in the batch image database. If the maximum cosine similarity is still less than the threshold, the sample is considered to have strong differences, and the sample is included in the batch image database. If the number of images exceeds the threshold, the cosine similarity between an image sampled from each image sequence and other images is calculated. The cosine similarities are accumulated, and the image with the maximum accumulated similarity is found. This image is considered to have low differences from other images and is excluded from the batch image database.
[0050] In this embodiment, the online continuous learning module performs iterative learning of the odometry network through an unsupervised deep estimation network structure.
[0051] In this embodiment, the unsupervised depth estimation network structure includes:
[0052] The monocular depth estimation network structure predicts the depth value of each pixel in the source image at time s using the monocular image as input.
[0053] The pose estimation network structure estimates the camera pose change between two consecutive frames of images taken at time s and time t.
[0054] More specifically, the online continuous learning module performs iterative learning of the odometry network through an unsupervised deep estimation network structure, including the following steps:
[0055] (1) Using the current image, the inverse matrix of camera intrinsics, and the predicted depth of the current image. It can restore all pixels in the current image to a point cloud in a three-dimensional coordinate system, that is...
[0056] (2) Transform the recovered point cloud in the 3D coordinate system using the camera pose transformation matrix. The coordinates of the point cloud at time t can be obtained. Then, through perspective projection transformation, the 3D point cloud at time t is projected onto the phase plane to reconstruct the image at time t.
[0057] (3) Loss is calculated by reconstructing the image and the original image at time t. The loss includes two parts: reconstruction loss and smoothing loss, which are used to control the similarity of the reconstructed image and the clarity of the image edge, respectively. The network weights can be continuously updated by iterating in reverse according to the loss to achieve the training effect.
[0058] In this embodiment, the online continuous learning module performs iterative learning of the odometry network using an unsupervised deep estimation method, specifically including:
[0059] Get the images at the current time and the previous and next times;
[0060] Perform diversity calculations to determine whether the data needs to be included in the batch image database;
[0061] Sampling from a batch image database, the depth and camera motion from time t-2 to time t-1 and from time t-1 to time t are estimated;
[0062] Calculate the loss and update the network through reverse iteration;
[0063] Repeat the above steps until the loss converges.
[0064] Among them, the real-time visual odometry inference application module outputs the camera motion from time t-1 to time t in the camera motion estimation network as visual odometry.
[0065] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.
[0066] The present invention has been described above with reference to the accompanying drawings. Obviously, the implementation of the present invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other situations without modification, are all within the protection scope of the present invention.
Claims
1. A continuous learning visual odometry estimation method for high-altitude mountainous areas, characterized in that, include: S1. Acquire image sequences to obtain multiple consecutive frames of images and IMU data for supervision; S2. Construct a batch image database and select samples based on a diversity strategy; S3, Online Continuous Learning Module, uses an unsupervised monocular depth estimation network as a branch task to continuously learn pose transformations between sequential images online; S4, the real-time visual odometry inference application module, continuously calculates visual odometry based on network output, providing odometry input for downstream tasks.
2. The continuous learning visual odometry estimation method for plateau and mountainous terrain as described in claim 1, characterized in that: The image sequences are acquired via vehicle-mounted or robot-mounted cameras, including RGB cameras, infrared cameras, or binocular cameras.
3. The continuous learning visual odometry estimation method for plateau and mountainous areas as described in claim 1, characterized in that: The image sequence consists of the image at the current moment and the images of the previous and next N frames, where N is 1.
4. The continuous learning visual odometry estimation method for plateau and mountainous areas as described in claim 1, characterized in that: In step 1, when acquiring image sequences, image sequences are acquired at different speeds, in different directions, and under different environments. When acquiring data, image sequences in which the camera is clearly in motion are selected for training.
5. The continuous learning visual odometry estimation method for plateau and mountainous areas as described in claim 1, characterized in that: A diversity strategy is used to select samples for the batch image database. The diversity strategy selects cosine similarity as the representation of images. Specifically, an image is taken from the current image sequence and its cosine similarity is calculated with an image from another image sequence in the batch image database. If the maximum cosine similarity is still less than the threshold, the sample is considered to have strong differences and is included in the batch image database. If the number of images exceeds the threshold, the cosine similarity between an image sampled from each image sequence and other images is calculated. The cosine similarities are accumulated, and the image with the maximum accumulated similarity is found. This image is considered to have low differences from other images and is excluded from the batch image database.
6. The continuous learning visual odometry estimation method for plateau and mountainous areas as described in claim 1, characterized in that: The online continuous learning module performs iterative learning of the odometry network through an unsupervised deep estimation network structure.
7. The continuous learning visual odometry estimation method for plateau and mountainous areas as described in claim 6, characterized in that: Unsupervised depth estimation network architectures include: The monocular depth estimation network structure predicts the depth value of each pixel in the source image at time s using the monocular image as input. The pose estimation network structure estimates the camera pose change between two consecutive frames of images taken at time s and time t.
8. The continuous learning visual odometry estimation method for plateau and mountainous terrain as described in claim 7, characterized in that: The online continuous learning module performs iterative learning of the odometry network using an unsupervised deep estimation network structure, including the following steps: (1) Using the current image, the inverse matrix of camera intrinsics, and the predicted depth of the current image. It can restore all pixels in the current image to a point cloud in a three-dimensional coordinate system, that is... (2) Transform the recovered point cloud in the 3D coordinate system using the camera pose transformation matrix. The coordinates of the point cloud at time t can be obtained. Then, through perspective projection transformation, the 3D point cloud at time t is projected onto the phase plane to reconstruct the image at time t. (3) Loss is calculated by reconstructing the image and the original image at time t. The loss includes two parts: reconstruction loss and smoothing loss, which are used to control the similarity of the reconstructed image and the clarity of the image edge, respectively. The network weights can be continuously updated by iterating in reverse according to the loss to achieve the training effect.
9. The continuous learning visual odometry estimation method for plateau and mountainous areas as described in claim 1, characterized in that: The online continuous learning module uses unsupervised deep estimation methods for iterative learning of the odometry network, specifically including: Get the images at the current time and the previous and next times; Perform diversity calculations to determine whether the data needs to be included in the batch image database; Sampling from a batch image database, the depth and camera motion from time t-2 to time t-1 and from time t-1 to time t are estimated; Calculate the loss and update the network through reverse iteration; Repeat the above steps until the loss converges.
10. The continuous learning visual odometry estimation method for plateau and mountainous areas as described in claim 9, characterized in that: The real-time visual odometry inference application module outputs the camera motion from time t-1 to time t in the camera motion estimation network as visual odometry.