Scale-aware monocular localization and mapping

Abstract

Disclosed is an image processing device comprising a processor configured to estimate the scale of image features by the steps of: processing multiple images of a scene by means of a first trained model to identify features in the images and to estimate the depths of those features in the images; processing the multiple images by a second trained model to estimate a scaling for the images; and estimating the scales of the features by adjusting the estimated depths in dependence on the estimated scaling. A method for training an image processing model is also disclosed.

Description

technical field [0001] The present invention relates to solving the problem of scale ambiguity when projecting images captured by a digital video camera. Background technique [0002] Digital cameras capture images by light falling on a digital sensor that converts the collected light into digital data, known as raw data. The raw data is then processed to create a raster image formed as a field of pixels, where each pixel represents the brightness and color received at an area of ​​the digital sensor representing an area of ​​the captured image. The field of pixels can then be displayed on the screen so it can be seen by a human. In this way, projection of the captured image is achieved. Both raw and processed data can be stored and transferred to another device, allowing further reproduction of captured images as and when required. [0003] If the image is a scene (could be an outdoor scene or an indoor scene), the image usually includes many objects or features. In som...

AI Technical Summary

Problems solved by technology

This suffers from ill-posed problems, so the results are inaccurate and cannot be generalized to previously unseen scenarios

[0007] (2) Additional sensors, such as a second camera, active depth sensor, or inertial measurement unit (IMU), enable the system to directly recover scale-aware measurements, but this adds additional hardware and requires fusing information from different modalities

The method relies on knowledge about objects, which means that the method can easily fail when faced with unknown objects or known objects that do not match previously seen objects of that type, for example, if the method is familiar with trees such as apple trees, are known to be about 3 meters (10 feet) tall, but it is difficult to identify a sequoia that may be about 100 meters (350 feet) tall as a tree

Images