Three-dimensional geospatial model

Abstract

A system uses models to relocalize a mobile device. The system accesses an input image of a scene in a real-world environment, where the image was captured by a mobile device. The system applies a two-dimensional (2D) foundation model to the input image. The 2D foundation model is trained to determine an image vector representing characteristics of the input image. The system accesses a map representation of the real-world environment, where the map representation includes visual data that describes the real-world environment. The system applies a three-dimensional (3D) geospatial model to the map representation and the image vector. The 3D geospatial model is configured to output 3D splats representing the real-world environment. The system determines a pose of a camera that captured the input image using the 3D splats.

Images