Monocular Depth and Semantic Mapping for GNSS-Independent Drone Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for localizing unmanned aerial vehicles (UAVs) within environments often rely on Global Navigation Satellite Systems (GNSS), which can be unreliable in certain situations, such as GPS outages or signal interference.
Innovation Solution
A machine learning model is trained to process two-dimensional images from a UAV's camera, producing both depth and semantic images. These images are then aligned with reference depth and semantic data to determine the UAV's location within the environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If GNSS systems are used for UAV localization, then location information can be obtained, but the system becomes unreliable in certain situations such as GPS outages or signal interference
Solution Approach 1:
The patent introduces an intermediary system consisting of a camera and machine learning model that processes visual data from the environment to determine UAV location. This intermediary visual localization system acts as a mediator between the unreliable GNSS and the control system, providing alternative location information when GNSS fails while maintaining overall system functionality.
Solution Approach 2:
The patent changes the fundamental parameters used for localization from radio signal-based GNSS coordinates to visual-based semantic scene understanding and depth estimation. By transforming the localization approach from relying on external satellite signals to using onboard camera data processed through machine learning models, the system achieves reliability independence from GNSS signal availability.
2Measurement precision
If machine learning models with separate depth and semantics branches are used, then accurate localization can be achieved, but the device complexity increases
Solution Approach 1:
The machine learning model is segmented into distinct functional branches: a semantics branch for identifying environmental features and categories, and a depth branch for estimating distances and spatial relationships. This segmentation allows each branch to specialize in specific tasks, improving overall localization precision while enabling independent optimization and processing of different types of spatial information.
Solution Approach 2:
The machine learning model performs multiple functions simultaneously - semantic segmentation, depth estimation, and localization - using a unified architecture that processes the same input image data through different branches. This multi-functionality reduces the need for separate specialized systems while achieving high measurement precision through the complementary nature of semantic and depth information.
Data Source
AI summary
A method includes receiving a two-dimensional (2D) image captured by a camera on a unmanned aerial vehicle (UAV) and representative of an environment of the UAV. The method further includes applying a trained machine learning model to the 2D image to produce a semantic image of the environment and a depth image of the environment, where the semantic image comprises one or more semantic labels. The method additionally includes retrieving reference depth data representative of the environment, wherein the reference depth data includes reference semantic labels. The method also includes aligning the depth image of the environment with the reference depth data representative of the environment to determine a location of the UAV in the environment, where the aligning associates the one or more semantic labels from the semantic image with the reference semantic labels from the reference depth data.