Semantic Robot Navigation Using Spatial Graphs for Out-of-View Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional semantic navigation methods for service robots struggle to navigate to out-of-view objects in indoor environments due to a lack of effective decision models when multiple objects are present, and map-based methods are limited in new environments, lacking global interpretability and view of the environment.
Innovation Solution
A method and system using a spatial graph and trajectory history, employing a pretrained Graph Neural Network (GNN) to compute embeddings for visible regions, calculate similarity scores, and select optimal actions based on spatial relationships and valid trajectory paths, enabling the robot to find out-of-view objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If map-based navigation methods are used, then global information is provided for the agent, but the methods are limited in new environments for obtaining maps
Solution Approach 1:
The system pre-trains a Graph Neural Network on spatial relationship graphs constructed from training scenes before deployment. This preliminary action allows the model to learn spatial patterns and object co-occurrence relationships in advance, enabling it to generalize to new environments without requiring pre-built maps or extensive exploration, thus resolving the contradiction between providing global information and adapting to new environments.
2Productivity
If deep reinforcement learning navigation method is used, then satisfactory performance is achieved, but it lacks interpretability and global view of the environment
Solution Approach 1:
The patent introduces a spatial relationship graph as an intermediary structure that explicitly represents spatial relationships and object co-occurrence patterns in the environment. This graph serves as a mediator between the reinforcement learning agent and the environment, providing interpretability by visualizing spatial structures and maintaining a global view while preserving the performance benefits of deep reinforcement learning.
3Adaptability or versatility
If conventional GNNs based model is used, then prior knowledge is embedded into Deep Reinforcement Learning framework, but it lacked a concrete decision model when two or more objects are in same scene
Solution Approach 1:
The patent applies local quality by computing distinct embeddings for different regions and objects within the same scene using the Graph Neural Network. Each object and region receives specialized embedding representations based on its local spatial relationships and co-occurrence patterns, enabling the model to make concrete decisions for multiple objects in the same scene while maintaining prior knowledge embedding.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure provides a model for semantic navigation for service robots to find out-of-view objects in an indoor environment. Initially, the system receives a target object to be reached by the mobile robot in the indoor environment. Further, a current location of the mobile robot is identified by a localization technique. An embedding corresponding to each of a plurality of visible regions is computed using a pretrained Graph Neural Network GNN. The GNN is pretrained using a trajectory data and a spatial relationship graph associated with the indoor environment. Further, a similarity score is computed for each of the plurality of visible regions based on the corresponding embedding using a scoring technique. An optimal visible region is identified by comparing the similarity score. Finally, a next action to be taken by the mobile robot selected from a plurality of actions based on the optimal visible region.