Scene Graph Generation via Appearance-Based Node Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating scene graphs are not scalable to large outdoor environments and fail to handle long-term object and scene reidentification due to their focus on spatial and contact relationships, making them unsuitable for loop closure in dynamic scenes.
Innovation Solution
A computer-implemented method that generates spatially consistent scene graphs by defining nodes based on sensor data, densely connecting them, predicting node and edge probabilities, and pruning connections to form a robust graph representation invariant to rotation and environmental changes, using a trained neural network to learn essential edges without explicit pose information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If current methods focus on spatial and contact relationships between objects, then scene graph construction works well indoors and in small environments, but the methods are not scalable to large outdoor environments and fail at long-term object reidentification
Solution Approach 1:
The patent changes the fundamental parameters used for scene graph construction from spatial/contact relationships to appearance-based features and graph matching metrics. This enables scalability to large outdoor environments while maintaining reliability in long-term object reidentification through appearance consistency tracking
Solution Approach 2:
Instead of building scene graphs from spatial relationships and then trying to match them, the patent inverts the approach by using appearance-based graph matching first and then refining with spatial constraints. This inversion enables better scalability and reidentification performance
2Measurement precision
If precise pose information is required to compute relationships between objects, then spatial accuracy is improved, but the system becomes more complex and less robust to environmental changes
Solution Approach 1:
The patent extracts and removes the requirement for precise pose information from the scene graph construction process. By using appearance-based features and graph matching instead of pose-dependent spatial relationships, the system achieves robustness to environmental changes without sacrificing measurement precision
Solution Approach 2:
The patent substitutes the mechanical/geometric system of pose-based spatial relationships with an appearance-based feature matching system. This replacement reduces complexity while maintaining accuracy through learned appearance representations
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented system and method of generating a graph representation of a scene comprising receiving sensor data representative of a perceived scene captured with a sensor; defining a plurality of nodes based on the received sensor data; creating a densely connected graph by connecting each node to a number of nearest neighbour nodes; predicting, for each pair of connected nodes of the densely connected graph, at least a node probability, wherein the node probability represents a probability that the pair of connected nodes represent the same object; and generating a graph representation of the perceived scene based at least on the densely connected graph, and the predicted node probability. The method may also be applied to two or more observations of a scene. The invention also discloses a training dataset, graph representation, use of the graph representation and a computer program.