Explainable Scene Clustering With Commonsense Knowledge Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scene clustering methods rely solely on visual features, neglecting the rich semantic information available from commonsense knowledge, which limits their ability to provide meaningful explanations and effective clustering in domains like autonomous driving.
Innovation Solution
Integrate commonsense knowledge graphs with scene graphs to enhance clustering by determining relations and embeddings that capture both visual and semantic properties, enabling explainable clustering through rule induction and semantic search.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If scene clustering methods rely solely on visual features, then the clustering process is simple and fast, but the ability to provide meaningful explanations and effective clustering in domains like autonomous driving is limited
Solution Approach 1:
The patent merges visual feature extraction with commonsense knowledge graph integration to create a hybrid clustering system. The scene graph captures visual relationships while the commonsense knowledge graph provides semantic context, combining both approaches to achieve meaningful explanations without sacrificing clustering effectiveness
Solution Approach 2:
The patent introduces knowledge graphs as intermediary structures that bridge visual features and semantic meanings. The knowledge graph serves as a mediator between low-level visual data and high-level semantic concepts, enabling the system to explain cluster assignments through semantic relationships
2Reliability
If commonsense knowledge graphs are integrated with scene graphs to enhance clustering, then meaningful explanations are provided, but the system complexity increases
Solution Approach 1:
The patent segments the knowledge representation into two distinct graphs: the scene graph for visual relationships and the commonsense knowledge graph for semantic relationships. This segmentation allows each graph to be optimized for its specific purpose while working together to provide reliable clustering results
Solution Approach 2:
The knowledge graphs serve multiple functions: they represent semantic relationships, provide clustering labels, generate explanations, and enable semantic search. This multi-functionality reduces the need for separate systems and justifies the integration complexity through versatile utility
3Loss of information
If rules are determined to map first relations to second relations for explainable clustering, then explainability is improved, but the computational requirements and time increase
Solution Approach 1:
The patent performs preliminary actions by pre-computing and storing rules that map first relations to second relations during system initialization or offline processing. This allows the actual clustering and explanation generation to proceed faster using pre-derived rules rather than computing explanations in real-time
Data Source
AI summary
A device and a computer implemented method for explainable clustering of a scene. The method includes determining a first relation that relates a first object class to a second object class, wherein determining the first relation includes determining, depending on the first object class and the second object class, a pair of entities in a first knowledge graph, in particular a commonsense knowledge graph, that represents information about a domain, wherein the pair of entities is related with the first relation in the first knowledge graph, determining a cluster in that the scene belongs depending on the scene and depending on other scenes, determining a second relation that relates the scene with the cluster depending on at least one feature of digital image data representing the scene, determining a rule that maps the first relation to the second relation.