Knowledge Graph Construction from User Interaction Trails
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current knowledge engineering processes face challenges in structuring and extracting patterns from complex user activities due to a lack of consistent representations that connect heterogeneous data and manage complex user interactions, particularly in immersive environments involving multiple modalities.
Innovation Solution
A system that captures user interaction data as trails of actions over time, structures it using an ontology, and iteratively updates a knowledge graph, incorporating contextual information like audio and video, to facilitate machine learning and knowledge expansion, utilizing graph neural networks for improved representation and pattern detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If user interaction data is captured in immersive environments with multiple modalities, then the representation of user activities becomes more comprehensive, but the complexity of tracking and structuring this knowledge increases significantly
Solution Approach 1:
The patent segments user interaction data into discrete events with specific types (selection, annotation, explanation, etc.) and structures them as trails of actions. Each event is broken down into manageable components including event type, target object, and contextual information, making complex multi-modal data tractable for processing and storage in knowledge graphs.
Solution Approach 2:
The patent introduces an intermediary layer of event schemas and trail structures that mediate between raw multi-modal user interactions and the knowledge graph. This intermediary representation standardizes diverse interaction modalities into a unified format that can be systematically processed, reducing the complexity of handling comprehensive user activity data.
2Ease of operation
If consistent representations are created to connect heterogeneous data, then the structuring and processing of user input becomes easier, but the difficulty of managing and populating such datasets increases
Solution Approach 1:
The patent creates universal event schemas and trail structures that can handle multiple types of user interactions across different modalities through a single unified framework. This multi-functional representation system accommodates various data types (text, audio, video, annotations) using consistent event types and structures, simplifying processing while providing a scalable foundation for dataset expansion.
Solution Approach 2:
The patent implements dynamic trail structures that can adapt to different user interactions and data types while maintaining consistent event schemas. The representation system is designed to evolve and accommodate new interaction patterns and data modalities without requiring fundamental changes to the underlying structure, enabling easier dataset population and management as systems grow.
3Loss of information
If user interaction data is structured as trails of actions matched onto ontology entities, then knowledge extraction becomes more effective, but the initial setup and ontology alignment become more complex
Solution Approach 1:
The patent performs preliminary structuring of user interactions into standardized event trails with explicit schemas before ontology alignment. By pre-organizing raw interaction data into consistent event structures with defined types and attributes, the system prepares data in advance for more efficient ontology matching and knowledge extraction, reducing the complexity of the alignment process.
Solution Approach 2:
The patent transforms unstructured user interaction data into structured event representations by changing key parameters such as event type, target object, and contextual attributes. This parameter transformation creates a standardized format that aligns more easily with ontology entities, improving knowledge extraction effectiveness while making the alignment process more systematic and manageable.
4Quantity of substance
If the knowledge graph is iteratively updated with continuously captured interaction data, then the knowledge base expands and improves, but the computational resources and processing time increase
Solution Approach 1:
The patent implements iterative updates of the knowledge graph with user interaction trails at periodic intervals rather than continuous real-time processing. This periodic action allows the system to accumulate and process batches of interaction data efficiently, expanding the knowledge base over time while managing computational resources more effectively through scheduled processing cycles.
Data Source
AI summary
A method and system of creating a knowledge graph includes capturing information of a user interacting with given data, as user interaction data. The user interaction data is structured as a trail of actions over time. An ontology for a domain related to the user interaction data is received. Each action of the trail of actions is matched onto entities of the ontology. The knowledge graph is created based on the ontology having the matched actions.