Knowledge Graph Data Structuring for Information Overload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In today's interconnected markets, market agents face 'information overload' which can lead to poor investment decisions and missed opportunities, necessitating advanced tools to discover and process relevant information effectively, particularly in complex systems where distinguishing hidden order from randomness is challenging.
Innovation Solution
A data management system structures data in a knowledge graph by processing both structured and unstructured data, identifying concepts and connections between them using semantic and topic vectors, and generating new concepts when necessary, to derive insights and facilitate information processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is processed using traditional methods, then processing speed is maintained, but information overload leads to poor decision-making and missed opportunities
Solution Approach 1:
The patent segments information processing into multiple specialized modules: ingestion module for data collection, concept assignment module for semantic categorization, new concept generation module for identifying novel patterns, and connection determination module for relationship mapping. This segmentation allows each module to handle specific aspects of information processing efficiently, preventing overload while maintaining comprehensive analysis capability
Solution Approach 2:
The patent introduces knowledge graphs as an intermediary structure between raw data and decision-making processes. The knowledge graph serves as a mediator that organizes unstructured and semi-structured data into structured relationships, enabling efficient information retrieval and analysis without overwhelming the decision-making system
2Reliability
If complex analysis tools are introduced to process information, then decision quality improves, but system complexity increases
Solution Approach 1:
The patent creates a multi-functional data management system where a single integrated platform performs diverse functions: data ingestion from multiple sources, semantic concept assignment, new concept generation, connection determination, and knowledge graph construction. This universal system reduces overall complexity by consolidating multiple specialized tools into one cohesive architecture that handles the entire information processing pipeline
Solution Approach 2:
The patent transforms flat, unstructured data into multi-dimensional knowledge graphs that add semantic, topical, and relational dimensions. This dimensional transformation allows complex information relationships to be visualized and analyzed in structured space, making complex systems more manageable and interpretable without increasing operational complexity
3Loss of information
If traditional data storage methods are used, then system simplicity is maintained, but connection discovery between concepts is limited
Solution Approach 1:
The patent performs preliminary organization of data into knowledge graphs before analysis operations. By pre-structuring unstructured and semi-structured data with assigned concepts, semantic vectors, and established connections during the ingestion phase, the system prepares information in advance for efficient pattern recognition and connection discovery, reducing the need for complex real-time processing
Data Source
AI summary
Disclosed are systems, methods, and non-transitory computer-readable media for structuring data in a knowledge graph. A data management system determines known concepts that are related to a data snippet. The data management system determines cosine similarity values indicating an intrinsic similarity between the data snippet and the known concepts, as well as pertinence values indicating a measure of topical similarity between the data snippet and the known concepts. The data management system determines, based on the cosine similarity values and the pertinence values, that the data snippet is related to a first known concept, and in response, assigns a concept identifier for the first known concept to the data snippet. Score indicating a strength of connection between the concepts added to the knowledge graph are determined and used to derive insights between the concepts.