Data Folding for Entity-Event Storage Contradiction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing databases fail to efficiently preprocess and store large volumes of varied data from multiple sources, leading to inefficient analysis and retrieval, especially for complex queries, and lack support for user-defined domain grouping.
Innovation Solution
A system that preprocesses data by extracting entities, events, and domain-specific associations, creating data lattices using correlation techniques like formal concept analysis, and storing them as optimized views for seamless analysis and retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing databases store large volume of varied data from multiple sources, then data storage capacity is improved, but data retrieval efficiency deteriorates
Solution Approach 1:
The patent segments data into distinct categories (entity data, event data, domain-specific association data) and stores each type in separate data stores. This segmentation allows for optimized retrieval paths for different query types, resolving the contradiction between storing large volumes of varied data and maintaining efficient retrieval by organizing data structure according to its type and usage patterns.
2Quantity of substance
If existing databases store pre-processed data, then storage capacity is improved, but analysis and retrieval efficiency deteriorates
Solution Approach 1:
The patent performs preliminary processing of data during the storage phase by extracting entities, events, and domain-specific associations, and organizing them into structured formats in separate data stores. This preliminary action enables efficient retrieval and analysis later without requiring additional processing, thus resolving the contradiction between storage capacity and retrieval efficiency.
3Ease of operation
If existing databases group data by user-defined domains, then data organization is improved, but support for complex queries deteriorates
Solution Approach 1:
The patent creates a multi-functional data architecture where data is organized by type (entity, event, association) in separate data stores, yet maintains universal query capability through the ability to join and correlate data across all stores. This universal structure supports both simple domain-based queries and complex multi-dimensional queries simultaneously, resolving the contradiction between data organization and query versatility.
Data Source
AI summary
The present invention relates to a system and method for pre-processing the data and storing the pre-processed data based on its type at the time of storage. During the pre-processing time, the system extracts all the relevant information such as entities, events/time stamped data, domain specific associations from the received content and stores them separately in a different data store. The system also generates different views based on the correlation between the identified entities, events and domain specific associations using data folding techniques. The system also stores the created views in a separate data store such as a view store, wherein the view store is optimized to serve end user queries by traversing through the stored views.