Data Relationships Storage Platform for Dynamic Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data management systems face challenges in detecting business value from relationships between diverse data points stored in a decentralized manner, as they often require re-structuring and re-analyzing data, which is time-consuming and difficult, especially with the exponential increase in data volume and changing business requirements.
Innovation Solution
A Data Relationships Storage Platform (DRSP) that collects, analyzes, and stores data pieces to determine relationships, creating 'data globs' that include both data and relationship information, allowing for dynamic analysis and flexible data management, enabling users to explore and manipulate relationships in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in a decentralized manner across several servers, then storage capacity and accessibility are improved, but relationship information between data is lost
Solution Approach 1:
The patent embeds relationship metadata within the decentralized data storage structure. Each data record contains nested relationship information that points to related records across the distributed system, allowing relationship data to be stored alongside decentralized data without requiring a centralized relationship database.
Solution Approach 2:
The patent introduces a relationship metadata layer that acts as an intermediary between decentralized data records. This metadata layer captures relationship information and enables queries to traverse across distributed servers, effectively mediating between the decentralized storage structure and the need for relationship查询.
2Productivity
If data is structured for task-specific computations, then computational efficiency is improved, but adaptability for new analysis is reduced
Solution Approach 1:
The patent implements dynamic schema evolution capabilities. The data structure includes flexible metadata that can be adapted without requiring complete schema migrations. When new analysis requirements emerge, the system can dynamically add new fields, relationships, or data types while maintaining existing data and computations, allowing the schema to evolve with changing business needs.
Solution Approach 2:
The patent segments data into modular records with independent metadata fields. This segmentation allows selective modification of specific data aspects without affecting the entire data structure. Users can query and analyze specific segments of data independently, enabling flexible reanalysis without restructuring the entire database schema.
3Speed
If data is analyzed at the outset and stored in a specific format, then initial query performance is improved, but time for re-structuring and re-analyzing is increased
Solution Approach 1:
The patent performs preliminary indexing and metadata organization during data ingestion, creating optimized data structures for fast queries. However, this preliminary structure is designed to be lightweight and modular, allowing rapid reconfiguration without requiring complete reindexing or data migration when analysis requirements change.
Data Source
AI summary
A data relationships storage platform for analysis of one or more data sources is described herein. A data processing system may be communicatively coupled to one or more data sources and one or more big-data databases. One or more collectors may collect data pieces from the one or more data sources. One or more analyzer may analyze the collected data pieces to determine whether one or more relationships exist between the collected data pieces. The analysis results in one or more data globs that include one or more of the data pieces and relationship information, such as tags. The tagged data globs may be communicated to and stored in one or more big-data databases.