Correlating Unstructured Data in Non-Relational Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database management systems face challenges in efficiently searching and correlating data across large and constantly updated non-relational tables, especially when these tables are not actively structured or indexed, leading to increased overhead and storage requirements.
Innovation Solution
A method is introduced that uses a novel application of relational set logic to organize unstructured data in non-relational DBMS, creating an Index Table, Stitch Object Table, and Stitch Map Object Table to efficiently determine correlations and monitor data across multiple tables, allowing for rapid and efficient data retrieval without requiring tables to be dynamically indexed or relationally maintained.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If tables are actively structured and indexed in a relational DBMS to enable efficient searching and correlation, then data retrieval speed improves, but overhead and storage requirements increase
Solution Approach 1:
The patent segments the data retrieval process into two distinct phases: an offline indexing phase where correlation data is pre-computed and stored in auxiliary structures, and an online query phase where pre-computed indexes are utilized for rapid correlation detection. This segmentation allows the system to achieve fast correlation queries without maintaining complex relational structures in the primary data tables.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing correlation indexes during an offline phase before actual queries are executed. The system pre-identifies potential correlations between tables and stores this information in auxiliary data structures, so that during online operations, correlation detection can be performed rapidly without real-time computation overhead.
2Adaptability or versatility
If tables are manually related by choosing common fields and performing individual operations, then data correlation capability is achieved, but processing time and operational complexity increase
Solution Approach 1:
The patent creates a universal correlation index structure that can handle multiple types of data correlations across different tables simultaneously. The auxiliary data structures are designed to store and retrieve correlation information for various field combinations, enabling the system to efficiently manage diverse correlation queries without requiring separate processing mechanisms for each correlation type.
Solution Approach 2:
The patent introduces auxiliary data structures as intermediaries between the raw table data and the correlation query operations. These intermediate structures pre-process and organize correlation information, serving as a mediator that translates complex multi-table correlation requirements into efficient lookup operations, thereby reducing processing time for correlation detection.
3Ease of operation
If relational DBMS functionality is used to maintain table relationships, then automated relationship management is achieved, but system resources are consumed and performance decreases
Solution Approach 1:
The patent extracts the correlation management functionality from the primary relational DBMS operations and implements it separately using auxiliary data structures. By taking out the correlation indexing and maintenance operations from the main data processing path, the system achieves automated relationship management without the performance overhead associated with maintaining relational structures in the primary database tables.
Solution Approach 2:
The patent creates simplified copies of correlation information in auxiliary data structures rather than maintaining full relational structures. Instead of copying and maintaining complete relational definitions and constraints, the system creates lightweight index structures that capture only the essential correlation information needed for query operations, reducing memory and processing overhead while preserving automated relationship management capabilities.
Data Source
AI summary
A system, process, and method for organizing unstructured data stored in a database environment with table structure such that the data is retrievable using relational set logic, even with a database environment that does not provide relational table structures, is disclosed. The method creates, updates, and deletes database objects as necessary to describe the relationships between disparate data object types for various unique values and to provide the necessary information to retrieve the desired data objects. A device embodying and enabling the practice of the method is also disclosed.