Virtual Data Sets Using Algebraic Relations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems are limited by predefined schema and structures that do not optimize for data storage and retrieval, leading to inefficiencies and difficulties in handling diverse data formats, such as flat files and XML, due to tight coupling between data relationships and storage structures.
Innovation Solution
The implementation of a system using algebraic relations to optimize data storage and retrieval, allowing for virtualization of data sets based on usage patterns and format translation between disparate data formats, enabling adaptive data restructuring and efficient query processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is stored using predetermined schema with predefined relationships, then data structure is organized and accessible, but the system becomes inefficient for diverse data formats and query operations
Solution Approach 1:
The patent segments data storage into multiple storage formats (relational, hierarchical, network, flat files, CSV, XML) that can be independently selected and combined based on query requirements and data characteristics, allowing efficient handling of diverse data formats without a single rigid schema
Solution Approach 2:
The system dynamically selects and switches between different storage formats and access methods based on query patterns and data characteristics, rather than using a fixed predetermined schema, enabling adaptation to diverse data formats and efficient query processing
2Reliability
If data is stored in a single centralized location, then data consistency is maintained, but query performance degrades due to multiple accesses required
Solution Approach 1:
The patent segments data into multiple storage locations and formats distributed across different storage systems, allowing queries to access only the necessary portions from appropriate storage formats rather than accessing a single centralized data store multiple times
Solution Approach 2:
The system performs preliminary data organization and preprocessing into multiple storage formats, so that when queries are executed, the data is already arranged for optimal access patterns, reducing the need for multiple accesses during query processing
3Device complexity
If predefined relationships are imposed on data, then database structure is simplified, but the system cannot efficiently capture actual data relationships
Solution Approach 1:
The patent changes the parameters of data representation by using multiple storage formats that can capture different aspects of data relationships (relational, hierarchical, network structures) simultaneously, allowing the system to preserve actual data structure information without imposing a single simplified predefined relationship model
Data Source
AI summary
Systems and methods for storing and accessing data using virtual data sets. Data sets may be removed from a data store and defined by algebraic relations between other data sets that are realized in the data store. A flag may be set to indicate that the data set is virtual. Criteria may be established for determining when a data set should be virtualized. For example, the criteria may be based on the size of the data set, the number of times it has been referenced and/or the frequency with which the data set has been accessed in the data store. A data set may also be optimized by partitioning the data set into subsets. The original data set may then be removed from the data store. An algebraic relation may be composed that defines the data set based on the subsets realized in the data store. The algebraic relation for the virtual data set may be used for optimizing access to other data sets even though the virtual data set is not realized.