Columnar Data Storage and Ranked Analytics for Time-Based Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database management systems face inefficiencies in managing large volumes of time-based data due to non-ordered storage, multiple disk reads, and cross-socket access penalties, leading to slow data retrieval and processing.
Innovation Solution
A massively parallel database management system that stores data by columns in coding blocks within silo systems, using local resources and direct access to disk drives, with a hierarchical manifest for efficient data location and reduced reads, and prioritizes analytical calculations based on characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored by rows in conventional database management systems, then data can be retrieved using standard SQL queries, but multiple disk reads are required and data retrieval speed is slow
Solution Approach 1:
The patent segments data storage into columnar coding blocks within silos, where each silo stores data for a specific socket. This segmentation allows the system to retrieve only the specific columns needed for analytical calculations rather than reading entire rows, significantly reducing disk reads and improving data retrieval speed for time-based analytical queries.
Solution Approach 2:
The patent transitions from traditional row-based storage to column-based storage, effectively changing the dimension of data organization. This dimensional change enables efficient retrieval of specific columns without reading unrelated data, directly addressing the problem of slow data retrieval in conventional systems.
2Ease of operation
If conventional database management systems access remote devices across sockets, then data can be shared between processing units, but cross-socket access penalties cause increased latency
Solution Approach 1:
The patent implements local data storage within silos, where each silo is associated with a specific socket and stores data locally. This local quality principle ensures that processing units access data from local storage rather than remote devices, eliminating cross-socket access penalties and reducing latency while maintaining data accessibility through the distributed silo architecture.
3Adaptability or versatility
If conventional database management systems store data non-ordered on physical disk drives, then data can be flexibly organized, but time-based data retrieval requires multiple disk reads
Solution Approach 1:
The patent applies preliminary action by organizing time-based data in chronological order within coding blocks before storage. Time stamps are used to sequence data records, allowing the system to efficiently retrieve time-based data ranges by reading only the relevant chronological segments rather than scanning dispersed records across multiple disk locations.
4Reliability
If conventional database management systems read all columns to retrieve requested data, then complete records are available, but unnecessary data increases disk read time
Solution Approach 1:
The patent extracts and stores only the specific columns needed for analytical calculations into separate coding blocks within silos. This extraction principle allows the system to retrieve only the necessary data columns without reading unrelated columns, maintaining data reliability for analytical purposes while significantly improving retrieval speed by eliminating unnecessary data reads.
Data Source
AI summary
A method for execution by a computer of a database management system includes obtaining a dataset that includes a set of data records, where the data set is associated with a set of data characteristics. The method includes executing a selected ranked analytical calculation of a ranked list of analytical calculations on the dataset to produce an analytical calculation result, where the ranked list of analytical calculations is generated by ranking a list of analytical calculations that are able to be executed on the dataset, based on a set of analytical calculation characteristics associated with the list of analytical calculations, where an analytical calculation characteristic of the set of analytical calculation characteristics indicates an estimated execution time to perform an analytical calculation of the list of analytical calculations, and the selected ranked analytical calculation is selected based on the set of data characteristics to produce the selected ranked analytical calculation.