Hybrid Data Storage Query Summary Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional relational database management systems face performance issues and increased storage needs when indexing large volumes of data, leading to inefficient query processing and result review in environments with millions or billions of entries.
Innovation Solution
A hybrid data storage system that selectively uses column and row data storage to optimize query execution, generates query summaries with histograms for quick result identification, and performs optimizations to store summaries in memory, allowing for scalable query summaries and drill-downs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If indexes are created to improve query performance, then query execution speed is improved, but storage requirements increase and write operations take longer
Solution Approach 1:
The patent extracts only the most frequently queried columns into index structures, rather than indexing all columns. This selective approach creates smaller indexes that consume less storage space while still providing significant query performance improvements for the most common query patterns.
Solution Approach 2:
The patent creates multi-purpose index structures that can serve multiple query types and patterns simultaneously. By designing indexes that accommodate various query scenarios, the system reduces the total number of indexes needed, thereby reducing overall storage requirements while maintaining query performance.
2Speed
If indexes are created for every column, then query performance is improved, but write operations take longer due to re-indexing requirements
Solution Approach 1:
The patent extracts only the essential columns needed for query optimization into index structures, avoiding the creation of indexes on every column. This reduction in the number of indexed columns directly decreases the time required for write operations while maintaining query performance for the most critical query patterns.
3Reliability
If large amounts of query results are generated, then comprehensive search coverage is achieved, but result review becomes unmanageable and time-consuming
Solution Approach 1:
The patent segments the large set of query results into meaningful groups or categories based on relevant criteria. This segmentation organizes the results in a more manageable structure, allowing users to navigate and review findings more efficiently without sacrificing the completeness of the search coverage.
Solution Approach 2:
The patent introduces intermediary summary structures or aggregation layers that mediate between the raw query results and the user interface. These intermediaries provide summarized views, filtering options, or hierarchical presentations that make large result sets more manageable while preserving access to the complete underlying data.
Data Source
AI summary
A data storage system includes a column store and a row store. The data storage system may generate a query summary. The data storage system determines query summary fields comprised of a subset of fields from a schema used to store data in the row store. The data storage system searches the column store or the row store for query results in the query summary fields. A query summary field is generated from the query results in the query summary fields.