Dense Data Structures for Database Query Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face inefficiencies in processing and querying multidimensional data due to the large size of fact tables and the need for extensive indexing or materialized views, which can be costly and resource-intensive, especially when dealing with numerous dimensions and foreign key columns.
Innovation Solution
The creation of dense data structures that compress and aggregate dimensional data, allowing for efficient processing by identifying actual value combinations rather than possible combinations, using dense set identifiers that uniquely represent sets of values across dimensions, reducing the need for extensive indexing and improving query evaluation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional database systems store and process multidimensional data in fact tables with many foreign key columns, then data completeness and query accuracy are improved, but processing cost and resource consumption increase significantly
Solution Approach 1:
The patent extracts only the necessary dimensional information from the full fact table by identifying actual value combinations present in the data. Instead of processing all rows with foreign key columns, the system extracts unique value combinations from dimension tables and creates a condensed representation that maintains query accuracy while reducing processing load.
Solution Approach 2:
The patent creates a compressed copy of the dimensional data structure using dense set identifiers. This copy represents the same dimensional information but in a condensed format that eliminates redundant foreign key references while preserving the relationships needed for accurate querying and analysis.
2Speed
If extensive indexing or materialized views are created to improve query performance, then query evaluation speed is improved, but system complexity and resource requirements increase
Solution Approach 1:
The patent performs preliminary processing by pre-identifying actual value combinations from dimension tables and creating a condensed dimensional structure before query execution. This preliminary action eliminates the need for extensive indexing during query evaluation, as the data is already organized in a query-efficient format.
Solution Approach 2:
The patent changes the representation parameters of dimensional data from full foreign key references to dense set identifiers. This parameter transformation compresses the data structure while maintaining queryability, eliminating the need for complex indexing mechanisms.
3Loss of information
If fact tables contain all occurrence characteristics and dimension information, then data completeness is improved, but storage requirements and processing resources increase
Solution Approach 1:
The patent extracts only the essential dimensional characteristics needed for analysis by identifying actual value combinations present in the data. This extraction process removes redundant information while preserving the core relationships between dimensions and facts, reducing storage requirements without compromising analytical completeness.
Solution Approach 2:
The patent merges dimensional information from multiple tables into a unified condensed structure using dense set identifiers. This merging process consolidates repeated dimensional data while maintaining referential integrity, reducing overall storage requirements while preserving data completeness for analysis.
Data Source
AI summary
Methods, computer systems, and stored instructions are described herein for densely grouping dimensional data and/or aggregating data using a data structure, such as one that is constructed based on dimensional data. When smaller tables are joined with a larger table, a server may analyze the smaller tables first to determine actual value combinations that occur in the smaller tables, and these actual value combinations are used to more efficiently process the larger table. A dense data structure may be generated by processing dimensional data before processing data from fact table. The dense data structure may be generated by compressing ranges of values that are possible in dimensions into a range of values that actually occurs in the dimensions. The compressed range of values may be represented by dense set identifiers rather than the actual compressed range of values.