Odometer Retriever for Large Multidimensional Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multidimensional database environments face challenges in efficiently handling large queries due to the potential size of query data exceeding manageable limits, leading to performance issues and increased storage requirements for pre-calculated values in sparse dimensions.
Innovation Solution
The implementation of a dynamic flow process coupled with a kernel-based odometer retriever manages pointers to data blocks, enabling efficient handling of large queries by reducing the need for pre-calculated values and optimizing data retrieval through bottom-up aggregation and hybrid use of storage containers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If pre-calculated values are stored for sparse dimensions to enable efficient query processing, then query retrieval speed is improved, but storage requirements increase significantly
Solution Approach 1:
The patent segments the large query space into smaller hyper-rectangles that can be processed independently. By dividing the query into manageable segments, the system can process each segment separately without requiring all pre-calculated values to be loaded into memory simultaneously, thus reducing peak storage requirements while maintaining query speed.
Solution Approach 2:
The patent introduces a new dimension to the data structure by adding a level index to the odometer array. This transforms the traditional multi-dimensional storage into a hierarchical structure with an additional indexing dimension, enabling more efficient memory management and reducing the storage footprint required for sparse dimension data.
2Adaptability or versatility
If the query size is increased to accommodate larger data ranges, then query comprehensiveness is improved, but the system's ability to manage and process the query deteriorates
Solution Approach 1:
The patent automatically segments large queries into smaller hyper-rectangular sub-queries based on the available memory and processing capacity. This segmentation strategy allows the system to handle queries of any size by breaking them down into manageable pieces that can be processed sequentially, thereby maintaining system controllability while supporting large query ranges.
Solution Approach 2:
The patent implements dynamic query processing where the segmentation and processing strategy adapts based on the actual query parameters and system resources. The system dynamically adjusts the number and size of segments, the processing order, and resource allocation based on runtime conditions, enabling flexible handling of queries with varying sizes and complexities.
3Reliability
If all query data is loaded into memory for processing, then query processing completeness is improved, but memory usage exceeds manageable limits
Solution Approach 1:
The patent divides the query result set into multiple segments that can be processed and returned in batches. Each segment fits within available memory constraints, while the union of all segments provides the complete query result. This approach ensures query completeness is maintained while memory usage stays within manageable limits through iterative processing.
Solution Approach 2:
The patent performs preliminary segmentation and planning of the query processing before actual data retrieval. By pre-dividing the query space into segments and planning the retrieval order, the system ensures that memory usage is controlled from the outset while still guaranteeing that all necessary data will be retrieved and processed for complete query results.
Data Source
AI summary
In accordance with an embodiment, the system provides support for large queries in a multidimensional database computing environment. A kernel-based data structure, referred to herein as an odometer retriever, or odometer, that manages pointers to data blocks, contains control information, or otherwise operates as an array of arrays of pointers to stored members. When used with a dynamic flow, the approach enables the system to be used, for example to handle grid queries, Multidimensional Expressions (MDX) queries, or other types of queries in which the potential size of the query can be up to 264 bits.