Tiled Data Tree Structure for Efficient Query Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data processing methods become inefficient and costly as data volume increases, particularly due to delays and high memory and processing loads when handling large datasets, and existing techniques for tile-based distribution and parallel processing do not adequately address these issues.
Innovation Solution
A computing system that uses a tree structure to subdivide data into tiles, allowing for parallel processing and caching precomputed information, which enables efficient computation and presentation of data by recursively determining information from sub-tiles and merging precomputed data with other information to quickly respond to user requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is aggregated for computation and presentation, then computation can be performed on complete datasets, but processing time and memory load become prohibitively expensive as data volume increases
Solution Approach 1:
The patent divides large datasets into hierarchical tile structures, where data is organized into parent tiles and child tiles at multiple levels. This segmentation allows the system to process only relevant tiles rather than entire datasets, dramatically reducing processing time while maintaining computation accuracy through selective aggregation of precomputed tile data.
Solution Approach 2:
The patent implements precomputation of data aggregates at the tile level before queries are executed. By precomputing and storing aggregate information for each tile in the hierarchical structure, the system avoids performing expensive computations on large datasets during query execution, thus reducing processing time while preserving accuracy through the use of precomputed values.
2Quantity of substance
If data is sharded into multiple data shards, then data storage capacity increases, but delays and latencies are introduced when data from multiple shards are required for computation
Solution Approach 1:
The patent merges multiple sharded data tiles into a hierarchical structure where parent tiles contain aggregate information from their child tiles. This merging allows the system to retrieve precomputed aggregate data from parent tiles without needing to access and combine data from multiple child shards, thus maintaining storage capacity while eliminating the latency associated with cross-shard data aggregation.
Solution Approach 2:
The patent performs preliminary aggregation of data from multiple shards at the tile level during data ingestion and preprocessing. By precomputing aggregates across shard boundaries and storing them in the hierarchical tile structure, the system eliminates the need for expensive cross-shard joins during query execution, thus improving processing speed while maintaining storage capacity.
3Measurement precision
If data is stored in detailed granular form, then query accuracy is improved, but memory load and processing requirements increase significantly
Solution Approach 1:
The patent segments data into a hierarchical tile structure where detailed granular data is stored at the leaf tile level while aggregate summaries are stored at parent tile levels. This segmentation allows the system to load only the necessary level of detail into memory based on query requirements, reducing memory load while maintaining query accuracy through selective access to detailed data when needed.
Solution Approach 2:
The patent implements a progressive refinement approach where queries first access aggregate data from parent tiles to determine if detailed data is needed. Only when necessary does the system load detailed granular data from child tiles into memory, thus minimizing memory load while ensuring query accuracy is maintained when detailed information is required.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Systems, methods, and computer readable media are provided for using tiled data. An information request for an area may be received. Information for the area may be accessed. The information for the area may be stored in a tree structure such that the information is subdivided into one or more levels of tiles. Individual tiles may correspond to a portion of the area. One or more tiles may include precomputed information for the corresponding portion(s) of the area. A response to the information request may be determined based on the precomputed information.