Web Analytics Aggregation via Hierarchical Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web analytics systems face inefficiencies in calculating and reporting aggregated data, such as total advertisement impressions, due to a growth rate of O(N) in execution time, which becomes problematic when instantaneous results are required, especially when dealing with large date ranges.
Innovation Solution
A system that stores episodic values and precomputed aggregates in separate data storage, allowing for efficient retrieval and summation of data by using a base number algorithm to determine the appropriate level of aggregation, reducing the complexity from O(N) to O(log N) by using a combination of episodic values and aggregate values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If precomputed aggregate values are stored and retrieved, then calculation speed is improved, but data storage complexity increases
Solution Approach 1:
The patent segments the data storage structure into multiple levels: episodic values (individual data points) and aggregate values (precomputed summaries). This segmentation allows the system to store data in a hierarchical manner where aggregate values can be quickly retrieved for large date ranges while episodic values remain available for detailed analysis, thus improving calculation speed without requiring complete restructuring of the entire storage system.
Solution Approach 2:
The patent implements preliminary computation of aggregate values that are stored in advance for frequently queried date ranges. By precomputing and storing these aggregate values, the system eliminates the need to perform O(N) calculations during runtime, thereby significantly improving calculation speed for common reporting scenarios while only storing preprocessed results.
2Measurement precision
If all episodic values are retrieved and summed, then measurement precision is maintained, but execution time increases
Solution Approach 1:
The patent applies partial action by retrieving only the necessary portion of data based on the requested date range. Instead of always retrieving all episodic values, the system intelligently determines whether to retrieve precomputed aggregate values or individual episodic values, or a combination thereof, depending on the specific query requirements. This approach maintains measurement precision by ensuring all relevant data is included while significantly reducing execution time by avoiding unnecessary data retrieval.
3Device complexity
If data is stored in a single flat structure, then device complexity is reduced, but query efficiency deteriorates
Solution Approach 1:
The patent introduces an additional dimension to the storage structure by organizing data hierarchically across multiple levels (episodic values and aggregate values at different aggregation levels). This dimensional change transforms the flat storage structure into a multi-level hierarchy that enables efficient querying through selective retrieval of appropriate aggregation levels, thereby significantly improving query efficiency while managing complexity through structured organization.
Data Source
AI summary
Disclosed is a computer implemented method of determining an aggregated web analytics value. A first data storage medium stores a sequence of episodic values, each associated with a respective time interval. A second data storage medium stores a sequence of aggregate values, each aggregating an equal number of the episodic values. Further aggregates are stored and provided in a like manner, until an additional sequence of higher level aggregate values cannot be created using the same base number. When a request for an aggregate value over a range of episodic values is received, the system queries the storage mediums in a parallel operation, retrieving as little aggregated values and episodic values needed to compute the requested value.