Polynomial Data Aggregation for Long-Term Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage techniques for large volumes of time-based data, such as online transactions, face challenges in managing increasing memory demands and predicting storage requirements, especially when storing 'raw' data over long periods, and aggregation methods often compromise data granularity for memory savings.
Innovation Solution
A novel data storage technique that aggregates data using a six-degree polynomial to represent data distributions across time periods, allowing for reduced storage space while preserving data granularity for detailed analysis, by storing aggregated values and coefficients for each level of granularity, enabling efficient storage and analysis of long-term data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If raw data is stored for long-term data storage, then data granularity and detail are preserved, but storage space requirements increase significantly
Solution Approach 1:
The patent transforms raw data into a different parameter representation using polynomial coefficients. Instead of storing individual data points, the system stores coefficients of a sixth-degree polynomial that mathematically represents the data distribution. This parameter transformation reduces storage requirements while preserving the ability to reconstruct and analyze the original data characteristics.
Solution Approach 2:
The patent creates a mathematical model (polynomial representation) that copies the essential characteristics of the raw data without storing the actual raw data points. This model copy allows for data analysis and reconstruction while occupying minimal storage space, effectively separating the storage function from the analysis function.
2Quantity of substance
If data is aggregated to reduce storage requirements, then storage space is reduced, but data granularity and detail are lost
Solution Approach 1:
The patent uses polynomial coefficient representation to capture the distribution characteristics of aggregated data. By storing coefficients that describe the polynomial fit to the data distribution, the system preserves information about data patterns, trends, and variability even when individual data points are aggregated, enabling detailed analysis without storing raw granular data.
3Loss of information
If memory resources are allocated for detailed data storage, then data analysis capability is maintained, but memory efficiency decreases
Solution Approach 1:
The patent creates a compact mathematical model that copies the essential analytical characteristics of the raw data. This polynomial representation serves as a simplified copy that retains the ability to perform data analysis, trend identification, and pattern recognition while consuming minimal memory resources compared to storing complete raw datasets.
Data Source
AI summary
Systems, methods, and articles of manufacture provide for rolling long-term data storage. Optimized or enhanced rolling long-term data storage may, for example, increase processing performance and reduce operational burdens on memory resources associated with execution of analytical models.