Database Cumulative Value Storage for Descriptive Statistics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional databases are inefficient in handling large amounts of contiguous data for descriptive statistics, particularly in high-frequency, high-volume data applications, leading to high costs and computational requirements, which limits smaller clients' access to complete market feeds.
Innovation Solution
A database configured to store data in a pre-computed or partially-processed form, allowing for efficient calculation of descriptive statistics by storing cumulative values, reducing the need for extensive data retrieval and computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional databases store and retrieve raw data for descriptive statistics, then data accuracy is maintained, but retrieval and computational costs increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-computing cumulative values (zn) during data insertion into the database. Instead of computing descriptive statistics from raw data at query time, the cumulative sums are calculated and stored in advance, allowing rapid retrieval with minimal computational overhead while maintaining full data accuracy.
Solution Approach 2:
The patent introduces an intermediary representation (cumulative values zn) that bridges raw data and final statistical results. These intermediate cumulative values serve as a compressed form that preserves all necessary information for computing descriptive statistics, reducing the amount of data that needs to be retrieved and processed while maintaining measurement precision.
2Loss of information
If complete market feeds are provided to all clients, then data completeness is improved, but bandwidth and computational requirements increase
Solution Approach 1:
The patent extracts only the essential cumulative values (zn) from the complete market feed and transmits these to clients. By taking out only the pre-computed cumulative data needed for descriptive statistics rather than transmitting the entire raw market feed, the system maintains data completeness for statistical purposes while dramatically reducing bandwidth consumption.
Solution Approach 2:
The patent changes the parameter representation from raw individual data points to cumulative aggregate values. This parameter transformation allows the same information content (for descriptive statistics) to be conveyed in a more compact form, reducing the quantity of data transmitted over bandwidth-constrained channels.
3Ease of manufacture
If raw data is stored in conventional databases, then data storage simplicity is maintained, but processing efficiency for contiguous data decreases
Solution Approach 1:
The patent changes the storage parameter from raw individual values to cumulative aggregate values. This parameter transformation enables efficient processing of contiguous data ranges because cumulative values allow O(1) computation of aggregates over any time period, dramatically improving productivity for descriptive statistics while maintaining straightforward database storage operations.
Data Source
AI summary
A computer system for calculating a quantity from a set of input values. The computer system comprises a database that is configured to store a sequence of first input values xn wherein n=1, 2, . . . i, wherein for any value of n, xn lies between xn+1 and xn−1 in the sequence. Each value of x has associated with it a second input value yn, a conversion value pn that is calculated from yn according to a conversion algorithm, and an output value zn, wherein zn=zn−1+pn. The database is configured to store the plurality of output values zn.