Database Memory Optimization via Epoch Delta Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database management systems face challenges in optimizing memory utilization due to the increasing volume of data, leading to space crunch situations, especially when handling large datasets for performance monitoring and analysis, where data integrity must be maintained.
Innovation Solution
A system and method that designates a first quantitative value as an epoch, compares subsequent values to determine differences, and stores these differences as delta values or designates them as new epochs based on a threshold, generating master tuples to optimize memory usage by avoiding repeated values in the database.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If huge amount of data is stored in the database for performance monitoring and analysis, then data analysis capability is improved, but memory utilization deteriorates due to space crunch
Solution Approach 1:
The patent segments continuous data into discrete tuples with epoch markers and delta values. Instead of storing every data point, the system divides data into segments separated by epoch boundaries, where each segment is represented by a starting epoch value and subsequent delta changes. This segmentation reduces the total number of stored values while preserving the ability to reconstruct original data sequences for analysis.
Solution Approach 2:
The patent transforms the storage parameter from absolute data values to relative delta values. By storing differences between consecutive data points rather than the actual values themselves, the system reduces memory requirements. When delta values are zero or negligible, they are not stored, further optimizing space utilization while maintaining data integrity for reconstruction when needed.
2Reliability
If continuous data is received and stored in real time from various sources, then data completeness is improved, but memory utilization deteriorates due to space constraints
Solution Approach 1:
The patent extracts only the essential components needed for data representation and storage. Instead of storing complete continuous data streams, it extracts key elements: epoch markers that define data segments and delta values that represent changes within segments. This extraction approach maintains data completeness for analysis purposes while significantly reducing the memory footprint by storing only necessary information.
Solution Approach 2:
The patent applies partial storage action by selectively storing only non-zero delta values and epoch markers. When data remains unchanged or changes are negligible, no storage action is taken for those particular data points. This partial action approach ensures data completeness where changes occur while avoiding redundant storage of unchanged values, optimizing memory utilization in real-time continuous data processing.
3Reliability
If all quantitative values are stored in the database, then data integrity is improved, but memory utilization deteriorates due to space crunch
Solution Approach 1:
The patent performs preliminary organization of data into epoch-based tuples before storage. By pre-structuring data with epoch markers and delta values in a standardized tuple format, the system ensures data integrity is maintained through consistent structure. This preliminary action allows for efficient storage and reliable reconstruction of original data, as the epoch-tuple structure preserves the relationship between consecutive data points while reducing storage requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is method and system for memory utilization in database capable of storing plurality of quantitative values. A first quantitative value may be designated as first epoch. A threshold may be used for comparing second quantitative value. Further, on comparison, difference may be determined. In one scenario, the difference may be stored as delta value in a tuple corresponding to the second quantitative value when an absolute value of the difference is less than the threshold. In another scenario, the second quantitative value may be designated as next epoch when the absolute value of the difference is equal to/greater than the threshold. Further, remaining quantitative values may also be compared with the first epoch and/or the second epoch. The remaining quantitative values may exclude the first quantitative value and the second quantitative value. Further, plurality of tuples may be compared amongst them for optimizing the memory utilization in the database.