Time-Series Data Preconditioning for Storage Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The massive size of time-series databases, resulting from tracking numerous entities at frequent intervals, poses challenges for efficient data storage, transmission, and querying, particularly due to redundancy and computational inefficiencies in finding specific data points over time.
Innovation Solution
A method that promotes new time-series data entries with changed values while maintaining previous entries with unchanged values, and generates synthetic entries for entities without data in a fixed period, optimizing storage and query efficiency by reducing redundancy and ensuring query results are always available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all time-series data entries are stored to ensure complete data availability, then data completeness is improved, but storage requirements and query complexity increase significantly
Solution Approach 1:
The patent extracts only the essential data points by identifying local extrema (peaks and valleys) in time-series data. Instead of storing all data entries, the system extracts and stores only those points that represent significant changes or boundaries in the data pattern, thereby reducing storage requirements while maintaining the ability to reconstruct and analyze the original data trends.
Solution Approach 2:
The patent transforms the time-series data by changing its representation parameters. Rather than storing raw sequential data points, the system converts the data into a normalized form that identifies extrema points and their relationships. This parameter transformation allows the same information to be stored more efficiently while preserving data integrity and query capabilities.
2Reliability
If all time-series data entries are stored to ensure complete data availability, then data completeness is improved, but query efficiency deteriorates due to increased data volume
Solution Approach 1:
The patent extracts only the essential data points by identifying local extrema (peaks and valleys) in time-series data. Instead of storing all data entries, the system extracts and stores only those points that represent significant changes or boundaries in the data pattern, thereby reducing storage requirements while maintaining the ability to reconstruct and analyze the original data trends.
Solution Approach 2:
The patent segments the continuous time-series data into discrete meaningful units by identifying and storing only the extrema points. This segmentation transforms the data from a continuous stream into discrete, queryable events, making it easier to search and analyze specific time periods or patterns without processing the entire dataset.
3Quantity of substance
If redundant data entries are eliminated to reduce storage requirements, then storage efficiency is improved, but data redundancy needed for query reliability is reduced
Solution Approach 1:
The patent transforms the time-series data by changing its representation parameters. Rather than storing raw sequential data points, the system converts the data into a normalized form that identifies extrema points and their relationships. This parameter transformation allows the same information to be stored more efficiently while preserving data integrity and query capabilities.
Solution Approach 2:
The patent performs preliminary processing of the time-series data by pre-identifying and storing the extrema points before any queries are executed. This preliminary action of data normalization and extrema identification ensures that the most important data characteristics are captured and stored in an optimized format, making subsequent queries more reliable and efficient.
Data Source
AI summary
A method of preconditioning time-series data for improved efficiency. The method includes repeatedly recognizing new time series entries for each of a plurality of tracked entities. For each of the plurality of tracked entities, a new time-series data entry for the tracked entity is promoted responsive to the new time-series data entry including a changed value in a normalized field. However, a previous time-series data entry for the tracked entity is maintained responsive to the new time-series data entry including an unchanged value in the normalized field. The method further incudes generating a synthetic time-series data entry for every tracked entity without a time-series data entry in a fixed time period.