Adaptive Time Series Encoding for Abnormal Data Storage Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for storing time series data using encoding techniques are inefficient due to the need for large descriptive information for abnormal data points, leading to poor storage performance and reduced space-saving effects.
Innovation Solution
A method and device for adaptive length encoding of time series data, which involves encoding previous n data values, determining storage space requirements, and selectively storing rule information and encoding data based on storage gain ratios to minimize storage space usage, while identifying and isolating abnormal data to maintain optimal storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If encoding techniques are used to store time series data, then storage space is reduced, but storage performance deteriorates due to large descriptive information requirements for abnormal data points
Solution Approach 1:
The patent implements dynamic encoding by adjusting the encoding window size n based on data characteristics. The system adapts the encoding strategy according to whether abnormal data is detected, switching between encoding modes to optimize both storage space and performance. This dynamic adjustment resolves the contradiction by making the encoding process flexible rather than static.
Solution Approach 2:
The patent applies different encoding strategies to different segments of time series data based on local characteristics. When abnormal data is detected in a segment, the system adjusts the encoding approach for that specific segment while maintaining efficient encoding for normal segments. This localized adaptation allows the system to achieve good storage performance overall while handling abnormal data points effectively.
2Quantity of substance
If encoding data with large n values is used to reduce storage space, then storage gain increases, but storage performance worsens due to increased descriptive information requirements
Solution Approach 1:
The patent uses partial action by encoding only n data points at a time rather than encoding the entire time series. This partial encoding approach allows the system to manage descriptive information in manageable chunks, reducing the overall information loss while still achieving significant storage space reduction. The parameter n is carefully selected to balance compression ratio with information preservation.
Solution Approach 2:
The patent segments the time series data into groups of n data points for encoding. This segmentation breaks down the large-scale encoding problem into smaller, more manageable segments. Each segment can be encoded independently with its own descriptive information requirements, preventing the exponential growth of descriptive information that would occur with monolithic encoding of large n values.
3Reliability
If abnormal data points are included in encoding, then data completeness is maintained, but storage space saving effect is reduced due to large descriptive information requirements
Solution Approach 1:
The patent extracts and separately handles abnormal data points from the main encoding process. When abnormal data is detected, it is identified and processed differently from normal data points. This extraction approach maintains data completeness by preserving abnormal points while allowing the majority of normal data to be encoded efficiently, thus maintaining storage space saving effects.
Solution Approach 2:
The patent introduces an intermediary mechanism (the storage gain calculation and comparison logic) that mediates between data completeness and storage efficiency. The system calculates storage gain for different encoding configurations and uses this intermediary metric to decide how to handle abnormal data points, balancing the need for data completeness with the goal of maintaining storage space savings.
Data Source
AI summary
Provided are a method and device for storing time series data with adaptive length encoding, including: acquiring data values corresponding to timestamps according to a sequential order of timestamps; using a ratio of storage space values required to pre-store the previous n data values to storage space values required to pre-store rule information of a preset encoding rule and encoding data according to the previous n data values as a storage gain corresponding to the time at which the n-th data value is acquired; storing the rule information of the preset encoding rule and the encoding data corresponding to a previous n−1 data values when the storage gain corresponding to the time at which the n-th data value is acquired is less than that corresponding to the time at which the (n−1)-th data value is acquired.