Time Series Data Storage Compression for Magnetic Disk Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for storing time series data result in significant storage space occupation and increased I/O traffic due to null values and variable time intervals, leading to inefficient use of magnetic disk space.
Innovation Solution
A method and apparatus that save index values of time series data to a time interval data table, perform data compression on unused storage areas, and write the compressed data to a magnetic disk, optimizing storage space and reducing I/O traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If storage areas are reserved for each time point according to fixed time intervals, then data can be written directly to predetermined locations, but large storage space is occupied due to null values and I/O traffic increases
Solution Approach 1:
The patent extracts only the actual data values from the time series, storing them in a compressed format without allocating storage space for null values. Instead of reserving fixed storage areas for every time point, the system stores only non-null data points with their corresponding timestamps, thereby eliminating wasted storage space while maintaining efficient data access
Solution Approach 2:
The patent changes the storage parameter from fixed-time-interval allocation to variable data-point storage. By transitioning from a structure that reserves space for every time point to one that stores only actual data values, the system optimizes both storage space utilization and I/O efficiency
2Device complexity
If storage areas are reserved for each time point according to fixed time intervals, then data structure remains simple and direct, but I/O traffic during subsequent data read/write is increased
Solution Approach 1:
The patent extracts only the essential data elements (timestamp and value pairs) from the time series, storing them in a compact format. This extraction approach eliminates unnecessary null value storage while maintaining a relatively simple data structure that consists of ordered data points with timestamps and values
Solution Approach 2:
The patent performs preliminary data filtering and compression before writing to storage, organizing data into an efficient format that reduces subsequent I/O traffic. By pre-processing the data to remove null values and compress the representation, the system reduces the amount of data that needs to be read and written during normal operations
Data Source
AI summary
A method and apparatus for storing time series data are disclosed. A method may include: saving an index value of time series data to a time interval data table according to a time point of the time series data, wherein the time series data includes: a time point and an index value associated with the time point, and each data row in the time interval data table records index values associated with at least one time point within a time interval of a predetermined length; performing data compression processing on a storage area having no written data, to obtain compressed time interval data; and writing the compressed time interval data to a magnetic disk.