Time Series Data Compression by Type for Faster Querying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Time series databases in IoT systems face inefficiencies in compression and storage due to indiscriminate data handling, leading to low compression rates and limited query capabilities, as they lack targeted compression and storage methods for different data types.
Innovation Solution
A method and apparatus for storing and querying time series data that determine the data type of incoming data, apply corresponding compression methods, and store it in specific data storage tables, allowing for efficient compression and targeted querying based on data type and time conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If all time series data is stored indiscriminately using a single compression method, then the storage system is simple to implement, but the compression rate is low and storage resources are wasted
Solution Approach 1:
The patent segments time series data into different data types (analog quantities, state quantities, cumulative quantities) based on their change characteristics over time. Each data type is then compressed using a specialized compression method tailored to its characteristics, thereby improving the overall compression rate while maintaining manageable system complexity through structured classification.
2Loss of substance
If different compression methods are applied to different data types, then the compression rate improves, but the device complexity increases
Solution Approach 1:
The patent applies local quality by matching specific compression methods to specific data types based on their local characteristics. Analog quantities use one compression approach, state quantities use another, and cumulative quantities use a third. This localized optimization improves compression efficiency without requiring a completely complex system, as each component handles only its specialized data type.
Solution Approach 2:
The patent changes the parameter of data classification by identifying and categorizing data based on its temporal change characteristics. By transforming raw time series data into classified data types with distinct compression strategies, the system achieves better compression rates while managing complexity through parameter-based organization rather than arbitrary complexity.
3Ease of operation
If data is stored without categorization, then the query process is simple, but the query efficiency and functionality are limited
Solution Approach 1:
The patent segments stored data into different data storage tables according to data types. When querying, the system can directly access the relevant data storage table based on the query data type, avoiding unnecessary searches through all data. This segmentation improves query efficiency and enables richer query functionalities while maintaining simplicity through type-based routing.
4Device complexity
If all data is stored in a single storage table, then the storage structure is simple, but the query capabilities are limited
Solution Approach 1:
The patent divides the storage structure into multiple data storage tables, each dedicated to a specific data type. This segmentation enables the system to provide targeted query capabilities for different data types (analog, state, cumulative quantities) while keeping each individual storage table relatively simple in structure. The overall system achieves versatility through the collection of specialized tables rather than a single complex table.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Disclosed are a method and apparatus for storing and querying time series data. The method includes: determining a data type of data to be stored; compressing the data to be stored by a data compression method corresponding to the data type; storing compressed data to a data storage table corresponding to the data type; receiving a query request including a query data type and a query time condition; querying target data that meets the query time conditions from a data storage table corresponding to the query data type. In the embodiments of the present disclosure, different compression methods are adopted for different types of data, which improves the compression efficiency of time series data and save storage resources. Moreover, when performing data query, time series data that meets a query time condition is searched in a data storage table corresponding to a query data type, which improves the query efficiency of different types of time series data.