Measurement Data Storage with Time-Based Compression Tiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data management systems face challenges in efficiently managing large volumes of measurement data from sensors, particularly in reducing data size without losing essential information, and in securing sufficient storage space while allowing for efficient data restoration.
Innovation Solution
A data management system that includes a data acquisition unit, a data compression unit for lossless compression, a data size reduction unit that deletes or reduces data samples or bit width, and a decompression unit, which reduces data size over time based on predetermined criteria, allowing for efficient storage and transmission of measurement data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data compression is performed on all measurement data, then storage space is reduced, but data access and processing time increase
Solution Approach 1:
The patent divides measurement data into multiple segments based on time periods, with recent data stored in uncompressed format for fast access and older data stored in compressed format for space efficiency. This segmentation allows the system to balance between storage space and access speed by applying different storage strategies to different data segments.
Solution Approach 2:
The patent implements a dynamic data management approach where data is automatically transitioned between compressed and uncompressed states based on access patterns and age. Frequently accessed or recent data is maintained in uncompressed form, while less frequently accessed older data is compressed, allowing the system to adapt to changing storage and performance requirements.
2Quantity of substance
If data size is reduced by deleting samples or reducing bit width, then storage capacity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent changes data representation parameters dynamically based on data age and importance. Recent measurement data is stored with full precision (original bit width and sample rate), while older data is stored with reduced precision (lower bit width and/or reduced sampling rate). This parameter change strategy allows the system to optimize storage capacity while preserving measurement precision for data that requires it.
3Reliability
If lossless compression is applied to all data, then data restoration accuracy is maintained, but processing complexity increases
Solution Approach 1:
The patent applies different compression strategies to different portions of the data based on local requirements. Critical recent data receives lossless compression treatment to ensure full restoration accuracy, while less critical older data uses lossy compression methods with reduced processing complexity. This local quality approach allows the system to maintain reliability where needed while reducing overall processing complexity.
Data Source
AI summary
Provided is a data management system which includes a data acquisition unit configured to acquire measurement data obtained by measuring a measurement target, a data storage unit configured to store the acquired measurement data, a data size reduction unit configured to deletes at least a part of the stored measurement data to reduce a data size of the measurement data, and a data compression unit configured to perform data compression on the measurement data reduced in data size.