Selective Data Compression via Red-Hot Indicators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
During the initial synchronization of mirrored storage resources, the backup storage system is subjected to unnecessary CPU and compression/decompression engine strain due to compressing and then decompressing data, as it conventionally compresses all data received and later uncompresses a portion based on the red-hot data map, leading to inefficient resource utilization.
Innovation Solution
A red-hot data indicator is added to each track of data transmitted from the primary storage system to the backup storage system, allowing the backup system to selectively compress or store data in uncompressed form based on the indicator, thereby reducing the need for unnecessary compression and decompression during the initial synchronization phase.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the backup storage system compresses all data received during initial synchronization, then storage space utilization is improved, but CPU and compression/decompression engine resources are excessively strained
Solution Approach 1:
The patent applies local quality by differentiating compression treatment for different portions of data. Instead of uniformly compressing all data, the system identifies 'red-hot' data (frequently accessed) and maintains it uncompressed while compressing cold data. This selective approach optimizes both storage utilization and resource consumption by applying compression only where beneficial.
Solution Approach 2:
The patent implements preliminary action by having the primary storage system pre-identify and mark red-hot data tracks before transmission. The red-hot data map is prepared in advance and transmitted to the backup system, enabling the backup system to avoid unnecessary compression operations from the outset rather than compressing all data and then decompressing later.
2Quantity of substance
If the backup storage system compresses all data during initial synchronization, then data is stored in compressed form, but subsequent decompression of red-hot data reduces resource efficiency
Solution Approach 1:
The system applies different storage treatments to different data portions based on their access characteristics. Red-hot data tracks are stored uncompressed for rapid access, while cold data tracks are stored compressed to maximize storage density. This local differentiation resolves the contradiction between storage efficiency and access efficiency.
Solution Approach 2:
The primary storage system performs preliminary identification of red-hot data and transmits this information via the red-hot data map before the backup system stores the data. This preliminary action enables the backup system to make informed storage decisions immediately, avoiding the need for later decompression operations and maintaining high synchronization productivity.
3Device complexity
If the backup storage system does not use selective compression, then resource utilization is simpler to manage, but unnecessary compression and decompression operations waste time and resources
Solution Approach 1:
The patent introduces a red-hot data map as an intermediary information structure that carries compression instructions from the primary to the backup system. This intermediary enables selective compression without requiring complex real-time analysis at the backup system, as the compression decisions are pre-determined and transmitted via the map.
Solution Approach 2:
The compression strategy is determined in advance by the primary storage system through preliminary analysis of data access patterns. The red-hot data map is prepared beforehand, allowing the backup system to simply follow pre-determined instructions rather than making complex decisions in real-time, thus reducing both time loss and management complexity.
Data Source
AI summary
A primary storage system appends a red-hot data indicator to each track of data transmitted on a remote data facility during an initial synchronization state. The red-hot data indicator indicates, on a track-by-track basis, whether the data associated with that track should be stored as compressed or uncompressed data by the backup storage system. The red-hot data indicator may be obtained from the primary storage system's extent-based red-hot data map. If the red-hot data indicator indicates that the track should remain uncompressed, or if the track is locally identified as red-hot data, the backup storage system stores the track as uncompressed data. If the red-hot data indicator indicates that the track should be compressed, the backup storage system compresses the track and stores the track as compressed data. After the initial synchronization process has completed, red-hot data indicators are no longer appended to tracks by the primary storage system.