Sub-block Data State Tracking for Partial Corruption Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Log-structured storage systems face unavailability issues due to data corruption, where a small error in a large block results in the entire block being reported as unavailable, amplifying the scope of data corruption.
Innovation Solution
Creating data state information in the virtual address space to indicate the availability of data stored in sub-blocks of a physical data block, allowing for finer granularity in determining data availability and selecting available sub-blocks for reading.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the storage system uses large block size for read/write operations, then the productivity is improved, but the reliability deteriorates because a small data error causes the entire block to be reported as unavailable
Solution Approach 1:
The patent divides the physical data block into multiple sub-blocks (e.g., 8 sub-blocks of 512 bytes each within a 4KB block) and introduces data state information to track the availability status of each sub-block independently. This segmentation allows the system to read only the available sub-blocks rather than the entire block, resolving the contradiction between maintaining large block sizes for efficiency and ensuring reliability when partial corruption occurs.
2Reliability
If the storage system reports entire blocks as unavailable due to corruption, then the reliability is maintained, but the loss of information increases because available data within the block cannot be accessed
Solution Approach 1:
By segmenting the data block into sub-blocks with individual availability tracking, the system can identify which sub-blocks are corrupted and which are intact. The data state information acts as a map that guides the reading process to only the available sub-blocks, preventing information loss while maintaining reliability through corruption detection.
Solution Approach 2:
The data state information serves as an intermediary between the physical data block and the reading operation. It mediates by providing availability status of each sub-block, enabling the system to make informed decisions about which data to read without accessing corrupted portions, thus preventing information loss while ensuring data integrity.
3Reliability
If the storage system uses fine-grained sub-block tracking, then the reliability is improved by reducing data corruption scope, but the device complexity increases due to additional data state information management
Solution Approach 1:
The segmentation into sub-blocks with individual status tracking does increase complexity, but it is managed by organizing the data state information in a structured manner that mirrors the sub-block structure. This systematic organization makes the complexity manageable and scalable.
Solution Approach 2:
The data state information structure serves multiple functions: it tracks availability status, guides reading operations to valid data, and enables selective reading of only available sub-blocks. This multi-functionality justifies the added complexity by providing comprehensive reliability monitoring and data recovery capabilities.
Data Source
AI summary
Techniques read data. Such techniques involve: in response to receiving a read request from the user for data on a physical data block, determining whether there is data state information corresponding to the physical data block. The data state information may include a plurality of units for respectively indicating availability of data stored in a plurality of sub-blocks of the physical data block. Such techniques further involve: in response to determining that there is data state information, selecting a target sub-block from the plurality of sub-blocks of the physical storage block based on the data state information. Such techniques further involve: providing the user with data stored in the target sub-block. Such techniques are capable of determining the availability of data at a finer granularity.