Erasure-Coded Storage Writes Without Data Journaling Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Erasure-coded storage systems face challenges in efficiently writing data without causing data corruption or requiring excessive storage, particularly due to issues with stripe write operations and the need for multiple write operations, which can lead to increased storage consumption and reduced system throughput.
Innovation Solution
The method involves distinguishing between different types of information (data and metadata) and employing appropriate storage processes and mechanisms, such as journaling only metadata and storing data without journaling, to ensure efficient and reliable writing to erasure-coded storage, while maintaining data integrity and avoiding double writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written to journaled storage with full journaling, then data integrity is ensured, but write performance decreases and storage overhead increases
Solution Approach 1:
The patent segments the storage system into journaled storage for metadata and non-journaled erasure-coded storage for data. This segmentation allows different journaling strategies to be applied to different data types, improving overall write performance while maintaining data integrity for metadata.
Solution Approach 2:
The patent applies journaling selectively based on local requirements: full journaling is applied to metadata where integrity is critical, while no journaling is applied to data where write performance is prioritized. This local quality approach resolves the contradiction by tailoring the reliability mechanism to the specific needs of each storage region.
2Reliability
If erasure-coded parity information is written, then data protection is improved, but storage overhead and write operations increase
Solution Approach 1:
The patent changes the journaling parameter for data storage from 'journaling enabled' to 'journaling disabled' when storing data in non-journaled erasure-coded storage volumes. This parameter change eliminates redundant journaling operations for data, reducing storage overhead and write operations while maintaining data protection through erasure coding.
3Reliability
If multiple write operations are performed (write to log then write to storage), then data consistency is ensured, but system throughput is reduced
Solution Approach 1:
The patent extracts the journaling step from the data write path, eliminating the redundant 'write to log then write to storage' sequence for data. By taking out the journaling operation that is unnecessary for data in non-journaled storage, the system achieves higher throughput while maintaining consistency through the erasure-coded storage mechanism.
Data Source
AI summary
Methods and the like according to the disclosure can include determining an information type of digital information, writing the digital information using a first process (in response to a determination that the information type of the digital information is a first information type), and, in response to a determination that the information type of the digital information is a second information type, determining erasure-coded parity information for the digital information and writing the digital information using a second process (where the digital information is to be written to erasure-coded storage). In the former case, the first process that includes writing the digital information to storage. The second process includes writing the digital information and the erasure-coded parity information to erasure-coded storage, without writing the digital information to a log.