Distributed Storage Updates Using Trim Records and Sequence Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solid-state drives are designed to conform to hard disk drive standards, limiting the ability to leverage unique aspects of flash and other solid-state memory for enhanced features and efficient data management.
Innovation Solution
A storage cluster architecture with storage nodes and units that utilize sequence numbers and trim records to manage data updates, ensuring consistency and fault tolerance through erasure coding and redundant copies, allowing proactive data rebuild and system resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If solid-state drives conform to hard disk drive standards, then compatibility is improved, but the ability to leverage unique aspects of flash memory is worsened
Solution Approach 1:
The patent segments data management into distinct components: sequence numbers for tracking updates, trim records for managing deleted data, and erasure coding for redundancy. This segmentation allows the system to maintain HDD compatibility while implementing flash-optimized data management techniques independently in specific layers.
Solution Approach 2:
The patent introduces intermediary data structures (sequence numbers and trim records) that mediate between the storage hardware layer and the data management layer. These intermediaries enable the system to translate HDD-compatible operations into flash-optimized execution, resolving the contradiction between compatibility and efficiency.
2Reliability
If sequence numbers and trim records are written to storage units, then data integrity is improved, but storage space consumption is worsened
Solution Approach 1:
The patent applies partial action by writing trim records only for sequence numbers that are allowed to not exist, rather than tracking every possible sequence number. This selective approach maintains data integrity for critical data while reducing unnecessary storage overhead for metadata.
Solution Approach 2:
The patent changes the parameter of data representation by using compact sequence number identifiers and trim record formats that efficiently encode metadata. This allows the system to maintain comprehensive tracking of data integrity while minimizing the storage space required for metadata.
3Reliability
If storage nodes are added to the cluster, then system availability is improved, but system complexity is worsened
Solution Approach 1:
The patent designs storage nodes with universal functionality where each node can perform multiple roles: storing data, maintaining sequence numbers, managing trim records, and participating in erasure coding. This multi-functionality reduces the need for specialized components, thereby improving availability without proportionally increasing complexity.
Solution Approach 2:
The patent uses copying through redundant copies of data and metadata across multiple storage nodes. Instead of creating complex centralized management systems, the system replicates essential information (sequence numbers, trim records, data) across nodes, allowing any node to function as a backup. This copying approach improves availability while keeping individual node complexity low.
Data Source
AI summary
A method for updates in a storage system is provided. The method includes writing identifiers, associated with data to be stored, to storage units of the storage system and writing trim records indicative of identifiers that are allowed to not exist in the storage system to the storage units. The method includes determining whether stored data corresponding to records of identifiers is valid based on the records of the identifiers and the trim records.