File Storage System Counter Value Management for Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed file systems face challenges in maintaining data consistency between nodes while preventing a decline in client IO performance, particularly due to access conflicts and performance issues related to pseudo-timestamps and journal updates.
Innovation Solution
A file storage system with a management information file that includes a counter value indicating a time-series sequence, along with an operation log detailing operation content, is implemented. This system updates the counter value when files are transferred between storage systems, allowing for consistent management information and reduced access conflicts, thereby maintaining performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a pseudo-timestamp is shared by clusters and attached to journal entries to ensure data consistency, then reliability is improved, but access conflicts occur between multiple nodes and response performance to client IO requests declines
Solution Approach 1:
The patent introduces a journal storage apparatus as an intermediary component that is separate from the client IO receiving node. The journal entry storage unit stores journal entries including pseudo-timestamps in a dedicated journal storage area, allowing multiple nodes to access the journal without creating access conflicts that would otherwise impact client IO performance. This mediator approach enables consistency checking while preserving IO responsiveness.
2Reliability
If a journal is applied to metadata to prevent loss of file virtualization management information, then reliability is improved, but the complexity of the system increases
Solution Approach 1:
The patent segments the storage system into distinct functional units: a journal storage apparatus separate from the main storage apparatus, and a journal entry storage unit that specifically manages journal data. This segmentation allows the journaling function to be implemented independently, providing metadata protection without entangling the entire system in complex interdependencies. The separated architecture reduces overall system complexity while maintaining reliability.
3Reliability
If the storage destination of a journal differs from the client IO receiving node to ensure consistency, then data integrity is improved, but response performance to client IO requests declines
Solution Approach 1:
The patent introduces a dedicated journal entry storage unit as an intermediary between the client IO receiving node and the journal storage apparatus. This intermediary handles journal entry operations independently, allowing the client IO node to respond to requests without waiting for journal operations to complete. The separation ensures data integrity through proper journaling while maintaining fast IO response performance.
Data Source
AI summary
When an operation request of a file is made, a first storage system performs an operation of the file stored in the first storage system on the basis of the operation request, describes, in an operation log, a counter value of a management information file, associated with the file on which the operation is to be performed, together with operation content with respect to the file, replies to the operation request, and updates the counter value of the management information file when the file stored in the first storage system is transferred to a second storage system.