Client Server Data Fragment Appending to Reduce Storage Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, the '0' filling operation required for incomplete data stripes leads to inefficient use of hard disk space and increases the total number of stripes needed, resulting in higher storage overhead and management complexity.
Innovation Solution
A data writing method that reduces the total number of stripes by writing data fragments into existing stripes based on available space, using write location information to determine the appropriate location within a stripe, thereby eliminating the need for '0' filling and optimizing storage utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If '0' filling operation is performed to complete data fragments for full-stripe write, then data can be stored in complete stripes, but hard disk space is wasted and storage overhead increases
Solution Approach 1:
The patent applies partial action by allowing incomplete data fragments to be written directly without requiring full-stripe completion. Instead of filling with '0's to create complete stripes, the system writes only the actual data fragments needed, reducing unnecessary storage overhead while maintaining reliability through erasure coding.
Solution Approach 2:
The patent extracts the unnecessary '0' filling operation from the data writing process. By removing this redundant step, the system eliminates wasted hard disk space and reduces storage overhead while still achieving reliable data storage through the erasure coding mechanism.
2Ease of manufacture
If '0' filling operation is performed to obtain complete data fragments, then data can be written into storage nodes, but total quantity of stripes needed increases
Solution Approach 1:
The patent uses partial action by allowing data to be written in incomplete stripes without requiring full-stripe completion. This reduces the total quantity of stripes needed compared to traditional methods that require '0' filling to create complete stripes, thereby reducing storage overhead.
Solution Approach 2:
The patent removes the unnecessary '0' filling operation that causes inflation of stripe quantity. By extracting this redundant step, the system achieves more efficient stripe utilization and reduces the total number of stripes required for data storage.
3Reliability
If data is divided into complete fragments for full-stripe write, then erasure coding can be applied, but data that does not meet full-stripe condition requires '0' filling
Solution Approach 1:
The patent introduces dynamic adaptability in the data writing process. Instead of rigidly requiring full-stripe completion with '0' filling, the system dynamically adjusts to handle incomplete data fragments directly, simplifying the writing process while maintaining erasure coding reliability.
Solution Approach 2:
The patent changes the parameter of stripe completion requirement from mandatory full-stripe to flexible partial-stripe writing. This parameter change allows the system to handle various data sizes without requiring '0' filling, simplifying the data writing process while maintaining reliability through erasure coding.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed are a data writing method, a client server, and a system. The method includes: receiving, by a client server, first data; obtaining write location information of a target stripe, where a data fragment of other data has been written in the target stripe, and the write location information indicates a location that is of the written data fragment in the target stripe and that is in the target stripe; and writing a data fragment of a different piece of data into the target stripe according to the location in the target stripe in an appending manner, to reduce a total quantity of stripes needed by data.