A RAID4 Data Layout Method Based on ZNS SSD

Abstract

The present invention relates to the technical field of computer storage system and file system data management, and provides a RAID4 data layout method based on ZNS SSD, including: 1. Initializing the system; 2. Dividing data blocks; 3. Mapping data blocks; 4. Judging whether there is enough space in the Zone to store the current data block; 5. Sequentially writing data blocks; 6. Calculating parity check blocks; 7. Judging whether the check blocks are successfully stored; 8. Establishing a block index table; 9. Verifying whether the stored data matches the calculated parity check data; 10. Completing data writing; 11. Judging whether a Zone to be adjusted is detected; 12. Judging whether there is a Zone with high-frequency writing that needs to be optimized; 13. Optimizing the Zone; 14. Monitoring data. The present invention can better optimize the performance, reliability and storage efficiency of traditional RAID4.

Description

Technical Field

[0001] The present invention relates to the technical field of computer storage system and file system data management, and in particular to a RAID4 data layout method based on ZNS (Zoned Namespaces) SSD. Background Art

[0002] With the deep integration and development of computer technology and network technology in various fields of society, the explosive growth of data volume has led to an increasing demand for high-performance storage. Traditional storage architectures are facing many challenges. The establishment of RAID technology can improve the performance and stability of storage systems, and improve the parallelism and expansion capabilities of I / O. Among them, RAID4 is a storage solution that provides data redundancy. It distributes data on multiple disks and stores parity data on a separate disk. This method enables the system to recover data when a single disk fails.

[0003] However, the traditional RAID4 system has certain performance bottlenecks, especially in high-concurrency write operations, the parity disk often becomes the bottleneck of system performance. In addition, the parity data of RAID4 is updated frequently, which easily causes the write amplification effect, further reducing the system performance and the service life of the SSD. Therefore, a RAID4 data layout method based on ZNSSSD is needed to solve the above problems. Summary of the invention

[0004] The present invention provides a RAID4 data layout method based on ZNS SSD, which aims to optimize the performance, reliability and storage efficiency of traditional RAID4.

[0005] According to a RAID4 data layout method based on a ZNS SSD of the present invention, the method comprises the following steps:

[0006] Step 1: Initialize the system;

[0007] Divide the SSD into multiple zones according to storage requirements;

[0008] Step 2: Data block division;

[0009] The original data is divided into multiple data blocks according to a fixed size. Each data block will be processed independently and written to different zones of ZNS SSD.

[0010] Step 3: Data block mapping;

[0011] Allocate the divided data blocks to different zones for storage;

[0012] Step 4: Determine whether the zone has enough space to store the current data block;

[0013] Read the size of the data block to be written, recorded as Size_block, and read the remaining capacity of the target zone from the zone metadata, recorded as Capacity_zone_remaining; if

[0014] Capacity_zone_remaining>=Size_block: Indicates that the zone has enough space to store the current data block, and the mapping and writing operations continue. If there is not enough space to store the current data block, a new zone is selected for mapping. When the current zone cannot accommodate the data block, the system selects another zone with enough free space to store the data block. After remapping, return to step 4.

[0015] Step 5: Write data blocks sequentially;

[0016] Write the data blocks into the mapped Zone in sequence;

[0017] Step 6: Parity block calculation;

[0018] Initialize a variable P to store the calculation result of the parity block, with an initial value of 0, and read data blocks from each zone in turn, denoted as D 1 ,D 2 ,D 3 ,...,D n , to ensure that no data is lost or damaged during the reading process; all read data blocks D are calculated according to the following formula i Perform a bitwise XOR operation:

[0019]

[0020] For each bit, the system compares the current value of P with the data block D i Perform XOR operation on the corresponding bits of to obtain the updated P;

[0021] Step 7: Determine whether the check block is stored successfully;

[0022] Read the parity block just written from the zone storing the parity block, denoted as P read , write to the specified parity zone; re-read all original data blocks D 1 ,D 2 ,...,D n According to the standard process of parity calculation, the bit-by-bit XOR operation is re-performed according to the following formula to calculate the new parity block P new :

[0023] P new =D 1 ⊕D 2 ⊕...⊕D n ,

[0024] P new With P read Perform a bit-by-bit comparison. If P new = =P read , indicating that the parity block storage is successful, and the process goes to step 8. Otherwise, it indicates that the parity block storage fails and needs to be rewritten. After rewriting, the process returns to step 6.

[0025] Step 8: Create a block index table;

[0026] The system generates a block index table to record the zone information of each data block and parity block; creates an empty block index table in the form of a hash table or tree structure; assigns a unique identifier Block ID to each data block to be written so that it can be identified in the index table, obtains the Block ID of the data block currently being written and the zone to which it belongs, obtains the offset of the data block in the zone and records it as Offset_1, and adds a record to the block index table: ID_1->(Zone_A,Offset_1). After each data block is written, the index table is updated immediately to record the new mapping relationship, and the index table is checked regularly to ensure that there are no errors or losses in the location records of the data blocks;

[0027] Step 9: Verify that the stored data matches the calculated parity data;

[0028] First, read all stored data blocks D 1 ,D 2 ,...,D n , read the corresponding parity check block P, that is, the previously calculated and stored check data, and then, according to the parity check calculation method, perform bit-by-bit XOR operation to recalculate the new parity check block P new , P new Compare with P bit by bit. If P new ==P, indicating that the stored data matches the calculated parity data, and the data is complete and correct, and then proceed to step 10. Otherwise, data recovery is performed, and the data blocks or parity blocks with mismatched data are identified. The parity data is tried for data recovery according to the following formula:

[0029] D error =P stored ⊕D 1 ⊕...⊕D err-1 ⊕D err+1 ⊕...⊕D n;

[0030] Step 10: Data writing is completed;

[0031] After the consistency check passes, the system records the completion status of this write operation and enters the data monitoring state;

[0032] Step 11: Determine whether a zone that needs to be adjusted is detected;

[0033] If zones that need to be adjusted are detected, the system marks these zones and prepares to perform adjustment operations. Otherwise, if all zones are in normal status, the current layout is maintained without adjustment.

[0034] Step 12: Determine whether there is a high-frequency write zone that needs to be optimized;

[0035] If it is determined that optimization is needed, prepare to perform the corresponding optimization operation; otherwise, record the current status and continue to monitor the write frequency;

[0036] Step 13: Zone optimization;

[0037] For zones with frequent writes, the system optimizes by adjusting mapping strategies, increasing redundancy, or migrating data.

[0038] Step 14: Data monitoring;

[0039] After the adjustment is completed, the system re-enters the monitoring state and continues to determine whether further optimization is needed based on the real-time situation.

[0040] Preferably, in the first step, a unique identifier ID is assigned to each Zone, and each Zone is marked as a data zone, a parity zone or a metadata zone.

[0041] Preferably, in the third step, the identifier of the data block is converted into the index value of the Zone through a hash function, thereby determining the storage location of the data block; the hash function needs to dynamically determine the specific data block allocation based on the current usage of the Zone and the remaining capacity factor.

[0042] As a preference, in step 11, the judgment condition is:

[0043] Condition 1: If the available space of a zone is lower than the set minimum threshold and there is no other spare space, the zone needs to be adjusted;

[0044] Condition 2: If a zone is written too many times and has a hotspot, causing accelerated wear, consider reallocating zones or balancing data;

[0045] Condition 3: If the wear level of a zone is close to the critical value and its service life is about to be exhausted, it is recommended to reallocate or migrate data.

[0046] Preferably, in step 12, the judgment condition is:

[0047] Condition a: If the high-frequency writing zone is accompanied by a high wear rate, optimization must be performed to prevent the zone from failing prematurely;

[0048] Condition b: If high-frequency writing causes system performance degradation, such as an I / O bottleneck, data migration or load balancing should be considered;

[0049] Condition c: If the frequently written zones still have sufficient remaining life and system performance is not affected, optimization may not be needed for the time being, but should be closely monitored.

[0050] The beneficial effects of the present invention are as follows:

[0051] (1) Improve system performance and storage efficiency: The present invention significantly improves the overall performance of the storage system by combining the sequential write characteristics of ZNS SSD and the optimized data striping strategy. Sequential writing significantly reduces the write amplification effect, making the data writing process more efficient and reducing the latency of write operations. In addition, the optimized data striping layout ensures the uniform distribution of data on multiple SSDs, avoids single-point bottlenecks, and further improves the throughput and response speed of the system. By reducing random write operations and rationally planning data distribution, the present invention improves the utilization of storage space, reduces space waste, and thus enhances the storage efficiency of the system.

[0052] (2) Enhanced data reliability and fault recovery capabilities: The present invention has made significant optimizations in data redundancy and verification management, ensuring that the system can quickly recover data when a hardware failure occurs. Through the verification data management mechanism of RAID4, the system can use the stored verification data to quickly rebuild the lost data blocks when a single SSD fails, ensuring data integrity and security. In addition, the optimized fault detection and automated recovery functions enable the system to immediately start the recovery process when a fault is detected, reducing downtime and ensuring business continuity. These improvements have greatly enhanced the overall reliability of the system, making it more robust and faster in the face of hard drive failures.

[0053] (3) Extending the service life of SSDs: Since the present invention reduces the write amplification effect and random write operations, the wear rate of SSDs is significantly reduced. The sequential write characteristics and optimized data layout of ZNS SSDs reduce frequent data movement and garbage collection operations, thereby extending the service life of SSDs. This improvement not only improves the durability of SSDs and reduces the frequency and cost of hardware replacement, but also improves the long-term stability and reliability of the entire storage system, especially in data-intensive application scenarios.

[0054] (4) Simplify data management and improve system scalability: The present invention significantly simplifies system management and maintenance by optimizing data management and storage architecture design. The improved verification data management mechanism and automated data recovery function make system maintenance more efficient and reduce the need for manual intervention. In addition, the solution optimizes the regional management of ZNS SSD, making the distribution of data on the SSD more reasonable, which helps to reduce management complexity. At the same time, the present invention also enhances the scalability of the system and can flexibly respond to future growth and changes in storage needs. By rationally planning storage resources, the system can easily expand capacity and performance to adapt to changing business needs and maintain long-term efficient operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of a RAID4 data layout method based on ZNS SSD in an embodiment;

[0056] Figure 2 It is an architectural diagram of a RAID4 data layout method based on ZNS SSD in an embodiment. DETAILED DESCRIPTION

[0057] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0058] Example

[0059] like Figure 1 As shown, this embodiment provides a RAID4 data layout method based on ZNS SSD, which includes the following steps:

[0060] Step 1: Initialize the system;

[0061] According to storage requirements, the SSD is divided into multiple zones; each zone is assigned a unique identifier (ID), and each zone is marked as a data zone, a parity zone, or a metadata zone.

[0062] Step 2: Data block division;

[0063] The original data is divided into multiple data blocks according to a fixed size (for example, 4KB or 8KB), and each data block will be processed independently and written to a different zone of the ZNS SSD.

[0064] Step 3: Data block mapping;

[0065] Allocate the divided data blocks to different zones for storage; convert the data block identifier (such as block ID) into the zone index value through the hash function to determine the storage location of the data block; the hash function needs to dynamically determine the specific data block allocation based on factors such as the current usage of the zone and the remaining capacity.

[0066] Step 4: Determine whether the zone has enough space to store the current data block;

[0067] Read the size of the data block to be written, recorded as Size_block, and read the remaining capacity of the target zone from the zone metadata, recorded as Capacity_zone_remaining; if

[0068] Capacity_zone_remaining>=Size_block: Indicates that the Zone has enough space to store the current data block and continues to perform mapping and writing operations. If there is not enough space to store the current data block, select a new Zone for mapping. When the current Zone cannot accommodate the data block, the system selects another Zone with sufficient available space to store the data block; after remapping, return to step 4.

[0069] Step 5: Write data blocks sequentially;

[0070] Write data blocks to the mapped zones in sequence to ensure the sequential nature of write operations, reduce write amplification effects, and optimize performance.

[0071] Step 6: Parity block calculation;

[0072] Initialize a variable P to store the calculation result of the parity block, with an initial value of 0, and read data blocks from each zone in turn, denoted as D 1 ,D 2 ,D 3 ,...,D n , to ensure that no data is lost or damaged during the reading process; all read data blocks D are calculated according to the following formula i Perform a bitwise XOR operation:

[0073] P=D 1 ⊕D 2 ⊕D 3 ⊕...⊕D n

[0074] For each bit, the system compares the current value of P with the data block D i Perform XOR operation on the corresponding bits of to obtain the updated P.

[0075] Step 7: Determine whether the check block is stored successfully;

[0076] Read the parity block just written from the zone storing the parity block, denoted as P read , write to the specified parity zone to ensure the integrity of the write operation, avoid partial write or data inconsistency, and improve fault tolerance; re-read all original data blocks D 1 ,D 2 ,...,D n According to the standard process of parity calculation, the bit-by-bit XOR operation is re-performed according to the following formula to calculate the new parity block P new :

[0077]

[0078] P new With P read Perform a bit-by-bit comparison. If P new = =P read , indicating that the parity block storage is successful, and the process goes to step 8. Otherwise, it indicates that the parity block storage fails and needs to be rewritten. After rewriting, the process returns to step 6.

[0079] Step 8: Create a block index table;

[0080] The system generates a block index table to record the Zone information of each data block and parity block; creates an empty block index table in the form of a hash table or tree structure for quick query; assigns a unique identifier (Block ID) to each data block to be written so that it can be identified in the index table, obtains the BlockID of the data block currently being written (such as ID_1) and its Zone (such as Zone_A), obtains the offset of the data block in the Zone and records it as Offset_1, and adds a record to the block index table: ID_1->(Zone_A,Offset_1). After each data block is written, the index table is updated immediately to record the new mapping relationship, and the index table is checked regularly to ensure that there are no errors or losses in the location records of the data blocks.

[0081] Step 9: Verify that the stored data matches the calculated parity data;

[0082] First, read all stored data blocks D 1 ,D 2,...,D n , read the corresponding parity check block P, that is, the previously calculated and stored check data, and then, according to the parity check calculation method, perform bit-by-bit XOR operation to recalculate the new parity check block P new , P new Compare with P bit by bit. If P new ==P, indicating that the stored data matches the calculated parity data, and the data is complete and correct, and then proceed to step 10. Otherwise, data recovery is performed, and the data blocks or parity blocks with mismatched data are identified. The parity data is tried for data recovery according to the following formula:

[0083] D error =P stored ⊕D 1 ⊕...⊕D err-1 ⊕D err+1 ⊕...⊕D n .

[0084] Step 10: Data writing is completed;

[0085] After the consistency check passes, the system records the completion status of this write operation and enters the data monitoring state.

[0086] Step 11: Determine whether a zone that needs to be adjusted is detected;

[0087] The judgment conditions are:

[0088] Condition 1: If the available space of a zone is lower than the set minimum threshold and there is no other spare space, the zone needs to be adjusted;

[0089] Condition 2: If a zone is written too many times and has a hotspot, causing accelerated wear, consider reallocating zones or balancing data;

[0090] Condition 3: If the wear level of a zone is close to the critical value and its service life is about to be exhausted, it is recommended to reallocate or migrate data.

[0091] If zones that need to be adjusted are detected, the system marks these zones and prepares to perform adjustment operations. Otherwise, if all zones are in normal status, the current layout is maintained without adjustment.

[0092] Step 12: Determine whether there is a high-frequency write zone that needs to be optimized;

[0093] The judgment conditions are:

[0094] Condition a: If the high-frequency writing zone is accompanied by a high wear rate, optimization must be performed to prevent the zone from failing prematurely;

[0095] Condition b: If high-frequency writing causes system performance degradation, such as an I / O bottleneck, data migration or load balancing should be considered;

[0096] Condition c: If the frequently written zones still have sufficient remaining life and system performance is not affected, optimization may not be needed for the time being, but should be closely monitored.

[0097] If it is determined that optimization is needed, prepare to perform the corresponding optimization operation; otherwise, record the current status and continue to monitor the write frequency.

[0098] Step 13: Zone optimization;

[0099] For zones that are frequently written, the system optimizes by adjusting mapping strategies, increasing redundancy, or migrating data to ensure the persistence and overall performance of the zone.

[0100] Step 14: Data monitoring;

[0101] After the adjustment is completed, the system re-enters the monitoring state and continues to determine whether further optimization is needed based on the real-time situation.

[0102] This embodiment discloses a novel ZNS SSD RAID4 data layout solution. The entire data layout process is divided into four processes: data division, zone division, data migration and mapping, and parity block recalculation and storage.

[0103] Data division: First, the original data is divided into fixed-size data blocks. This process is crucial in the entire data layout because it determines how the data is stored and managed. In the ZNS SSD architecture, the size and division of data blocks will directly affect the sequentiality and storage efficiency of writing. The data is divided into several small, manageable data blocks. These data blocks will be written to each zone in a certain logical order.

[0104] Zone division: The physical storage space of ZNS SSD is divided into multiple zones, each zone is equivalent to an independent logical storage unit. The data in each zone must be written in sequence to maximize storage efficiency and reduce write amplification effect. Figure 2 As shown in the ZNS SSD data layout, different application data blocks are allocated to different zones. The data blocks in each zone are written sequentially to ensure data consistency and efficiency. The three zones in the figure show the data block layout of applications 1, 2, and 3 respectively.

[0105] Data migration and mapping process: Migrate data from traditional SSDs to ZNS SSDs, and remap data blocks according to the zone division. During the mapping process, it is necessary to ensure that each data block is stored in order within the zone to maintain data consistency and reliability. The migration arrows (from left to right) in the figure represent the data migration process from traditional SSDs to ZNS SSDs. The data blocks originally scattered in the FTL layer are remapped to the zones of the ZNS SSD. ZNS SSD data is grouped based on applications, which simplifies data management and improves storage efficiency.

[0106] Parity block recalculation and storage: After the data blocks are divided and migrated to the ZNS SSD, the parity blocks need to be recalculated to ensure the integrity and recoverability of the data. The recalculated parity blocks are stored in a specific zone for recovery when the data is damaged. The dark blocks in the figure represent parity blocks. In the ZNS SSD architecture, parity blocks are centrally stored in a specific zone to facilitate subsequent data verification and recovery operations.

[0107] This embodiment achieves more efficient storage management, longer SSD service life, and more reliable data protection and recovery capabilities by optimizing data striping, verification data management, fault recovery, and system scalability.

[0108] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. A RAID4 data layout method based on ZNS SSD, characterized by: The following steps are involved: Step 1: Initialize the system; Divide the SSD into multiple zones according to storage requirements; Step 2: Data block division; The original data is divided into multiple data blocks according to a fixed size. Each data block will be processed independently and written to different zones of ZNSSSD. Step 3: Data block mapping; Allocate the divided data blocks to different zones for storage; Step 4: Determine whether the zone has enough space to store the current data block; Read the size of the data block to be written, recorded as Size_block, and read the remaining capacity of the target zone from the zone metadata, recorded as Capacity_zone_remaining; if Capacity_zone_remaining>=Size_block: it means that the zone has enough space to store the current data block, and continue to perform mapping and writing operations. If there is not enough space to store the current data block, select a new zone for mapping. When the current zone cannot accommodate the data block, the system selects another zone with enough free space to store the data block; after remapping, return to step 4; Step 5: Write data blocks sequentially; Write the data blocks into the mapped Zone in sequence; Step 6: Parity block calculation; Initialize a variable P to store the calculation result of the parity block, with an initial value of 0, and read data blocks from each zone in turn, denoted as D1, D2, D3, ..., D n , to ensure that no data is lost or damaged during the reading process; all read data blocks D are calculated according to the following formula i Perform a bitwise XOR operation: P=D1⊕D2⊕D3⊕...⊕D n For each bit, the system compares the current value of P with the data block D i Perform XOR operation on the corresponding bits of to obtain the updated P; Step 7: Determine whether the check block is stored successfully; Read the parity block just written from the zone storing the parity block, denoted as P read , write to the specified parity zone; re-read all original data blocks D1, D2, ..., D n According to the standard process of parity calculation, the bit-by-bit XOR operation is re-performed according to the following formula to calculate the new parity block P new : P new =D1⊕D2⊕...⊕D n , P new With P read Perform a bit-by-bit comparison. If P new = =P read , indicating that the parity block storage is successful, and the process goes to step 8. Otherwise, it indicates that the parity block storage fails and needs to be rewritten. After rewriting, the process returns to step 6. Step 8: Create a block index table; The system generates a block index table to record the zone information of each data block and parity block; Create an empty block index table, implemented as a hash table or tree structure; Assign a unique identifier BlockID to each data block to be written so that it can be identified in the index table. Obtain the Block ID of the data block currently being written and the Zone to which it belongs. Obtain the offset of the data block in the Zone and record it as Offset_1. Add a record to the block index table: ID_1->(Zone_A,Offset_1). After each data block is written, update the index table immediately to record the new mapping relationship. Check the index table regularly to ensure that there is no error or loss in the location record of the data block. Step 9: Verify that the stored data matches the calculated parity data; First, read all stored data blocks D1, D2, ..., D n , read the corresponding parity check block P, that is, the previously calculated and stored check data, and then, according to the parity check calculation method, perform bit-by-bit XOR operation to recalculate the new parity check block P new , P new Compare with P bit by bit. If P new ==P, indicating that the stored data matches the calculated parity data, and the data is complete and correct, and then proceed to step 10. Otherwise, data recovery is performed, and the data blocks or parity blocks with mismatched data are identified. The parity data is tried for data recovery according to the following formula: D error =P stored ⊕D1⊕...⊕D err-1 ⊕D err+1 ⊕...⊕D n ; Step 10: Data writing is completed; After the consistency check passes, the system records the completion status of this write operation and enters the data monitoring state; Step 11: Determine whether a zone that needs to be adjusted is detected; If zones that need to be adjusted are detected, the system marks these zones and prepares to perform adjustment operations. Otherwise, if all zones are in normal status, the current layout is maintained without adjustment. Step 12: Determine whether there is a high-frequency write zone that needs to be optimized; If it is determined that optimization is needed, prepare to perform the corresponding optimization operation; otherwise, record the current status and continue to monitor the write frequency; Step 13: Zone optimization; For zones with frequent writes, the system optimizes by adjusting mapping strategies, increasing redundancy, or migrating data. Step 14: Data monitoring; After the adjustment is completed, the system re-enters the monitoring state and continues to determine whether further optimization is needed based on the real-time situation.

2. According to claim 1, a RAID4 data layout method based on ZNS SSD is characterized in that: In the first step, a unique identifier ID is assigned to each zone, and each zone is marked as a data zone, a parity zone, or a metadata zone.

3. A RAID4 data layout method based on ZNS SSD according to claim 2, characterized in that: In the third step, the data block identifier is converted into the zone index value through the hash function to determine the storage location of the data block; the hash function needs to dynamically determine the specific data block allocation based on the current usage of the zone and the remaining capacity factors.

4. The RAID4 data layout method based on ZNS SSD according to claim 3, characterized in that: In the eleventh step, the judgment conditions are: Condition 1: If the available space of a zone is lower than the set minimum threshold and there is no other spare space, the zone needs to be adjusted; Condition 2: If a zone is written too many times and has a hotspot, causing accelerated wear, consider reallocating zones or balancing data; Condition 3: If the wear level of a zone is close to the critical value and its service life is about to be exhausted, it is recommended to reallocate or migrate data.

5. The RAID4 data layout method based on ZNS SSD according to claim 4, characterized in that: In step 12, the judgment condition is: Condition a: If the high-frequency writing zone is accompanied by a high wear rate, optimization must be performed to prevent the zone from failing prematurely; Condition b: If high-frequency writing causes system performance degradation, such as an I / O bottleneck, data migration or load balancing should be considered; Condition c: If the frequently written zones still have sufficient remaining life and system performance is not affected, optimization may not be needed for the time being, but should be closely monitored.

Images