A RAID-5 optimization method based on high-speed write request log

By introducing a high-performance solid-state disk as a log disk in disk RAID-5, recording write requests and optimizing stripe verification, the problem of high write latency of traditional disk RAID-5 is solved, and faster response time and higher data security.

CN118672500BActive Publication Date: 2024-11-22QINGHAI NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410690745.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-30
Publication Date
2024-11-22
Estimated Expiration
2044-05-30

AI Technical Summary

Technical Problem

Traditional disk RAID-5 systems have high latency when writing requests, which cannot meet the needs of high-speed data access, and are low in cost, but their performance is not comparable to that of solid-state disk RAID-5.

Method used

Introducing a high-performance solid-state disk as a log disk in disk RAID-5, logging write requests and treating it as completion before striping verification, reducing latency and allowing log disks to be responsible for multiple disk arrays, reducing costs.

Benefits of technology

It significantly reduces the write request response time, improves data consistency and recovery capabilities, is suitable for high-load application scenarios, and reduces the risk of data loss.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118672500B_ABST
    Figure CN118672500B_ABST
Patent Text Reader

Abstract

The present invention discloses a RAID‑5 optimization method based on a high-speed write request log. A log disk is added to the disk RAID‑5 to optimize the reliability and write delay of the disk RAID‑5. The reliability of the disk RAID‑5 can be improved by using log technology. When the system is powered off or crashes, the incompletely written data can be restored by traversing the log metadata; wherein the data with a persistence flag of 0 is the incompletely written data, and the data consistency of the disk RAID‑5 can be ensured by re-executing these requests. At the same time, the present invention uses a high-performance solid-state disk as a log disk, and the log disk quickly responds to write requests to reduce the delay of data verification and the write delay of the disk RAID‑5.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of high-performance storage system write request, and in particular to a RAID-5 optimization method based on a high-speed write request log. Background Art

[0002] With the rapid development of information technology, data storage and processing are becoming increasingly important. Enterprises and organizations need to store large amounts of data and be able to access this data quickly and reliably. However, traditional single hard disk storage systems may not meet the requirements in terms of performance, capacity, and data redundancy, so more advanced storage technologies are needed to solve these problems. RAID-5 technology came into being. By combining multiple hard disk drives, it improves the performance, capacity, and reliability of the storage system and has become one of the commonly used storage solutions for enterprises and individual users. Compared with solid-state drives, the manufacturing cost of disks is relatively low, so the total cost of building a disk RAID-5 system is usually lower. This makes disk RAID-5 an affordable option, especially for scenarios with large storage requirements. However, due to the relatively slow read and write speed of disks, especially in random read and write operations, the performance of disk RAID-5 may not be comparable to that of solid-state drive RAID-5. This means that in application scenarios that require high-speed data access, disk RAID-5 may not perform well. There are two reasons for the high latency of disk RAID-5: (1) The disk itself has a slow write speed and (2) the stripe verification operation will generate a lot of latency. Therefore, there is an urgent need for an optimization method for a low-latency RAID-5 solution that can solve the above shortcomings and is low-cost. Summary of the invention

[0003] To solve the problems existing in the prior art, the present invention provides a RAID-5 optimization method based on a high-speed write request log. A high-performance solid-state disk (NVMe SSD) is added to the original RAID-5 design as a log disk to record write requests. The high-performance solid-state disk has the characteristic of low latency. If the write request is written to the log disk before the user's write request is striped and verified and the request is regarded as completed, the latency of the disk RAID-5 can be effectively reduced, and the log disk can be responsible for multiple groups of disk arrays without a significant impact on the cost, thereby solving the problems mentioned in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a RAID-5 optimization method based on a high-speed write request log, comprising the following steps:

[0005] S1. A log disk is added to the disk RAID-5 to monitor and record write requests;

[0006] S2, initializing the log disk system, including initializing the log space size, log offset, and log metadata quantity information;

[0007] S3, obtaining the write request offset, the write request length, and basic information of the write data written by the write request, and calculating the stripe accessed by the write request according to the basic information of the write request;

[0008] S4. Read the data required for updating the parity check from the stripe according to the stripe check calculation rule, and convert the basic information of the write request into log metadata information;

[0009] S5, determining whether the log metadata has reached the upper limit or the log space is insufficient, if the log metadata has reached the upper limit or the log space is insufficient, first clearing the metadata and log space; otherwise, writing log data to the log disk according to the log metadata;

[0010] S6. After the log data is written, the user is sent a request completion message.

[0011] S7. Execute stripe checksum calculation to calculate the updated check block and checksum data, write the calculated checksum data and write request data into the disk array, and after writing, update the persistence flag of the metadata to 1, indicating that the current data has been updated into the array. When the log space is full, the log metadata is cleared first to complete the write request optimization.

[0012] Preferably, the initialization of the log disk system refers to: defining a log space of size 2G; defining a log metadata quantity threshold, set to 256, and initializing the log metadata quantity to 0, indicating the current log metadata quantity; initializing the log offset to 0, indicating the amount of data that has been written to the current log space.

[0013] Preferably, the log metadata includes four fields, namely log data, log data offset, persistence flag, and log data length; the log data is the same as the write request data, the log data offset is obtained by the dynamic log offset, the persistence flag is initially 0, and the log data length is the same as the write request data length.

[0014] Preferably, the determining whether the log metadata reaches the upper limit or the log space is insufficient specifically refers to: determining whether the number of log metadata is less than 256, and whether the remaining log space is less than 80% of the total space.

[0015] Preferably, each time another new log metadata information is converted, the log metadata quantity is increased by 1 to the original quantity; the new log offset is equal to the original log offset + the write request length.

[0016] Preferably, after cleaning up the metadata and log space, each time a log data is cleaned up, the metadata quantity is reduced by 1 and the corresponding log space is released, and the log data offset in the current log metadata is updated at the same time.

[0017] The beneficial effects of the present invention are:

[0018] 1) Faster write request response time: The present invention significantly reduces the time users have to wait for write operations to complete. By first recording the write request to a log disk with low latency characteristics, the write request can be quickly considered "completed", while the background stripe verification and actual data writing to the disk array can be performed asynchronously, so as not to directly affect the user's operating experience. This fast response is particularly important for application scenarios that require a large number of write operations, such as online trading platforms, big data analysis, etc.

[0019] 2) Provides a safer guarantee for data: Using log disks to record write requests enhances data consistency and recovery capabilities. In the event of a failure, the data on the log disk can be used to quickly restore to the last consistent state, reducing the risk of data loss. In addition, the log mechanism allows the system to reconstruct lost write operations by replaying logs after an abnormal shutdown or system failure, ensuring data integrity and consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the system architecture of a RAID-5 optimization method based on a high-speed write request log in an embodiment of the present invention;

[0021] Figure 2 The figure is a flowchart of the steps of a RAID-5 optimization method based on a high-speed write request log in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] See also Figure 1-Figure 2 The present invention provides a technical solution: a RAID-5 optimization method based on a high-speed write request log, in which a log disk is added to the disk RAID-5, such as Figure 1As shown, it is used to optimize the reliability and write delay of disk RAID-5. When the system is powered off or crashes, the incompletely written data can be recovered by traversing the log metadata; among them, the data with a persistence flag of 0 is the incompletely written data. Re-executing these requests can ensure the data consistency of disk RAID-5, thereby improving the reliability of disk RAID-5. At the same time, the present invention uses a high-performance solid-state disk as a log disk. The log disk quickly responds to write requests to reduce the delay of data verification and the write delay of disk RAID-5. The optimization method, the steps are as follows Figure 2 As shown, including the following:

[0024] S1. A log disk is added to the disk RAID-5 to monitor and record write requests;

[0025] S2. Initialize the log disk system, including initializing the log space size, log offset, and log metadata quantity information; customize a log space size, usually 2GB; customize a log metadata quantity threshold, usually 256; initialize the log offset to 0, indicating the amount of data already written to the current log space. Initialize the log metadata quantity to 0, indicating the current log metadata quantity.

[0026] S3, obtaining the write request offset, the write request length, and basic information of the write data written by the write request, and calculating the stripe accessed by the write request according to the basic information of the write request;

[0027] S4. Read the data required for updating the parity check from the stripe according to the stripe check calculation rule, and convert the basic information of the write request into log metadata information;

[0028] Log metadata includes four fields: log data, log data offset, persistence flag, and log data length; log data indicates the data to be written to the log disk; log data offset indicates the starting offset of the data in the log; persistence flag indicates whether the data in the current write request has been written to the disk array; log data length indicates the length of the data written to the log disk.

[0029] The log data is the same as the write request data, the log data offset is obtained from the dynamic log offset, and is the same as the current log offset; the persistence flag is initially 0, and the log data length is the same as the write request data length.

[0030] Thus, a log metadata information is constructed. Whenever another new log metadata information is converted, the number of log metadata is increased by 1. The new log offset is equal to the original log offset + write request length.

[0031] S5, determining whether the log metadata has reached the upper limit or the log space is insufficient, if the log metadata has reached the upper limit or the log space is insufficient, first clearing the metadata and log space; otherwise, writing log data to the log disk according to the log metadata;

[0032] Determining whether the log metadata has reached the upper limit or the log space is insufficient specifically refers to: determining whether the number of log metadata is less than 256, and whether the remaining log space is less than 80% of the total space.

[0033] If the number of log metadata is greater than 256 or the log offset is greater than 1.6GB, it means that the number of log metadata has reached the upper limit or the log space is full, and you need to clear the log data with a persistence flag of 1 to free up log space. After cleaning the metadata and log space, each time a log data is cleaned up, the number of metadata is reduced by 1 and the corresponding log space is released, and the log data offset in the current log metadata is updated at the same time.

[0034] If the number of log metadata is less than 256 and the log offset is less than 1.6GB, it means that the number of log metadata has not reached the upper limit and the log space is not full, so data is written to the log disk.

[0035] S6. After the log data is written, the user is sent a request completion message.

[0036] S7, perform stripe check calculation, calculate the updated check block and check data, write the calculated check data and write request data into the disk array, and after writing, update the persistence flag of the metadata to 1, indicating that the current data has been updated into the array. When the log space is full, the log metadata is cleared first. When the next read request arrives, the operation is still performed according to steps S2 to S7 to complete the write request optimization.

[0037] The present invention uses a log disk to record write requests, which enhances data consistency and recovery capabilities. In the event of a failure, the data on the log disk can be used to quickly restore to the last consistent state, reducing the risk of data loss. In addition, the log mechanism allows the system to reconstruct lost write operations by replaying logs after an abnormal shutdown or system failure, ensuring data integrity and consistency.

[0038] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A RAID-5 optimization method based on high-speed write request log, It is characterized in that The steps include: S1. A log disk is added to the disk RAID-5 to monitor and record write requests. The log disk is a high-performance solid-state disk. S2, initializing the log disk system, including initializing the log space size, log offset, and log metadata quantity information; S3, obtaining the write request offset, the write request length, and basic information of the write data written by the write request, and calculating the stripe accessed by the write request according to the basic information of the write request; S4. Read the data required for updating the parity check from the stripe according to the stripe check calculation rule, and convert the basic information of the write request into log metadata information; S5, determining whether the log metadata has reached the upper limit or the log space is insufficient, if the log metadata has reached the upper limit or the log space is insufficient, first clearing the metadata and log space; Otherwise, the log data is written to the log disk according to the log metadata; S6. After the log data is written, the user is sent a request completion message. S7, perform stripe checksum calculation, calculate the updated check block and checksum data, write the calculated checksum data and write request data into the disk array, and after writing, update the persistence flag of the metadata to 1, indicating that the current data has been updated into the array. When the log space is full, the log metadata is cleared first to complete the write request optimization; Initializing the log disk system means: defining a log space of 2G; defining a log metadata quantity threshold value, set to 256, and initializing the log metadata quantity to 0, indicating the current log metadata quantity; The initial log offset is 0, which indicates the amount of data that has been written to the current log space; The log metadata includes four fields, namely log data, log data offset, persistence flag, and log data length; the log data is the same as the write request data, the log data offset is obtained by the dynamic log offset, the persistence flag is initially 0, and the log data length is the same as the write request data length; Determining whether the log metadata has reached the upper limit or the log space is insufficient specifically refers to: determining whether the number of log metadata is less than 256, and whether the remaining log space is less than 80% of the total space; Whenever another new log metadata information is converted, the number of log metadata is increased by 1 to the original number; the new log offset is equal to the original log offset + write request length; After cleaning the metadata and log space, each time a log data is cleaned up, the metadata quantity is reduced by 1 and the corresponding log space is released, and the log data offset in the current log metadata is updated at the same time.