Solid state disk data protection and recovery system

Abstract

The invention belongs to the technical field of storage systems, and particularly discloses a solid state disk data protection and recovery system. The system adopts a full flash memory array architecture and comprises a main control disk and a data disk, and hardware-level data protection is realized through data fragmentation and redundancy coding. The system is provided with a dynamic write-in scheduling layer, the SSD health state is monitored in real time, and a health degree scoring model is constructed based on monitoring data. And the data protection module adopts an intelligent write-in scheduling algorithm, a triple data protection layer and self-adaptive garbage collection optimization. And the data recovery module supports a multi-level recovery mechanism, including instantaneous recovery, rapid recovery and complete recovery. In addition, the system further has the functions of fault prediction, atomic write transaction guarantee, wear-leveling dynamic adjustment, read-only mode switching and the like. The system can effectively protect the solid state disk data, improves the data recovery efficiency and reliability, and is suitable for application scenes needing high data protection and high recovery speed.

Description

Technical Field

[0001] The present invention relates to the technical field of storage systems, and in particular to a solid state hard disk data protection and recovery system. Background Art

[0002] With the rapid development of information technology, solid-state drives (SSDs) have gradually become the mainstream choice in the field of data storage due to their high performance and low latency. However, in practical applications, SSDs face problems such as insufficient data reliability, degraded write performance, and coarse recovery granularity, which seriously affect the stability and availability of data storage systems.

[0003] In traditional storage systems, the failure of a single hard drive often results in data loss. Although RAID (Redundant Array of Independent Disks) technology improves data reliability through data redundancy, the RAID reconstruction process is usually very time-consuming in SSD arrays, especially in large-scale data environments. Once an SSD fails, the time required for reconstruction may be as long as several hours, which not only affects the availability of the system, but also increases the risk of further data loss. In addition, as the capacity of SSDs increases, the amount of data loss that may be caused by a single disk failure also increases, which poses a higher challenge to data protection.

[0004] The write performance of SSDs is significantly affected by garbage collection (GC) and the write amplification effect. Garbage collection is an important mechanism within SSDs that is used to reclaim space occupied by invalid data, but frequent garbage collection will increase additional write operations, thereby reducing write performance. The write amplification effect refers to the fact that the amount of data actually written to the SSD is much larger than the amount of data requested by the user, which not only accelerates the wear of the SSD but also shortens its service life. In traditional SSD management strategies, there is a lack of effective mechanisms to optimize garbage collection and reduce the write amplification effect, which leads to a gradual decline in write performance.

[0005] In terms of data recovery, traditional storage systems usually rely on backup and snapshot technologies. However, the recovery granularity of these technologies is coarse and cannot meet the needs of key businesses for instant recovery capabilities. In the event of system failure or data loss, traditional recovery methods often take a long time to restore data, which not only affects business continuity but may also cause huge economic losses. Especially in industries such as finance and medical care that have extremely high requirements for data real-time and integrity, coarse-grained recovery capabilities have become a key bottleneck restricting business development.

[0006] In summary, existing SSD storage systems have many deficiencies in data reliability, write performance, and recovery granularity. Summary of the invention

[0007] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a solid state drive data protection and recovery system to solve the problems in the prior art such as easy data loss, difficulty in recovery, imperfect protection mechanism, and difficulty in balancing performance and security.

[0008] To achieve the above-mentioned and other related purposes, the present invention provides a solid-state drive data protection and recovery system, including: an all-flash array architecture module, which is composed of multiple solid-state drives, at least one of which is configured as a master control drive for metadata management, and the remaining solid-state drives constitute data drives, and hardware-level data protection is achieved through data sharding and redundant coding; The dynamic write scheduling layer includes an SSD health status perception module, which monitors the remaining life, bad block rate, garbage collection pressure, and queue depth of each SSD in real time, and builds an SSD health scoring model based on the monitoring data; Data protection module, including intelligent write scheduling algorithm, triple data protection layer and adaptive garbage collection optimization module. The intelligent write scheduling algorithm selects the target storage unit based on the SSD health score and real-time queue depth. The triple data protection layer consists of a memory mirror layer, a cross-SSD real-time mirror layer and an incremental snapshot layer. The data recovery module supports multi-level recovery mechanisms, including instant recovery based on memory mirroring, fast recovery across SSD mirroring, and complete recovery combining redundant coding and incremental snapshots.

[0009] Optionally, the redundant coding includes at least one of RAID 5, RAID 6 or erasure coding, and consistency management is achieved between the master disk and the data disk through a distributed metadata synchronization protocol.

[0010] Optionally, the execution logic of the intelligent write scheduling algorithm includes: Select SSDs whose health scores are higher than the preset threshold and whose garbage collection pressure is lower than the upper limit as candidate storage units; Select the SSD with the smallest queue depth among the candidate units as the write target; If the candidate cell is empty, the degraded protection mode is triggered, user writing is suspended and data migration is started.

[0011] Optionally, in the triple data protection layer: The memory mirroring layer uses NVDIMM to store real-time data copies, and uses supercapacitors to ensure data persistence during power outages; Perform at least two independent SSD write operations for each write request across the SSD real-time mirroring layer; The incremental snapshot layer records data changes in a log structure and generates differential snapshots with configurable time intervals.

[0012] Optionally, the adaptive garbage collection optimization module performs the following operations: Actively trigger full disk garbage collection during SSD idle time; A lazy recycling strategy is used during high-load periods, where only blocks that meet the minimum recycling threshold are recycled; Reserve no less than 25% of the reserved space to mitigate the write amplification effect.

[0013] Optionally, the data recovery module further includes: The fault prediction submodule monitors the bad block growth rate and erase / write cycle usage rate, marks high-risk SSDs in advance, and triggers hot migration to migrate data from high-risk SSDs to healthy SSDs. The atomic write transaction guarantee submodule adopts a two-phase commit protocol to store the pre-write log in a dedicated NVDIMM area to ensure data consistency in the event of abnormal interruption.

[0014] Optionally, the system supports the following recovery modes: Instant recovery: Reconstruct the data within the last second through memory mirroring; Fast recovery: Use cross-SSD mirroring data to complete single-disk failure reconstruction within seconds; Full recovery: Restore all data within minutes based on erasure code sharding and incremental snapshots.

[0015] Optionally, the system further comprises: Wear leveling dynamic adjustment module, which periodically redistributes write load according to the SSD health score model; The read-only mode switching module automatically switches the SSD to read-only state and migrates data when the erase and write cycles of the SSD exceed the preset threshold.

[0016] Optionally, the performance indicators of the system meet: Write amplification factor ≤ 1.2; Single disk failure recovery time ≤ 5 seconds; The time required to restore all disk data to the most recent snapshot point is ≤ 5 minutes.

[0017] As described above, the solid state drive data protection and recovery system proposed by the present invention has the following beneficial effects: The present invention realizes hardware-level data protection and intelligent write scheduling through the all-flash array architecture module and dynamic write scheduling layer, effectively reducing the risk of data loss and damage; the memory mirror layer, cross-SSD real-time mirror layer and incremental snapshot layer provide a multi-level data protection mechanism, further enhancing the reliability and security of data; The present invention supports a multi-level recovery mechanism, including instantaneous recovery, fast recovery and complete recovery, which can quickly recover data under different failure conditions and reduce service interruption time. Through fast recovery across SSD images and complete recovery combining redundant coding and incremental snapshots, data recovery can be completed within seconds or minutes, greatly improving the efficiency and speed of data recovery. The write load is periodically redistributed according to the SSD health scoring model, effectively balancing the wear between SSDs and extending the service life of the entire storage system. The dynamic adjustment strategy of the adaptive garbage collection optimization module also helps to reduce the write amplification effect and improve the performance and life of the SSD. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a block diagram of the solid state drive data protection and recovery system of the present invention; Figure 2 Shown is a structural block diagram of the all-flash array architecture module of the present invention; Figure 3 Shown is a structural block diagram of the dynamic write scheduling layer of the present invention; Figure 4 Shown is a structural block diagram of the data protection module of the present invention; Figure 5 Shown is a structural block diagram of the data recovery module of the present invention. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0020] It should be noted that the diagram provided in the present embodiment only illustrates the basic concept of the present invention in a schematic manner, so the diagram only shows the components related to the present invention rather than drawing according to the number, shape and size of the components during actual implementation. The type, quantity and ratio of each component during actual implementation can be a random change, and the component layout type may also be more complicated. The structure, ratio, size, etc. illustrated in the drawings of the present specification are only used to match the content disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions that the present invention can implement, so they have no technical substantive significance. Any modification of the structure, change of the proportional relationship or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the effect that the present invention can produce and the purpose that can be achieved. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" quoted in this specification are only for the convenience of narration, and are not used to limit the scope of the present invention. The change or adjustment of its relative relationship should also be regarded as the scope of the present invention without substantially changing the technical content.

[0021] Reference Figures 1 to 5 As shown, the present invention provides a solid state drive data protection and recovery system, which aims to achieve efficient data storage, protection and rapid recovery through innovative architecture design and algorithm strategy. The system includes multiple components such as an all-flash array architecture module, a dynamic write scheduling layer, a data protection module and a data recovery module, which jointly ensure the integrity, availability and security of data.

[0022] The all-flash array architecture module consists of multiple solid-state drives, at least one of which is configured as a master drive for metadata management, such as address mapping tables, snapshot versions, and redundant coding parameters. The remaining solid-state drives constitute data drives, which implement hardware-level data protection through data sharding and redundant coding. In the specific implementation, the system uses the PCIe 4.0 x8 interface to directly connect to all SSDs, and introduces a dual-plane crossbar topology, that is, the main control disk is connected to two sets of data disks through two independent PCIe channels, and each set of data disks contains 3 SSDs, thus effectively avoiding the single-channel bandwidth bottleneck. The main control disk also establishes a direct connection with the data disk through a PCIe switch. This dual-plane design further prevents single-point failures and significantly improves data transmission efficiency.

[0023] User data is cut into fixed-size data blocks (e.g. 4KB) and stored in different SSDs to achieve data sharding. Redundancy coding includes at least one of RAID 5, RAID 6 or erasure coding. For example, when erasure coding is used, data block D is encoded into n shards, of which k are data shards and m are check shards (n=k+m), allowing data to be recovered when at most m shards are lost.

[0024] A distributed metadata synchronization protocol is implemented between the master disk and the data disk, and the Raft consensus algorithm is used to ensure metadata consistency. The Raft algorithm uses an election mechanism to ensure that the backup master node is quickly switched when the master disk fails. The specific process is as follows: 1. When the master disk fails, the data disk initiates an election and selects the node with the most complete log as the new master disk; 2. The new master disk broadcasts a heartbeat signal to confirm its leadership; 3. The new master disk continues to process metadata update requests and synchronizes logs to other data disks.

[0025] The dynamic write scheduling layer includes an SSD health status perception module, which monitors the remaining life, bad block rate, garbage collection pressure, and queue depth of each SSD in real time, and builds an SSD health rating model based on the monitoring data; Among them, the health score of the SSD health score model Calculated by the following formula: In the formula, The current number of erase and write cycles, reflecting the life consumption of the SSD. is the maximum number of erase and write cycles of the SSD, The value is 10,000 times (the nominal erase and write life of a typical TLC SSD); The number of bad blocks is obtained from the SSD's SMART (Self-Monitoring Analysis and Reporting Technology) log. is the total number of blocks in the SSD, =Total number of SSD blocks × 5% (maximum bad block rate allowed by the manufacturer); is the garbage collection pressure value, and the calculation formula is , is the upper limit of garbage collection pressure, The value is 32; is the current queue depth, indicating the number of IO requests currently pending for processing by the SSD. The upper limit of the queue depth; , , , is the weight coefficient, In this embodiment, , , , They are 0.4, 0.3, 0.2 and 0.1 respectively.

[0026] Data protection module, including intelligent write scheduling algorithm, triple data protection layer and adaptive garbage collection optimization module. The intelligent write scheduling algorithm selects the target storage unit based on the SSD health score and real-time queue depth. The triple data protection layer consists of a memory mirror layer, a cross-SSD real-time mirror layer and an incremental snapshot layer. The execution logic of the intelligent write scheduling algorithm is as follows: Select SSDs whose health scores are higher than the preset threshold and whose garbage collection pressure is lower than the upper limit as candidate storage units; Select the SSD with the smallest queue depth among the candidate units as the write target; If the candidate cell is empty, the degraded protection mode is triggered, user writing is suspended and data migration is started.

[0027] Specifically, below SSDs with a value higher than 10000 are considered unhealthy and writing is prohibited. SSDs with 100% garbage collection pressure indicate that the garbage collection pressure is too high, which may cause long tail delays and prohibit writes. and The SSDs enter the candidate pool. Among them, the minimum health threshold , In the candidate pool, the algorithm will further select the SSD with the smallest queue depth as the write target to ensure that the SSD with the smallest current load is selected; The degraded protection mode means that when all SSDs do not meet the conditions, user IO is suspended and data migration to healthy SSDs is started. The migration priority is The values ​​are arranged in descending order to ensure the continuous security of data and the stable operation of the system.

[0028] In the triple data protection layer of this specific embodiment, The memory mirroring layer uses non-volatile dual in-line memory modules (NVDIMMs) to store real-time data copies and uses supercapacitors to ensure data persistence during power outages. Two NVDIMM memory sticks are configured to form a mirror pair, and an asynchronous disk flushing strategy is used to balance performance and data persistence.

[0029] Perform at least two independent SSD write operations for each write request across the SSD real-time mirror layer; specifically, perform two independent SSD writes for each write request to ensure that the data is written to the mirror SSD at the same time as the primary SSD. This process uses atomic write operations to ensure data consistency, and uses the "Compare-and-Swap (CAS)" instruction to ensure that the write operations of the two SSDs are successful or rolled back at the same time. The atomic write operation process is: write the data version number to the NVDIMM log, and write it to the primary SSD and mirror SSD concurrently; if both are written successfully, commit the log and update the metadata; otherwise, roll back the operation.

[0030] The incremental snapshot layer records data changes in a log structure and generates differential snapshots with configurable time intervals. Specifically, snapshots are generated based on time-triggered and event-triggered policies. The time-triggered policy generates a global snapshot every 5 minutes to record the status of the entire storage system; the event-triggered policy generates a local snapshot when the data change reaches 1GB, recording the data changes within a specific time period.

[0031] Among them, the adaptive garbage collection optimization module performs the following operations: Actively trigger full disk garbage collection during SSD idle time; A lazy recycling strategy is used during high-load periods, where only blocks that meet the minimum recycling threshold are recycled; Specifically, the threshold, granularity, and execution frequency of garbage collection are dynamically adjusted according to the load status. When the SSD is idle, full disk garbage collection is triggered, and the recycling granularity is block level; when When the SSD is in normal load state, it is recycled on demand (blocks to be recycled ≥ 10%), and the recycling granularity is page level; when When the SSD is in a high-load state, lazy recycling is used (only completely invalid blocks are recycled), and the recycling granularity is page level.

[0032] Reserve no less than 25% of the over-provisioning (OP) space to mitigate the write amplification effect.

[0033] Specifically, 25% of the physical capacity is initially reserved as reserved space for garbage collection and data reorganization, based on the garbage collection pressure value. Dynamically adjust the OP ratio to ensure that the system can maintain good performance under high load. In this embodiment, the OP ratio is as follows: .

[0034] The data recovery module supports multi-level recovery mechanisms, including instant recovery based on memory mirroring, fast recovery across SSD mirroring, and complete recovery combining redundant coding and incremental snapshots.

[0035] Among them, the system supports the following recovery modes: Instant recovery (<1 second): Reconstruct the data within the last second through memory mirroring; when the SSD is abnormally powered off or the IO times out (>500ms), load the log of the last second from the NVDIMM. The log format is: operation type, LBA, data version number, data content; when parsing, replay uncommitted transactions in chronological order, reconstruct the memory mapping table, and write the data to the backup SSD.

[0036] Fast recovery (seconds): Single-disk failure reconstruction is completed within seconds using cross-SSD mirror data. When a single SSD fails (SMART warning or IO error), a complete data copy is read from the mirror SSD, the lost shards are recalculated using erasure coding, and the recovered data is written to the replacement disk.

[0037] Full recovery (minutes): Restore the full data in minutes based on erasure code sharding and incremental snapshots. When multiple disks fail or the system crashes, load the most recent global snapshot, replay the incremental snapshot logs in chronological order, use erasure codes to repair missing shards, and rebuild the complete data set.

[0038] Preferably, the data recovery module includes: The fault prediction submodule monitors the bad block growth rate and erase / write cycle usage rate, marks high-risk SSDs in advance, triggers thermal migration, and migrates data from high-risk SSDs to healthy SSDs. Specifically, the ARIMA (AutoRegressiveIntegrated Moving Average) time series analysis method is used to input parameters such as the historical bad block growth curve, temperature fluctuation data, and write amplification factor, and output the risk level. In this embodiment, the parameters of the ARIMA model are set to (p=2, d=1, q=1), where p is the autoregressive order, d is the difference order, and q is the moving average order. The model training uses the historical 7-day bad block growth curve, temperature fluctuation data, and write amplification factor as input, and fits the parameters using the least squares method.

[0039] The higher the risk level, the greater the possibility of SSD failure. Combine the risk level and remaining life to set the hot migration trigger condition. When the risk level is greater than 0.8 or the remaining life is less than 15%, the hot migration process is triggered to migrate data to a healthier SSD. This process is designed to prevent potential failures and reduce the risk of service interruption and data loss caused by SSD failure.

[0040] The atomic write transaction guarantee submodule adopts a two-phase commit protocol to store the write-ahead log in a dedicated NVDIMM area to ensure data consistency in the event of abnormal interruption. In the two-phase commit protocol, the write operation is divided into two phases: the preparation phase and the commit phase. In the preparation phase, the system will attempt to perform the write operation and store the write-ahead log (recording relevant information about the write operation) in a dedicated NVDIMM area. If the write-ahead log is successfully stored, the system will enter the commit phase and formally perform the write operation. If an abnormal interruption occurs at any stage, the system can recover the unfinished transaction by parsing the write-ahead log to ensure data consistency.

[0041] Preferably, the system provided by the present invention further includes: The wear leveling dynamic adjustment module periodically redistributes the write load according to the SSD health score model; in this embodiment, the write weight of each SSD is adjusted every 24 hours according to the health score H , the formula is as follows: In the formula, For the The health score of each SSD, is the total number of SSDs, is a variable, which is the index of traversing SSD. (Total bandwidth).

[0042] The read-only mode switching module automatically switches the SSD to a read-only state and migrates data when the SSD erase / write cycle exceeds a preset threshold. When the target SSD is set to read-only, the erasure code is used to recalculate the data shards, distribute them to the healthy SSDs, update the metadata mapping table and release the original SSD space.

[0043] During the performance test, the FIO tool was used to simulate random write load (4KB, QD=32), and the test environment was a storage group consisting of 6 1TB TLC SSDs, running the Linux 5.15 kernel. The performance indicators of the system provided by the present invention meet the following: Write amplification factor ( ): The ratio of the amount of data actually written to the SSD to the amount of data requested by the user, calculated using the formula , through OP space reservation and lazy recycling strategy, Controlled at ≤1.2.

[0044] Single disk failure recovery time ≤ 5 seconds: Mirror data is directly copied (rate ≥ 2GB / s), and 1TB of data can be recovered within 5 seconds.

[0045] The time for restoring all disk data to the most recent snapshot point is ≤ 5 minutes: Based on parallel reconstruction of snapshots and erasure codes (rate ≥ 500MB / s), 10TB of data can be restored within 5 minutes.

[0046] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A solid state drive data protection and recovery system, characterized in that: include: The all-flash array architecture module consists of multiple solid-state drives, at least one of which is configured as a master drive for metadata management, and the remaining solid-state drives constitute data drives, which implement hardware-level data protection through data sharding and redundant coding; The dynamic write scheduling layer includes an SSD health status perception module, which monitors the remaining life, bad block rate, garbage collection pressure, and queue depth of each SSD in real time, and builds an SSD health scoring model based on the monitoring data; Data protection module, including intelligent write scheduling algorithm, triple data protection layer and adaptive garbage collection optimization module. The intelligent write scheduling algorithm selects the target storage unit based on the SSD health score and real-time queue depth. The triple data protection layer consists of a memory mirror layer, a cross-SSD real-time mirror layer and an incremental snapshot layer. The data recovery module supports multi-level recovery mechanisms, including instant recovery based on memory mirroring, fast recovery across SSD mirroring, and complete recovery combining redundant coding and incremental snapshots.

2. The solid state drive data protection and recovery system according to claim 1, characterized in that: The redundant coding includes at least one of RAID 5, RAID 6 or erasure coding, and consistency management is achieved between the master disk and the data disk through a distributed metadata synchronization protocol.

3. The solid state drive data protection and recovery system according to claim 1, characterized in that: The execution logic of the intelligent write scheduling algorithm includes: Select SSDs whose health scores are higher than the preset threshold and whose garbage collection pressure is lower than the upper limit as candidate storage units; Select the SSD with the smallest queue depth among the candidate units as the write target; If the candidate cell is empty, the degraded protection mode is triggered, user writing is suspended and data migration is started.

4. The solid state drive data protection and recovery system according to claim 1, characterized in that: The triple data protection layer: The memory mirroring layer uses NVDIMM to store real-time data copies, and uses supercapacitors to ensure data persistence during power outages; Perform at least two independent SSD write operations for each write request across the SSD real-time mirroring layer; The incremental snapshot layer records data changes in a log structure and generates differential snapshots with configurable time intervals.

5. The solid state drive data protection and recovery system according to claim 1, characterized in that: The adaptive garbage collection optimization module performs the following operations: Actively trigger full disk garbage collection during SSD idle time; A lazy recycling strategy is used during high-load periods, where only blocks that meet the minimum recycling threshold are recycled; Reserve no less than 25% of the reserved space to mitigate the write amplification effect.

6. The solid state drive data protection and recovery system according to claim 1, characterized in that: The data recovery module further comprises: The fault prediction submodule monitors the bad block growth rate and erase / write cycle usage rate, marks high-risk SSDs in advance, and triggers hot migration to migrate data from high-risk SSDs to healthy SSDs. The atomic write transaction guarantee submodule adopts a two-phase commit protocol to store the pre-write log in a dedicated NVDIMM area to ensure data consistency in the event of abnormal interruption.

7. The solid state drive data protection and recovery system according to claim 1, characterized in that: The system supports the following recovery modes: Instant recovery: Reconstruct the data within the last second through memory mirroring; Fast recovery: Use cross-SSD mirroring data to complete single-disk failure reconstruction within seconds; Full recovery: Restore all data within minutes based on erasure code sharding and incremental snapshots.

8. The solid state drive data protection and recovery system according to claim 1, characterized in that: The system further comprises: Wear leveling dynamic adjustment module, which periodically redistributes write load according to the SSD health score model; The read-only mode switching module automatically switches the SSD to read-only state and migrates data when the erase and write cycles of the SSD exceed the preset threshold.

9. The solid state drive data protection and recovery system according to claim 1, characterized in that: The performance indicators of the system meet the following requirements: Write amplification factor ≤ 1.2; Single disk failure recovery time ≤ 5 seconds; The time required to restore all disk data to the most recent snapshot point is ≤ 5 minutes.

Images