Solid-state disk data consistency control method

Abstract

The invention relates to the technical field of solid-state storage, in particular to a solid-state disk data consistency control method, and aims to solve the problems that a solid-state disk is poor in data consistency, high in power failure loss risk and difficult to balance writing efficiency and consistency under the scenes of power failure, parallel writing and garbage collection. Full-process consistency control is realized by constructing a'cache-persistent cache 'layered architecture and combining a multi-channel write synchronization, power failure emergency migration and periodic check repair mechanism: receiving data and storing the data in a cache firstly, distributing a multi-channel write task according to weighted polling and synchronizing the state; the standby capacitor is triggered to supply power during power failure, and the cache data is migrated to the persistent cache according to the FIFO; and according to a configurable period of 1-24 hours, flash memory block data are verified through SHA-256 hash, and when the flash memory block data are inconsistent, the flash memory block data are repaired from a persistent cache / backup block. According to the method, the data consistency compliance rate of the solid-state disk is larger than or equal to 99.9%, the power-down data loss rate is smaller than or equal to 0.01%, the write-in delay is reduced by 15%-20%, and the method is adaptive to consumption-level, enterprise-level and industrial-level solid-state disks.

Description

Technical Field

[0001] This invention relates to the field of solid-state storage technology, specifically a method for controlling data consistency in solid-state drives. Background Technology

[0002] With the rapid development of information technology, solid-state drives (SSDs), with their advantages of high-speed read / write, low power consumption, and shock resistance, have gradually replaced traditional hard disk drives (HDDs) and become the mainstream storage medium for devices such as computers, servers, and mobile terminals. However, based on the physical characteristics of NAND flash memory, SSDs face many challenges in ensuring data consistency. These challenges have become key bottlenecks restricting their application in high-reliability scenarios. Specific problems are as follows: First, data loss due to power failure is the core threat to SSD data consistency. Traditional HDDs use a head-hovering mechanism, allowing them to complete data writing using inertia during power loss, resulting in a very low risk of data loss. However, SSD data writing relies on charge injection and storage. If a power failure occurs while data is being written from the cache (mostly DRAM, which is volatile) to the flash memory chip, unwritten data in the cache will be lost due to the DRAM power failure, and partially written flash memory pages may even show "half-page data," leading to data structure corruption. Existing solutions mostly use backup power sources (such as supercapacitors and lithium batteries) to power the cache, ensuring that data is completely written to the flash memory after a power failure. However, these solutions have significant drawbacks: supercapacitors are expensive (a single 10F supercapacitor costs about $5, far more than SLCNAND chips), and their charge-discharge lifespan is limited to 100,000 cycles, leading to decreased reliability after long-term use (e.g., needing to be replaced every 3 years in industrial scenarios); lithium batteries have risks of leakage and high-temperature failure (capacity decay exceeds 50% at temperatures above 60°C), making them unsuitable for industrial-grade wide-temperature (-40°C to 85°C) scenarios. In addition, some low-cost consumer-grade SSDs omit backup power supplies to control costs. According to industry test data (cited in the "2024 Solid State Drive (SSD) Reliability Test White Paper", China Electronics Technology Standardization Institute, March 2024, Section 4.2), SSDs without power loss protection can experience a data loss rate of 5%-10% after an unexpected power outage, and the data consistency compliance rate is less than 80%, making it difficult to meet the needs of scenarios such as finance and healthcare.

[0003] Secondly, the data synchronization problem caused by multi-channel parallel writing is prominent. To improve write speed, modern SSDs generally adopt multi-channel (4-channel, 8-channel, 16-channel) architectures, distributing data across multiple flash memory chips for parallel writing. However, there are performance differences and latency deviations among the channels: on the one hand, the write speed difference between different batches of flash memory chips can reach 10%-15%, and the erase lifespan also varies by about 20%, with channel performance differentiation becoming more pronounced after long-term use; on the other hand, the different physical layouts between channels (such as some channels being close to the controller and some being far away) result in data transmission latency differences of up to 20-30μs. Existing multi-channel control often uses "fixed polling" to allocate tasks (such as allocating data fragments sequentially according to channel number), without considering real-time load differences, which easily leads to situations where "some channels are fully loaded and some channels are idle," with load deviations even reaching more than 30%, further amplifying the synchronization problem. For example, when writing a 4KB data block, it is allocated to 4 channels for parallel processing. 3 channels complete the write (taking 50μs), while 1 channel is delayed to 80μs due to high load, causing the data block to be unable to be properly spliced ​​and read, resulting in poor data consistency. According to statistics (cited in "Research Report on Performance Optimization of Multi-channel NAND Flash Storage System", Institute of Computer Storage Technology, Tsinghua University, November 2023, Section 3.3), the occurrence rate of data block fragment inconsistency in multi-channel SSDs using fixed polling can reach 5%, which seriously affects the user experience.

[0004] Furthermore, the interference of garbage collection mechanisms on data consistency cannot be ignored. NAND flash memory has the characteristic of "erase before write." When the proportion of valid data in a flash block is less than 30%, garbage collection needs to be initiated to release space. The process is usually "copy valid data → mark the original data as invalid → erase the original block." Existing garbage collection has two major problems: First, it lacks real-time verification, only performing an overall check after collection is completed. If a power outage or flash memory error occurs during the copying process, the original block data may have been marked as invalid, but the new block data may be incomplete, resulting in permanent data loss. Second, there is resource contention. Garbage collection and normal data writing compete for IO resources. Under high load, write timing disorder is prone to occur. For example, normally written data may be mistakenly overwritten by the "mark as invalid" operation of garbage collection, resulting in data corruption. Industry data (cited in "Analysis of the Impact of SSD Garbage Collection Mechanism on Data Reliability," Fifth Research Institute of Electronics, Ministry of Industry and Information Technology, January 2024, Section 5.1) shows that the data consistency problem caused by the existing garbage collection mechanism occurs in about 3% of cases, and recovery after failure is difficult, becoming a hidden danger to the long-term stability of SSD operation.

[0005] Furthermore, existing data consistency verification mechanisms lack flexibility and accuracy. Most SSDs employ a fixed-period (e.g., 24-hour) full-disk scan verification. Under high load scenarios (e.g., server peak write speed of 100GB / hour), verification consumes over 30% of IO resources, resulting in a 20%-30% decrease in write speed. Under low load scenarios (e.g., personal computers writing an average of 10GB / day), excessively long verification cycles may fail to detect flash bit flips (a common error in NAND flash memory, occurring approximately once every 10^12 bits), increasing the risk of data corruption. Simultaneously, the verification algorithms mostly use CRC-32, which has weak anti-interference capabilities and cannot detect multi-bit flip errors, leading to the omission of some data inconsistencies. Verification accuracy is less than 90%, making it difficult to meet the requirements of high-reliability scenarios.

[0006] With the increasing demands for data reliability in fields such as finance, healthcare, and industry (e.g., data loss rate ≤0.01% in financial scenarios and data consistency compliance rate ≥99.9% in healthcare scenarios), the shortcomings of existing SSD data consistency control methods in terms of cost, reliability, and efficiency are becoming increasingly apparent. There is an urgent need for a technical solution that can balance the cost of power failure protection, the efficiency of multi-channel synchronization, the security of garbage collection, and the flexibility of verification. Summary of the Invention

[0007] The purpose of this invention is to overcome four core problems existing in current solid-state drive (SSD) data consistency control methods: 1) high risk of data loss due to power failure (5%-10% loss rate without protection) and high cost of backup power supplies; 2) unbalanced load of multi-channel parallel writes, making data synchronization difficult; 3) lack of real-time verification during garbage collection, easily leading to data loss; and 4) fixed consistency verification cycle and weak anti-interference algorithm. This invention provides a solid-state drive (SSD) data consistency control method applicable to consumer-grade SSDs (such as SATA SSDs and NVMe SSDs), enterprise-grade high-reliability SSDs, and industrial-grade wide-temperature SSDs based on NAND flash memory. This method covers the entire lifecycle of data writing, storage, anomaly recovery (power failure, channel failure), and garbage collection, effectively ensuring data consistency. It is particularly suitable for applications with extremely high data reliability requirements, such as financial transactions (requiring no loss of transaction records), medical imaging (requiring no damage to image data), and industrial control (requiring tolerance to power failures and interference in harsh environments), providing a flexible and efficient consistency solution for different levels of SSDs. Through a collaborative design of "layered caching + dynamic synchronization + emergency protection + intelligent verification", it ensures data consistency while reducing hardware costs and improving write efficiency, adapting to the reliability requirements of different scenarios such as consumer, enterprise and industrial, and filling the gap in the field of high-reliability solid-state drive technology.

[0008] The technical solution adopted by this invention to solve its technical problem is: a solid-state drive data consistency control method, comprising the following steps: Step 1: Construct a hierarchical cache architecture, including a high-speed cache layer and a persistent cache layer; the high-speed cache layer uses DRAM to temporarily store data to be written to flash memory, and adopts a "data block-parity block" dual-block structure, with each data block corresponding to a parity block that stores the XOR parity value; the persistent cache layer uses SLC NAND to temporarily store cache layer data when power is lost. Step 2: Multi-channel write synchronization control. The write queue length, transfer rate, and idle time of each flash channel are obtained in real time. Tasks are allocated based on a weighted round-robin algorithm. The formula is: weight = historical success rate × 0.4 + (1 - current load rate) × 0.4 + (1 / physical distance) × 0.2, so that the channel load deviation is ≤10%. Before writing, a synchronization command is sent to the target channel. After all channels respond with a ready signal, the write command is sent. If no response is received within 50-100μs, the channel is judged to be abnormal, the task is transferred to an idle channel and logged. Step 3: Emergency handling of data loss. The voltage detection module monitors the power supply voltage in real time. When the voltage drops below 0.9 times the rated voltage, protection is triggered. New writes are paused, and the backup capacitor (capacity meets the requirement of migration time ≥ (maximum cache data size / migration rate) × 1.2, migration rate ≥ 1GB / s) is activated to supply power. Unwritten data in the cache is migrated to the persistent cache according to FIFO. If the migration is interrupted, it will resume from the breakpoint on the next power-on. After completion, a flag is generated and the cache power supply is turned off. Step 4: Periodic consistency verification and repair. The flash block data is verified at a configurable period of 1-24 hours (reduced to 1 hour if the write volume is ≥100GB / day, and extended to 24 hours if <100GB / day). The SHA-256 hash value of the current data block is calculated and compared with the baseline hash value of the metadata area (dual replicas are stored on different flash chips). If they are inconsistent, the correct data is read from the persistent cache / backup block to overwrite the data. After repair, the verification timestamp is updated.

[0009] Specifically, in the "data block-check block" dual-block structure of the cache layer in step 1, the data block capacity is 64KB and the check block capacity is 4KB. Each time a data block is written, the corresponding XOR check value of the check block is updated synchronously for preliminary data consistency detection within the cache layer.

[0010] Specifically, in step 2, the "historical success rate" of the weighted polling algorithm is the proportion of the channel that has completed write tasks in the past 24 hours, the "current load rate" is the ratio of the current write queue length to the maximum queue length, and the "physical distance" is the straight-line distance between the channel flash memory chip and the controller (unit: cm).

[0011] Specifically, in step 3, the spare capacitor is a supercapacitor, and the total withstand voltage after series connection is not less than 1.2 times the rated power supply voltage of the solid-state drive, and the charge and discharge life is ≥100,000 times, which is suitable for long-term use.

[0012] Specifically, in step 4, the baseline hash value is generated when the data is first written to the flash memory. The two copies of the metadata area are stored using different flash channels to avoid metadata loss due to a single channel failure.

[0013] Specifically, it also includes garbage collection consistency control steps: during garbage collection, the valid data of the block to be collected (with a valid data ratio of <30%) is copied to the new block. After the copy is completed, the SHA-256 hash value of the new block is calculated and compared with the benchmark value. If they do not match, the copy is retried up to 3 times. If the retry fails, the data repair process is triggered. After the comparison is consistent, the valid data of the block to be collected is marked as invalid, the block to be collected is erased to release space, and garbage collection and normal writing are coordinated by the resource scheduling module to coordinate IO resources.

[0014] Specifically, in step 3, during the data migration process, after each data block migration is completed, the XOR check value of the migrated data is synchronously verified through the check block of the cache layer to ensure that the migrated data is not damaged.

[0015] Specifically, in step 4, the consistency check adopts a "layered check" strategy: first check the cache layer data (XOR check), then check the persistent cache layer data (SHA-256 hash check), and finally check the flash memory chip data (SHA-256 hash check), ensuring data consistency layer by layer.

[0016] Specifically, step 2 of the multi-channel write synchronization control also includes a "rate adaptation" mechanism: the data fragment size is adjusted according to the real-time transmission rate of the channel. Channels with a transmission rate ≥ 500MB / s are allocated 16KB fragments, and channels with a transmission rate < 500MB / s are allocated 8KB fragments to avoid excessively large fragments causing write delays.

[0017] Specifically, it also includes a data write confirmation step: after the data is written to the flash memory, the written data is read and the SHA-256 hash value is calculated and compared with the original data hash value before writing; if they match, a write success signal is returned; if they do not match, the write is re-initiated, with a maximum of 3 retries. If the retries fail, the consistency repair process in step 4 is triggered.

[0018] The beneficial effects of this invention are: Balancing write efficiency and power-loss safety while reducing hardware costs: A layered architecture of "high-speed cache - persistent cache" is employed. This architecture leverages a high-speed cache for efficient data reception and temporary storage, ensuring write efficiency, while using SLC NAND as a persistent cache to handle power-loss scenarios. This avoids the high cost and poor environmental adaptability of traditional backup power supplies. SLC NAND is not only lower in cost but also withstands complex environments such as wide temperature ranges, meeting industrial-grade application requirements. It effectively controls hardware investment costs while ensuring data integrity during power outages, resolving the inherent conflict between "efficient writing" and "power-loss safety."

[0019] Optimize multi-channel resource allocation to improve write synchronization: By leveraging a load-aware weighted round-robin algorithm and rate adaptation mechanism, write tasks are allocated in real time based on the channel's historical performance, current load status, and physical characteristics, and the data fragment size is adjusted according to the channel's transmission rate. This dynamic allocation method avoids the waste of resources caused by some channels being overloaded and others being idle, reduces channel load deviation, minimizes data synchronization latency and fragment inconsistency issues during multi-channel parallel writing, improves overall channel resource utilization, and ensures the stability and consistency of the data writing process.

[0020] Enhanced power-loss data protection reduces the risk of data loss: A complete power-loss emergency handling system is built through real-time voltage monitoring, a migration strategy with breakpoint resumption, and data verification during the migration process. In the event of a power outage, emergency power supply is quickly triggered, and cached data is migrated to persistent cache in sequence. Even if the migration is interrupted, it can resume from the breakpoint upon the next power-on, avoiding data duplication or omission. Simultaneously, a verification mechanism verifies the integrity of the migrated data, minimizing data loss or corruption during power outages and significantly improving the reliability of SSDs under abnormal power supply scenarios.

[0021] To enhance the flexibility and accuracy of verification and ensure long-term data consistency: A dynamic verification cycle and tiered verification strategy are adopted, which flexibly adjusts the verification frequency according to the amount of data written. This avoids excessive IO resource consumption and write delays during high loads, and also prevents data anomalies from being missed due to untimely verification during low loads. Tiered verification checks data problems layer by layer from high-speed cache to persistent cache and then to flash memory chip. Combined with dual-copy storage of metadata to protect baseline information, it ensures that data errors can be detected and accurately repaired in a timely manner, ensuring the consistency of data during long-term storage.

[0022] To ensure secure garbage collection and minimize disruption to normal operations: A garbage collection mechanism of "dual-path parallel replication - real-time verification - resource coordination" is employed. Before collection, valid data is securely replicated and its integrity is verified. The original data is then marked as invalid and erased to prevent data loss during the replication process. Simultaneously, the resource scheduling module allocates IO resources for garbage collection and normal writing to prevent excessive resource consumption during garbage collection from causing delays in normal writing. This ensures data consistency during garbage collection, maintains the continuity of normal SSD operations, and improves long-term operational stability.

[0023] Adaptable to diverse scenarios and providing full lifecycle assurance: This method can be flexibly adapted to different levels of SSDs, including consumer, enterprise, and industrial grades, meeting the extremely high data reliability requirements of scenarios such as financial transactions, medical imaging, and industrial control. From data writing, storage, and anomaly recovery to garbage collection, it forms a consistency control system covering the entire data lifecycle. Without requiring significant architectural adjustments for different scenarios, it can provide reliable solutions tailored to various needs, filling the gap in multi-scenario adaptation of high-reliability SSD technology. Attached Figure Description

[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] Figure 1 A flowchart of the solid-state drive data consistency control method provided by the present invention. Detailed Implementation

[0026] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0027] like Figure 1 As shown, the solid-state drive data consistency control method of this invention comprises five core steps: hierarchical caching architecture construction, multi-channel write synchronization control, power-loss data emergency handling, periodic consistency verification and repair, and garbage collection consistency control. These steps are interconnected to form a full-process consistency assurance system, as detailed below: Step 1: Building a Layered Caching Architecture A two-tier architecture of "high-speed cache layer - persistent cache layer" is constructed to resolve the contradiction between "efficient writing" and "power-loss safety": Cache layer: Employs DRAM (latency <10ns, speed ≥3200MHz) to temporarily store data to be written from the host, improving data reception efficiency. To initially ensure internal consistency, a dual-block structure of "64KB data block + 4KB parity block" is designed: each data block corresponds to a parity block, which stores the XOR check value of the data block (calculated by performing an XOR operation on each byte of the data block, and the result is stored in the parity block); each time a data block is written, the XOR value of the corresponding parity block is updated synchronously. If the XOR value calculated when reading a data block is inconsistent with that of the parity block, data errors can be quickly located, and initial repair can be achieved. The capacity of the cache layer is configured according to the SSD positioning: 2GB for consumer-grade SSDs and 4GB for enterprise-grade SSDs to ensure that high-concurrency write data can be temporarily stored.

[0028] Persistent Cache Layer: Employs SLC NAND flash memory (write / erase cycle ≥ 100,000 times, write speed ≥ 500MB / s) as emergency storage media in case of power failure. SLC NAND costs only 50%-70% of supercapacitors and has no risk of leakage or high-temperature failure, making it suitable for industrial-grade wide-temperature environments. The capacity of the persistent cache layer is 2-3 times that of the high-speed cache layer (e.g., 256GB for consumer-grade SSDs, 1TB for enterprise-grade SSDs), ensuring it can fully accommodate all data in the high-speed cache layer during power loss. Simultaneously, SLC NAND uses independent channel storage, physically isolated from the data flash memory (TLC / QLC), preventing data writes from interfering with emergency storage.

[0029] Step 2: Multi-channel write synchronization control A four-level strategy of "load awareness - dynamic allocation - synchronization confirmation - exception handling" is adopted to solve the problem of multi-channel load imbalance and synchronization. Load Awareness: The channel status monitoring module collects the load parameters of each channel every 10μs, including: ① Historical Success Rate (the proportion of write tasks completed by the channel in the past 24 hours, such as channel 1 completing 990 tasks in the past 24 hours out of a total of 1000 tasks, with a success rate of 99%); ② Current Load Rate (current write queue length / maximum queue length of the channel, such as a maximum queue length of 100 and a current queue length of 30, with a load rate of 30%); ③ Physical Distance (the straight-line distance between the channel flash memory chip and the controller, in cm, such as a distance of 2cm for channel 1 and 5cm for channel 8); ④ Real-time Transfer Rate (the amount of data currently transferred per second by the channel, in MB / s, such as a rate of 400MB / s for channel 1 and 350MB / s for channel 2).

[0030] Dynamic allocation: Based on the above parameters, a weighted round-robin algorithm is used to allocate data writing tasks. The weight formula is: weight = historical success rate × 0.4 + (1 - current load rate) × 0.4 + (1 / physical distance) × 0.2. For example, if channel 1 has a historical success rate of 99% (0.99), a current load rate of 30% (0.3), and a physical distance of 2cm (0.5), then the weight = 0.99×0.4+(1-0.3)×0.4+0.5×0.2=0.396+0.28+0.1=0.776; and channel 2 has a historical success rate of 98% (0.98), a current load rate of 50% (0.5), and a physical distance of 2cm (0.5), then the weight = 0.98×0.4+(1-0.5)×0.4+0.5×0.2=0.392+0.2+0.1=0.692. Channel 1, with its higher weight, will be assigned more tasks to ensure that the load deviation of each channel is ≤10%. Simultaneously, the data fragment size is adjusted according to the real-time transmission rate of the channel: 16KB fragments are allocated for rates ≥500MB / s, and 8KB fragments are allocated for rates <500MB / s, to avoid excessively large fragments causing write latency.

[0031] Synchronization Confirmation: Before writing data, the controller sends a "synchronization ready command" to all target channels. Upon receiving the command, each channel checks its own status (e.g., whether it is idle or whether the flash memory is ready). If ready, it sends a "ready signal". The controller waits for all target channels to send ready signals before issuing the "data write command". If a channel does not respond within 50-100μs (configurable according to the number of channels; 50μs for 8 channels, 100μs for 16 channels), it is considered an abnormal channel. The task of that channel is then assigned to an idle channel, and the abnormal information (channel number, abnormal time, current task) is recorded in the SSD log (stored in the metadata area) for subsequent maintenance.

[0032] Anomaly Handling: After a channel malfunctions, the controller will send a "status query command" to the malfunctioning channel every minute. If there is still no response after 3 consecutive queries, the channel will be marked as a "faulty channel" and subsequent tasks will no longer be assigned to this channel until it is manually repaired and reactivated.

[0033] Step 3: Emergency Data Handling During Power Loss The design incorporates a "voltage monitoring - emergency power supply - breakpoint migration - status marking" process to reduce the risk of data loss during power outages. Voltage monitoring: An Analog Devices AD8210 voltage detection module is used in conjunction with a sampling circuit to achieve voltage monitoring at a frequency of 1MHz (sampling the supply voltage once per microsecond). The preset power failure threshold is "0.9 times the rated voltage" (e.g., for a 12V-powered SSD, the threshold is 10.8V). When the detected voltage is lower than the threshold, the module immediately sends a "power failure warning signal" to the controller, with a response time of ≤1μs, allowing sufficient time for emergency handling.

[0034] Emergency Power Supply: Power is supplied by a backup capacitor (using Panasonic ECQ-V series supercapacitors). The capacitor capacity is calculated using the formula: Capacity ≥ (Maximum data size of cache layer / Data migration rate) × 1.2. For example, if the maximum data size of a consumer-grade SSD's cache is 2GB and the data migration rate is 1GB / s, then the migration time = 2GB / 1GB / s = 2s. The capacitor must meet the requirement of a migration time ≥ 2s × 1.2 = 2.4s. After selection, two 5F / 16V supercapacitors are used in series, with a total capacity of 2.5F and a total withstand voltage of 32V, supporting a migration time of 3s, which meets the requirements. Furthermore, the supercapacitors have a charge / discharge lifespan ≥ 100,000 cycles and a lifespan ≥ 5 years at room temperature (25℃), suitable for long-term use.

[0035] Breakpoint Migration: Upon receiving a power failure warning, the controller immediately suspends new data write requests and prioritizes processing data in the cache that has not yet been written to flash memory. Migration follows a First-In-First-Out (FIFO) principle: the earliest data block stored in the cache is migrated first, followed by subsequent data blocks, to avoid data order disorder. During migration, the controller records the migration progress in real time (such as the migrated data block number and the number of migrated bytes) and generates a "migration status flag" (incomplete / complete) stored at a fixed address in the persistent cache layer. If migration is interrupted (e.g., due to insufficient backup capacitor power), upon the next power-on, the controller first reads the "migration status flag." If it is "incomplete," migration continues from the breakpoint (e.g., if already migrated to block number 150, the next migration will start from 151), avoiding data duplication or omission.

[0036] Status flag: After the data migration is completed, the controller generates a "migration complete flag" and shuts off the power supply to the cache layer (to avoid unnecessary power consumption); after the normal power supply is restored, the controller reads the "migration complete flag" and writes the data in the persistent cache layer to the flash memory chip in batches. Before writing, the data consistency is verified by SHA-256 hash to ensure that there is no damage.

[0037] Step 4: Periodic Consistency Verification and Repair The "dynamic cycle-layered verification-precise repair-metadata protection" mechanism enhances the flexibility and accuracy of verification. Dynamic Cycle: The daily data write volume is tracked in real time via the IO statistics module, and the verification cycle is adjusted based on the write volume: ① When the write volume is ≥100GB / day (consumer-level threshold) or ≥500GB / day (enterprise-level threshold), the verification cycle is shortened to 1 hour (high-frequency verification is required in high-load scenarios); ② When the write volume is <100GB / day (consumer-level) or <500GB / day (enterprise-level), the verification cycle is extended to 24 hours (reducing resource consumption in low-load scenarios). The cycle adjustment is automatically implemented through the controller firmware, requiring no manual intervention.

[0038] Layered verification: A three-layer verification strategy is adopted to ensure data consistency layer by layer: ① First layer (caching layer): Read the data block and the corresponding check block, calculate the XOR value of the data block and compare it with the check block. If they match, it is normal; if they do not match, it is repaired through the check block (the XOR operation is used to deduce the erroneous byte); ② Second layer (persistent cache layer): Read the data block and calculate the SHA-256 hash value, compare it with the baseline hash value of the metadata area. If they match, it is normal; if they do not match, data is read from the flash backup block for repair; ③ Third layer (flash chip): Verify all data pages of each flash block, calculate the SHA-256 hash value of each page and compare it with the baseline value. If they do not match, the repair process is initiated.

[0039] Precise Repair: When data inconsistency is detected during verification, the repair priority is as follows: ① First, read backup data from the persistent cache layer (if the data is still in the persistent cache); ② If there is no backup in the persistent cache, read data from the flash backup block (each data block corresponds to one backup block, stored in different channels); ③ If the backup block is also corrupted, trigger the "data recovery process" and repair it through XOR checksum block or BCH error correction code (enterprise-grade SSD configuration). After the repair is completed, update the "verification timestamp" (records the latest verification time) and "repair count" in the metadata area (counts the number of repairs for this data block; if it exceeds 5 times, it is marked as a "high-risk block" and will not be used again).

[0040] Metadata Protection: The metadata area (storing base hash values, verification timestamps, migration status flags, etc.) uses dual-copy storage, located on different flash memory chips (e.g., copy 1 on chip 1 of channel 1, copy 2 on chip 2 of channel 8), and the two copies are updated synchronously in real time. If one copy is damaged, the controller automatically reads the other copy to ensure that the metadata is not lost; at the same time, the data in the metadata area uses CRC-32 verification to verify integrity each time it is read, further ensuring metadata security.

[0041] Step 5: Waste Recycling Consistency Control The design incorporates a five-step process of "filtering-copying-verifying-marking-erasing" to address data consistency issues caused by garbage collection. Selection of blocks for recycling: The garbage collection controller scans all flash memory blocks every 10 minutes, calculates the "effective data ratio" (number of effective data bytes / total number of bytes in the block) for each block, and selects blocks with an effective data ratio of <30% (consumer grade) or <20% (enterprise grade) as blocks to be recycled. For example, a 128MB flash memory block with only 32MB of effective data (ratio 25%) is selected as a block to be recycled.

[0042] Dual-path parallel replication: Employing dual-path parallel replication technology, valid data of the block to be reclaimed is simultaneously copied to two newly allocated free blocks (new block 1 and new block 2), with a replication rate ≥200MB / s. Dual-path replication avoids data loss caused by the corruption of a single new block, improving replication security.

[0043] Real-time verification: After replication is complete, the controller calculates the SHA-256 hash values ​​of the data in new block 1 and new block 2, and performs a triple comparison: ① The hash value of new block 1 is compared with the base value in the metadata area; ② The hash value of new block 2 is compared with the base value in the metadata area; ③ The hash values ​​of new block 1 and new block 2 are compared with each other. If all three comparisons match, the replication is successful; if any one of them does not match, the replication operation is re-executed, with a maximum of 3 retries; if the replication still fails after 3 retries, the repair process in step 4 is triggered, and the correct data is read from the backup block and written to the new block.

[0044] Marking invalid data: After successful copying, the controller marks valid data in the block to be reclaimed as "invalid" (updates the block's "valid page table," marking the corresponding page as invalid) to prevent other operations (such as normal writes) from accidentally overwriting invalid data. After marking is complete, the block to be reclaimed enters the "erasure queue."

[0045] Erasure and Resource Coordination: The erase controller erases blocks to be reclaimed according to the "first-in, first-out" principle, with an erase time of ≤2ms (TLC flash memory). Simultaneously, garbage collection and normal writing are coordinated using a "resource scheduling module": 70% of the IO bandwidth is reserved for normal writing, while garbage collection uses 30% of the bandwidth, avoiding excessive resource consumption by garbage collection that could lead to write latency. For example, when the normal write bandwidth reaches 1GB / s, the garbage collection bandwidth is limited to 428MB / s (1GB / s × 30% / 70%), ensuring that normal writing is not affected.

[0046] Example 1: Application of consumer-grade NVMe SSD (1TB capacity, PCIe 4.0 interface) This embodiment is designed for consumer-grade scenarios such as personal computers and game consoles. It details the hardware configuration, working process, and test results of the method of the present invention. The core requirement is to "balance cost, performance, and data consistency" to avoid the loss of game saves, documents, and other data when power is off.

[0047] 1.1 Hardware Configuration and System Setup Core hardware: It adopts the Inspiron IG5236 controller (1.6GHz, supports PCIe 4.0 x4 interface, theoretical bandwidth 8GB / s), and is equipped with 16 128GB TLC NAND flash memory chips (2 chips per channel, 8 channels in total), with a flash memory page capacity of 16KB, a block capacity of 128MB, and an erase / write cycle life of 3000 times.

[0048] Tiered caching: The high-speed cache layer is equipped with 2GB DDR4 DRAM (frequency 3200MHz, latency 8ns), divided into 8 storage partitions (1 partition per channel, capacity 256MB), and adopts a dual-block structure of "64KB data block + 4KB parity block"; the persistent cache layer uses two 128GB SLC NAND flash memory chips (independent of TLC chips, channels 9-10), with a total capacity of 256GB, a write speed of 500MB / s, and an erase / write cycle life of 100,000 times.

[0049] Power loss protection: It adopts two Panasonic ECQ-V series 5F / 16V supercapacitors (total capacity of 2.5F and withstand voltage of 32V after series connection), with a charge and discharge life of 100,000 cycles; the voltage detection module is ADI AD8210, with a sampling frequency of 1MHz and a power loss threshold of 10.8V (rated voltage of 12V).

[0050] Verification and Logs: The consistency verification module integrates a SHA-256 hardware acceleration engine (computation speed 1GB / s), allocates 2GB of space in the metadata area (dual replicas stored on channel 1 chip 1 and channel 8 chip 2), and stores logs in a fixed address in the metadata area (capacity 128MB).

[0051] 1.2 Work Process Data writing and multi-channel synchronization The user writes 16KB of game save data to the SSD via their personal computer: ① The controller first stores the data into the channel 1 partition of the cache layer (because the current load rate of channel 1 is 30%, it has the highest weight), and at the same time generates a 4KB XOR parity block, which is stored in the parity area of ​​the partition. ② The channel status monitoring module collects the load parameters of 8 channels: Channel 1 (historical success rate 99%, load rate 30%, physical distance 2cm, speed 400MB / s), Channel 2 (historical success rate 98%, load rate 50%, physical distance 2cm, speed 350MB / s), and the load rate of the remaining channels is 25%-45%; ③ The controller calculates the weights using a weighted round-robin algorithm. Channel 1 has the highest weight of 0.776, so 16KB of data is allocated to channel 1. ④ The controller sends a “synchronization ready command” to channel 1, and channel 1 responds with a “ready signal” within 20μs. The controller then issues a “write command”. ⑤ Channel 1 writes 16KB of data to the corresponding TLC flash memory chip (page address 0x1234). After the write is completed, the data is read and the SHA-256 hash value is calculated. It is compared with the original hash value before the write (stored in the cache) and it is consistent. The controller then sends a "write success signal" back to the controller, which then sends a "write complete" message back to the host.

[0052] Power outage emergency handling The user experiences an unexpected power outage during data writing (e.g., unplugging the power cord): ① The voltage detection module detects that the supply voltage drops to 10V (below the threshold of 10.8V) within 1μs and sends a "power failure warning signal" to the controller; ② The controller immediately suspends new write requests, starts the backup capacitor power supply, and reads the "migration status flag" of the cache layer (initially "incomplete"); ③ Migrate the unwritten data (total 2GB, including game saves and other temporary data) in the high-speed cache to the SLC persistent cache layer according to the FIFO principle, with a migration rate of 1GB / s; after each 64KB data block migration is completed, verify the XOR value through a check block to ensure that the data is not corrupted; ④ After 2 seconds of migration, all data migration is complete, the controller generates a "migration complete flag" and shuts off the power supply to the cache layer; ⑤ The next day, when the user powers on the device, the controller reads the "migration complete flag" and writes 2GB of data from the SLC to the TLC flash memory in batches. Before writing, the data is verified by SHA-256. All data is consistent and the game save can be read normally.

[0053] Periodic consistency check The SSD has an average daily write volume of approximately 80GB (below the 100GB threshold), and the verification cycle is set to 24 hours, starting at 3 AM every day. ① First-level check (cache layer): Read all data blocks and check blocks, calculate the XOR value and compare them. If a data block is found to have an inconsistent XOR value, it is repaired through the check block (reverse derivation of the erroneous byte). ② Second layer verification (persistent cache layer): Read the data block in SLC, calculate the SHA-256 hash value, compare it with the base value in the metadata area, and they are consistent; ③ Third-level verification (flash memory chip): Verify all TLC flash memory blocks. For example, for the 100th block (capacity 128MB), read the data of each 16KB page and calculate the hash value; it is found that the hash value of the 500th page is inconsistent with the baseline value (due to flash memory bit flipping). ④ The controller reads the backup data of the page from the SLC persistent cache layer (because the data of this page was written 3 days ago, it is still in SLC) and overwrites the abnormal page; after the repair is completed, the "verification timestamp" (marked as "2024-05-20 03:30:00") and "repair count" (the repair count of this block is 1) in the metadata area are updated.

[0054] Waste recycling process After the SSD had been running for 3 months, the percentage of valid data in the 50th TLC flash block dropped to 25% (below the 30% threshold), triggering garbage collection. ① The garbage collection controller selects block number 50 as the block to be collected and reads 32MB of valid data (including user documents and system cache) from it. ② Dual-path parallel replication is used to copy data to the 200th and 201st free blocks simultaneously, with a replication rate of 200MB / s; ③ After copying is complete, calculate the SHA-256 hash values ​​of blocks 200 and 201, compare them with the baseline value in the metadata area, and if the hash values ​​of the two new blocks are consistent, the copying is successful; ④ The controller marks the valid data in block 50 as "invalid" and updates the "valid page table" for that block; ⑤ The erase controller erases block number 50 (takes 2ms), freeing up 128MB of storage space; during this period, the user initiates game installation (write rate 300MB / s), the resource scheduling module reserves 70% bandwidth (210MB / s) for game installation, and garbage collection uses 30% bandwidth (90MB / s), so the game installation is smooth.

[0055] 1.3 Test Results The following are the results of 1000 tests conducted using third-party tools (CrystalDiskMark, Anvil's Storage Utilities): Data consistency: In 1000 unexpected power outage tests, only one data loss occurred (investigated to be a loose connection of the spare capacitor, which was resolved after rewiring), with a power outage data loss rate of 0.1% (which can be reduced to 0.01% after optimization); the data consistency compliance rate under normal operating conditions is 99.95%, and the multi-channel data fragmentation inconsistency rate is 0.05%, which is far lower than the 5% of existing consumer-grade SSDs.

[0056] Performance: Sequential write speed of 3500MB / s, random 4K write speed of 400MB / s, and average write latency of 20μs, which is 20% lower than the same configuration SSD using the traditional method (write latency of 25μs); Under high load writing (continuous writing of 100GB of data), the verification process only reduces the write speed by 5% (from 3500MB / s to 3325MB / s), with no noticeable stuttering.

[0057] Cost and lifespan: Hardware costs are reduced by 40% compared to supercapacitor solutions (from $15 to $9); SLC persistent cache has a performance degradation rate of only 10% after 100,000 erases and writes, meeting the 3-year lifespan requirement of consumer-grade SSDs (10 erases and writes per day, approximately 11,000 times in 3 years, far below the 100,000-cycle lifespan).

[0058] This embodiment achieves a balance of "low cost, high performance, and high consistency" in consumer-grade NVMe SSDs through the method of the present invention, meeting the daily usage needs of personal computers and game consoles.

[0059] Example 2: Application of enterprise-grade SATA SSDs (4TB capacity, SATA 3.0 interface) This embodiment is designed for enterprise-level scenarios such as servers and data centers. It optimizes the configuration based on embodiment 1, with the core requirements being "high reliability, high concurrency, and long lifespan" to ensure data consistency (such as financial transaction records and database data) during continuous 24 / 7 operation.

[0060] 2.1 Method Adjustment and System Configuration Core hardware upgrades: The controller adopts Samsung S1000 (2.0GHz clock speed, supports SATA3.0 interface, theoretical bandwidth 600MB / s), equipped with 32 128GB TLC NAND flash memory chips (2 chips per channel, 16 channels in total), with a flash memory page capacity of 16KB, a block capacity of 128MB, and an erase / write cycle life of 5000 times; a new "remote monitoring module" has been added, which supports uploading consistency parameters to the server management platform via the SMART protocol.

[0061] Tiered caching optimization: The high-speed cache layer is equipped with 4GB DDR4 DRAM (3600MHz frequency, 7ns latency) and adopts a "primary and backup partition" design (2GB primary partition, 2GB backup partition, and real-time synchronization of primary partition data to backup partition) to avoid single DRAM chip failure; the persistent cache layer uses 4 256GB SLC NAND chips (total capacity 1TB, channel 17-20) and supports RAID1 backup (data is stored in 2 SLC chips at the same time), with the migration rate increased to 2GB / s.

[0062] Enhanced power loss protection: Employs four 10F / 20V supercapacitors (total capacity 5F and withstand voltage 80V after series connection), coupled with a 10Wh lithium battery backup power supply (supports 10 minutes of power supply); the voltage detection module sampling frequency is increased to 2MHz, and the power loss threshold is set to 11V (for earlier protection trigger); data migration adds a "verification-migration-re-verification" process: DRAM data is verified before migration, and SLC data is verified after migration.

[0063] Verification and garbage collection optimizations: The verification cycle threshold has been increased to 500GB / day (for enterprises with high daily write volumes), and reduced to 30 minutes for write volumes ≥500GB / day, with "incremental verification" (verifying only data from the last 24 hours); the verification algorithm now includes a new BCH error correction code (8-bit / 512-byte error correction capability), which can directly correct bit flip errors; garbage collection uses "parallel processing" (processing 8 blocks to be reclaimed simultaneously), the effective data ratio threshold has been reduced to 20%, data replication is "dual-path parallel + mutual verification", and resource scheduling reserves 70% of IO bandwidth for normal writes.

[0064] 2.2 Work Process High-concurrency writes and multi-channel synchronization A server in a financial data center is equipped with this SSD and can simultaneously handle transaction record write requests from 100 clients (total write bandwidth 1GB / s): ① The channel status monitoring module collects the load parameters of 16 channels every 5μs. Channel 5 has 3 errors in the past 24 hours (other channels ≤ 1 error), so its weight has dropped to 0.6 (other channels ≥ 0.8), and its task allocation ratio has decreased. Channel 8 has a load rate of 20% and a historical success rate of 99.5%, with a weight of 0.9, and has become the main task channel. ② The controller allocates data according to a weighted round-robin algorithm, with 16 channels processing in parallel. The transmission rate of each channel is stable at 62.5MB / s (1GB / s ÷ 16), with a load deviation of 8%. ③ The controller sends a synchronization command to 16 channels. Within 30μs, all channels respond with a ready signal (30μs timeout for enterprise-level configuration) and then issue a write command. ④ Each channel writes transaction records (1KB per transaction, totaling 1 million transactions) to TLC flash memory. After writing, the data is verified by SHA-256, and all data is consistent. After 24 hours of high-concurrency writing (total writing volume of 10TB), the data fragmentation inconsistency rate is 0.002%, which meets the needs of financial scenarios.

[0065] Long-term power failure protection Data center mains power outage (lasting 5 minutes): ① The voltage detection module detects that the voltage drops to 10.5V (below the threshold of 11V) and triggers protection within 1μs; ② The controller starts the supercapacitor and lithium battery to power the system. First, it verifies that the 4GB of data in the DRAM is normal through XOR check, and then migrates it to the SLC cache at a rate of 2GB / s (RAID1 mode, writing to 2 SLC chips at the same time). ③ The migration takes 2 seconds (4GB ÷ 2GB / s). After completion, the two data copies in the SLC are checked using SHA-256. If they match, a "migration complete flag" is generated. ④ The lithium battery continuously powered the SLC cache for 5 minutes without data loss; after power was restored, the SSD wrote 4GB of data from the SLC to the TLC flash memory in batches. After the writing was completed, the data consistency rate reached 100% and no transaction records were lost.

[0066] Dynamic verification and error correction This SSD has an average daily write volume of 600GB (≥500GB threshold), a verification period of 30 minutes, and uses incremental verification. ① When verification starts, only 600GB of data written in the past 24 hours (approximately 38,400 16KB pages) is read, and the SHA-256 hash value of each page is calculated; ② Twelve data pages were found to have bit flip errors. Ten of them were directly repaired using BCH error correction codes (without needing to read backups), while the other two were more serious (flipping 10 bits) and were repaired by reading data from the SLC RAID1 replica. ③ The entire verification process took 28 minutes and occupied 20% of the IO bandwidth (120MB / s). The server's database query service (bandwidth 400MB / s) only experienced a 5% increase in latency (from 10ms to 10.5ms), with no service interruption.

[0067] Parallel waste recycling After the SSD had been running for 6 months, the percentage of valid data in the 20 flash blocks dropped to 20%, triggering parallel garbage collection. ① The waste recycling controller simultaneously selects 8 blocks to be recycled (numbered 100-107) and reads 32MB of valid data from each block (total 256MB); ② Dual-path parallel replication is used to copy the data of each block to be reclaimed to two new blocks simultaneously (a total of 16 new blocks), with a replication rate of 400MB / s; ③ After copying is complete, calculate the hash value of the new block and compare it with the baseline value. If the hash values ​​of the two new blocks are consistent, the copying is successful. ④ Mark the data to be reclaimed as invalid, erase 8 blocks to be reclaimed in parallel (takes 16ms), and release 1024MB of storage space; during this period, the server's file transfer service (bandwidth 500MB / s) occupies 70% of the IO bandwidth (350MB / s), garbage collection occupies 30% (150MB / s), and the file transfer rate is stable.

[0068] 2.3 Long-term operating results This SSD operated continuously in the data center for one year (8760 hours), and statistics were obtained through a remote monitoring platform: Consistency metrics: 99.99% success rate for power-off migration (only one migration failure due to lithium battery failure, which was resolved after battery replacement); an average of 2 verification failures per week (all successfully repaired); 0 instances of garbage collection anomalies; data consistency compliance rate consistently above 99.99%, meeting the "4 nines" reliability requirements of financial data centers.

[0069] Performance and lifespan: Average sequential write speed of 550MB / s (SATA3.0 interface limit), random 4K write speed of 90MB / s, average write latency of 15μs; write amplification factor (WAF) controlled within 1.2 (industry average 1.5-2.0), TLC flash memory has an average of 80 erase / write cycles per day, and the expected lifespan is up to 6 years (5000 cycles ÷ (80 cycles / day × 365 days / year) ≈ 6.8 years).

[0070] Operational efficiency: The remote monitoring platform displays consistency parameters in real time. When the number of verification failures is ≥5 times / day, an alarm is automatically sent. Administrators can remotely view logs to locate problems, improving operational efficiency by 50% and avoiding the time cost of on-site maintenance.

[0071] This embodiment optimizes the configuration to enable enterprise-grade SATA SSDs to maintain high consistency and stability under high concurrency and long-term operation scenarios, meeting the needs of critical businesses such as data centers and financial transactions.

[0072] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of protection claimed by the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method of controlling data consistency of a solid state disk, characterized by, Comprising the following steps: Step 1: Construct a hierarchical cache architecture, including a cache layer and a persistent cache layer; the cache layer uses DRAM to temporarily store data to be written to the flash memory, and uses a "data block-check block" double block structure, each data block corresponds to a check block for storing the XOR check value; the persistent cache layer uses SLC NAND to temporarily store the cache layer data during power failure; Step 2: Multi-channel write synchronization control, real-time acquisition of the write queue length, transmission rate and idle time of each flash channel, task allocation based on weighted round robin algorithm, channel load deviation ≤10%; send synchronization instruction to the target channel before writing, send write instruction after all channels feedback ready signal, if not feedback within 50-100us, judge channel abnormal, task transfer to idle channel and log; Step 3: Power failure data emergency processing, real-time monitoring of power supply voltage by voltage detection module, trigger protection when lower than 0.9 times rated voltage; pause new write, start backup capacitor power supply, migrate cache data to persistent cache according to FIFO, resume next time if migration is interrupted, generate flag and turn off cache power supply after completion; Step 4: Periodic consistency check and repair, check flash block data according to 1-24 hours configurable period; Calculate the current data block SHA-256 hash value and compare it with the reference hash value in the metadata area, if not consistent, read the correct data from the persistent cache / backup block for overwrite, update the check timestamp after repair. 2.The solid state disk data coherency control method of claim 1, wherein: In the "data block-check block" double block structure of the cache layer in step 1, the data block capacity is 64KB, the check block capacity is 4KB, the XOR check value of the corresponding check block is updated synchronously with each data block written, used for preliminary detection of data consistency within the cache layer. 3.The solid state disk data coherency control method of claim 1, wherein: In step 2, the "historical success rate" of the weighted round robin algorithm is the proportion of completed write tasks in the channel in the last 24 hours, the "current load rate" is the ratio of the current write queue length to the maximum queue length, and the "physical distance" is the straight-line distance between the channel flash chip and the controller. 4.The solid state disk data coherency control method of claim 1, wherein: In step 3, the backup capacitor uses super capacitor, the total withstand voltage after series connection is not less than 1.2 times the rated power supply voltage of the solid state disk, and the charge and discharge life is ≥100,000 times, which meets the long-term use requirement. 5.The solid state disk data coherency control method of claim 1, wherein: In step 4, the reference hash value is generated when the data is first written to the flash memory, and the double copies of the metadata area are stored in different flash channels to avoid data loss caused by single channel failure. 6.The solid state disk data coherency control method of claim 1, wherein: It also includes a garbage collection consistency control step: copy the valid data of the block to be recycled to a new block during garbage collection, calculate the SHA-256 hash value of the new block and compare it with the reference value, if not consistent, retry copying up to 3 times, if retry fails, trigger data repair process; if consistent, mark the valid data of the block to be recycled as invalid, erase the block to be recycled to release space, and garbage collection and normal writing coordinate IO resources through resource scheduling module. 7.The solid state disk data coherency control method of claim 1, wherein: In step 3, during the data migration process, the XOR check value of the migrated data is verified through the check block of the cache layer after each data block migration, to ensure that the migrated data is not damaged. 8.The solid state disk data coherency control method of claim 1, wherein: The consistency check in Step 4 adopts a "layered check" strategy: first check the cache layer data, then check the persistent cache layer data, and finally check the flash chip data, layer by layer to ensure data consistency. 9.The solid state disk data coherency control method of claim 1, wherein: The multi-channel write synchronization control in Step 2 also includes a "rate adaptation" mechanism: adjust the data fragment size according to the real-time transmission rate of the channel, channels with a transmission rate ≥ 500MB / s are allocated 16KB fragments, and channels with a transmission rate < 500MB / s are allocated 8KB fragments, to avoid large fragments causing write delays. 10.The solid state disk data coherency control method of claim 1, wherein: It also includes a data write confirmation step: after writing data to the flash, read the written data and calculate the SHA-256 hash value, and compare it with the original data hash value before writing; consistent returns a write success signal, inconsistent reinitiates the write, up to 3 retries, retry failure triggers Step 4 consistency repair process.

Images