Data storage method based on partition namespace solid state disk and persistent memory

Abstract

The invention provides a data storage method based on a partition namespace solid state disk and a persistent memory. The data storage method comprises the following steps: designing a structure of small-granularity system metadata; performing persistent storage on the small-granularity system metadata through two alternately used circular queues maintained in a persistent internal PM, and performing ZNS SSD disk refreshing; setting a reading strategy to ensure correct reading of cross-physical equipment data; redesigning a distribution rule of a physical partition Zone of the ZNS SSD by adopting a scoring mechanism; multi-thread writing of data is achieved based on the distribution rule, and competition between threads for Zone locks is reduced in a PM asynchronous storage mode; according to the method, metadata in a Rokcdb system realized on the basis of an LSM tree is aggregated into a persistent memory PM; the persistent memory PM has the characteristics of high-speed reading, no loss in power failure, high capacity and the like, and is a very suitable physical medium as a cache; and the data can be quickly read while the risk that the data is lost due to power failure is avoided.

Description

Technical Field

[0001] This invention belongs to the field of data storage, specifically relating to a data storage method based on a partitioned namespace solid-state drive and persistent memory. Background Technology

[0002] The widespread adoption of IoT terminals, the training and iteration of AI models, and the high frequency of financial transactions are collectively driving a surge in global data volume. According to predictions by IDC and Statista, the total global data volume will reach 221 ZB in 2026 and climb to 2142 ZB by 2035, with the proportion of data from industrial sensors collecting equipment in real time, e-commerce platform transaction logs, and autonomous driving road condition images continuing to expand. This type of big data is characterized by massive volume, high speed, diversity, and low value density, placing stringent demands on the timeliness, reliability, and throughput capabilities of storage systems.

[0003] Traditional hard disk drives (HDDs) are no longer suitable for high-throughput scenarios due to their high seek latency and limited concurrency performance. For example, in securities trading systems, the millisecond-level latency of HDDs cannot support the writing of tens of millions of transaction data per day. Solid-state drives (SSDs) based on NAND flash memory have thus become the mainstream. SSDs, with their electronic storage characteristics, achieve microsecond-level latency and high throughput, performing exceptionally well in scenarios such as peak order processing during e-commerce promotions and real-time data analysis. However, traditional SSDs reveal significant shortcomings in large-scale mixed read / write scenarios. Due to the non-overwriteable nature of NAND flash memory, data updates require writing to a new location, leading to severe write amplification. For example, in the 24 / 7 data writing scenario of a video surveillance platform, the write amplification factor can reach more than 5 times, resulting in wasted I / O resources. Simultaneously, garbage collection (GC) requires reserving 7%-28% redundant operation space (OP), significantly increasing storage costs. Taking a 10PB-level cloud storage cluster as an example, the hardware waste of OP space is extremely serious.

[0004] To address these issues, Streaming SSDs (S-SSDs) and Open Channel SSDs (OCSSDs) have emerged. S-SSDs aggregate data with the same lifecycle through Stream IDs, theoretically reducing GC overhead. However, in smart healthcare medical record data storage, due to inaccurate data lifecycle labeling, their actual performance is not significantly different from traditional SSDs, and the additional mapping management increases the DRAM burden. OCSSDs offload FTL functionality to the host, reducing write amplification, but in distributed database deployments, the host needs to develop complex address translation logic, resulting in extremely high compatibility and debugging costs.

[0005] Partition Namespace SSDs (ZNS SSDs) emerged as a load balancing solution. They enforce sequential writes through Zone partitions, with the host managing write pointers and zone resets, eliminating wasted OP space. This characteristic is a natural fit with the LSM-Tree log structure. LSM-Trees store writes in a MemTable first, then sequentially flush them to SST files when full; the compression and merging process is also sequential, significantly improving throughput on ZNS SSDs. Currently, mainstream optimizations are based on RocksDB databases and the ZenFS file system. ZenFS optimizes data layout by dividing the lifecycle of SST files, but in the burst write scenarios of e-commerce flash sales, Zone allocation latency can cause foreground write pauses; furthermore, fine-grained metadata writes can lead to severe partition fragmentation and write amplification in log systems.

[0006] The maturity of persistent memory (PM) provides a new path to overcome bottlenecks. It combines DRAM-level speed with storage persistence; for example, Intel Optane PM can achieve microsecond-level data persistence in real-time financial clearing scenarios. Addressing the shortcomings of ZenFS, introducing PM to replace part of the DRAM in a hybrid architecture can solve the zone allocation latency problem through the persistence characteristics of PM, while also aggregating small metadata file writes, becoming a key direction for optimizing LSM-Tree storage systems.

[0007] In summary, existing storage technologies suffer from numerous challenges in performance adaptation, resource utilization efficiency, and data management adaptability: HDD performance is insufficient to meet high throughput demands; traditional SSDs are limited by write amplification and OP space, leading to resource waste; S-SSDs and OCSSDs face the dilemma of complex data management or poor adaptability; even the combination of ZNS SSDs and LSM-Trees has inherent advantages, but still suffers from shortcomings such as zone allocation latency and metadata fragmentation. This urgently necessitates an innovative hybrid storage architecture and data layout optimization method to overcome these challenges and meet the ever-growing storage and processing demands of the big data era. Summary of the Invention

[0008] To address the problems of existing technologies and achieve efficient utilization and storage of physical storage space, overcoming issues such as write amplification and write pauses, this invention proposes a data storage method based on partitioned namespace solid-state drives (SSDs) and persistent memory (PM). This method includes: designing a structure for small-granularity system metadata; persistently storing the small-granularity system metadata by maintaining two alternately used circular queues in the persistent memory (PM) and flushing it to the ZNS SSD; setting a read strategy to read data; redesigning the allocation mechanism of the ZNS SSD's physical partition zones using a scoring mechanism; performing multi-threaded writing of data through the allocation mechanism and reducing thread contention for zone locks by asynchronously storing the PM.

[0009] The beneficial effects of this invention are:

[0010] This invention aggregates metadata from the Rocksdb system, which is based on an LSM tree, into persistent memory (PM). Persistent memory (PM) possesses characteristics such as high-speed read speeds, non-data loss during power outages, and large capacity, making it a very suitable physical medium for caching. It ensures data is not lost during power outages while also providing the advantage of fast data retrieval. This invention uses persistent memory (PM) to cache data during Rocksdb thread contention and employs a novel scoring mechanism for allocating ZNS SSD physical partitions (Zones). Because PM combines DRAM-level microsecond-level access speeds with non-volatility, it serves as a dedicated cache layer for contentionable data, replacing the contention buffer in traditional DRAM and reducing thread switching and lock waiting overhead. Furthermore, the new physical partition allocation method is more rational, balancing physical device wear and lifespan while significantly reducing write amplification issues caused by garbage collection during LSM tree merging operations. Attached Figure Description

[0011] Figure 1 This is a diagram illustrating the overall architecture of data storage distribution in this invention.

[0012] Figure 2 This is a schematic diagram of the small file aggregation process in this invention;

[0013] Figure 3 This is a schematic diagram of the multi-threaded lock merging lock contention solution in this invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] A data storage method based on partitioned namespace solid-state drives (SSDs) and persistent memory (PM) includes: designing a structure for fine-grained system metadata; persistently storing the fine-grained system metadata by maintaining two alternately used circular queues in the persistent memory (PM) and flushing it to the ZNS SSD; setting a read strategy to read data; redesigning the allocation mechanism of the physical partition zone of the ZNS SSD using a scoring mechanism; performing multi-threaded writing of data through the allocation mechanism and reducing contention for zone locks between threads by asynchronously storing the PM.

[0016] To make the objectives and advantages of the present invention more readily apparent, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0017] Example 1

[0018] This invention designs a hybrid data storage structure based on Dynamic Random Access Memory (DRAM), Persistent Memory (PM), and Partition Namespace Solid State Drive (ZNS SSD), such as... Figure 1 As shown. The three types of physical storage devices included in this structure and the data they store are as follows: In DRAM, the write-ahead log (WAL) of the log structure merge tree (LSM), the memory table (memtable), and the immutable memory table (Immutable memtable) are stored; in persistent memory (PM), small files after aggregation are cached, as well as ordered string tables (SSTable) data generated when persistent threads conflict; in the partitioned namespace solid-state drive (ZNS SSD), the core ordered string table (SSTable) files of the log structure merge tree (LSM) are stored. Except for some SSTable data retained in persistent memory (PM), all other SSTable files are persistently stored through ZNS SSD.

[0019] Next, we will combine Figure 1 Explain in detail where the data is written and how it is read.

[0020] When the system receives a write operation command, the data first enters the Memtable in DRAM. As the active data structure of the LSM tree in memory, the Memtable is responsible for receiving and temporarily storing newly written data in real time, supporting efficient write operations. As data accumulates in the Memtable, it transitions to an Immutable state when it reaches a preset size threshold or meets specific trigger conditions. A memory table in the Immutable state means that its data no longer accepts new writes and can only be read; this transition prepares for subsequent data persistence to storage devices.

[0021] To ensure data security during data transfer within DRAM, the Write-Ahead Log (WAL) records relevant operations synchronously. The WAL records write operations to the Memtable, allowing data recovery from the Memtable even in the event of a system failure, preventing data loss.

[0022] For data that needs to be persisted, it will further interact with the persistent memory (PM). The PM acts as a cache. On the one hand, it caches small files that have been aggregated, which reduces storage fragmentation and improves storage and access efficiency. On the other hand, when conflicts occur in the persistence threads, the generated SSTable data will also be temporarily stored in the PM.

[0023] The ZNS SSD, a partitioned namespace solid-state drive, is the final persistent storage medium for the LSM tree core SSTable files. Except for the portion of SSTable data retained in the PM (Performance Management Panel), all other SSTable files are persisted via the ZNS SSD. The ZNS SSD employs a partitioned namespace architecture, enabling more efficient management of storage areas. For datasets with specific access patterns, such as SSTable files, it optimizes write and read performance, reduces wear and tear on the storage media, and extends device lifespan.

[0024] The redesigned allocation mechanism for ZNS SSD physical partition zones using a scoring system includes: determining the lifecycle of the corresponding SSTable file based on the LSM tree hierarchy; initially filtering out ZNSSSD physical partition zones with the same lifecycle based on this lifecycle; traversing all filtered zones, selecting the zone with the highest score that is unlocked, and if not found, selecting a zone with a smaller lifecycle difference according to the degradation strategy; and updating the score of the allocated zone after file storage is completed.

[0025] In this embodiment, the read operation includes: when the system needs to read data, it first searches the various levels of the LSM tree structure. The LSM tree uses a hierarchical storage method, where newly written data is initially stored in higher levels such as L0, and gradually sinks to lower levels such as L1 and L2 as data is continuously merged and persisted. During the search process, the system combines the memory table in DRAM, the cached data in PM, and the persistent SSTable file in ZNS SSD to comprehensively obtain the required data, ensuring that the read operation can efficiently and accurately obtain the target data.

[0026] In addition, the PM area in the diagram also contains metadata files. These metadata files are used to record relevant metadata information of the storage system, such as data location mapping and the status of each storage device. This provides metadata-level support for the efficient operation of the entire hybrid data storage structure, ensuring that data management, reading and writing operations can be carried out in an orderly manner.

[0027] Example 2

[0028] This invention develops a small file aggregator, which is used to aggregate and persist small files that waste writing space and time in the system, such as... Figure 2 As shown, this structure mainly includes three important components: a file filter, an alternating circular queue, and metadata reconstruction. Because the granularity of small files generally does not exceed the minimum granularity of the ZNS SSD namespace solid-state drive, the writing method is aligned to the partition block size. On the other hand, because multiple writes of small files require frequent device opening for writing, this severely impacts the system's I / O performance.

[0029] The main structure of the small file aggregator is as follows.

[0030] File Filter: This is the first step in the small file aggregation process. Its core function is to accurately filter out small files that need aggregation while excluding large files that do not require optimization, thus avoiding unnecessary computation that consumes system resources. In storage systems, not all files need to be aggregated. Some system metadata and log files often have large file sizes, typically exceeding the partition block size of the ZNS SSD. These large files are already well-suited to the storage characteristics of the ZNS SSD. Forcing them into the aggregation process will not only fail to improve storage efficiency but will also increase the complexity of data processing and may even lead to a decrease in read performance due to data splitting.

[0031] Based on this, the file filter employs a threshold-based judgment mechanism for precise filtering: using the partition block size of the ZNS SSD as the core judgment threshold, it scans and performs volume checks on files to be stored in the system. When a file size is detected to be less than or equal to the threshold, it is determined to be a small file that needs aggregation and is included in the subsequent aggregation process; when a file size is detected to exceed the threshold, it is directly determined to be a large file that does not need aggregation, skips the aggregation stage, and is written to the ZNS SSD according to the normal storage process. This filtering method ensures the targeting of aggregation operations and avoids the invalid processing of large files, improving the overall operating efficiency of the aggregation system from the source. Furthermore, to adapt to storage needs in different scenarios, the file filter also supports dynamic threshold configuration. Users can flexibly adjust the judgment threshold according to the actual ZNS SSD model used and the business scenario, further enhancing the system's applicability.

[0032] Alternating Circular Queues: This is the core support for the small file aggregator to achieve parallel foreground writing and background persistence, and it is also a key structure to ensure that the system's read and write performance is not disconnected. This structure is not a traditional single queue, but consists of two independent circular queues, both of which are deployed in persistent memory (PM). The persistent memory feature combines the high-speed access speed of memory with the persistence capability of storage, providing a hardware foundation for the efficient operation of the queues. It can ensure the rapid temporary storage of foreground written data, while avoiding the loss of temporary data due to unexpected events such as power outages.

[0033] These two circular queues serve as the write queue and persistent queue, respectively, and their functions are dynamically switched through an alternating switching mechanism. The specific workflow is as follows: During normal system operation, one queue is in write queue state, specifically responsible for receiving small file data transmitted by the foreground thread. The foreground thread does not need to wait for data to be written directly to the ZNS SSD; it only needs to quickly write the small file data to this queue to complete the data submission and synchronization operations, greatly shortening the waiting time of foreground services and improving the response speed of user operations. When the amount of data in the write queue reaches the preset capacity limit, the system triggers the queue switching mechanism: first, the currently full write queue is switched to the persistent queue; then, a background thread is started, which aligns the aggregated data in the persistent queue according to the partition block size of the ZNS SSD before flushing the entire data to the ZNS SSD device; simultaneously, the other queue, which was originally in an idle state, immediately switches to the new write queue to continue receiving new small file data transmitted by the foreground thread.

[0034] This alternating cyclical working mode completely breaks through the traditional serial processing bottleneck of write-then-persistence, realizing parallel execution of foreground data writing and background data persistence. The foreground thread can continuously write new data without waiting for the background persistence operation to complete, while the background thread steadily completes data persistence without affecting foreground business operations. The two operate independently and collaborate efficiently. Furthermore, to further optimize queue scheduling efficiency, the alternating cyclic queue also incorporates a capacity warning mechanism: when the amount of data in the write queue reaches 80% of its capacity limit, the system sends a warning signal to the background thread in advance. This warning information obtains the execution status of the background thread and, by agreeing on the status of the persistence queue, determines whether to wait for the background thread to complete the persistence operation. Finally, when the write queue is full, the persistence operation can be started immediately, minimizing the waiting time after queue switching and further improving the overall throughput of the system.

[0035] Metadata Reconstruction Module: This is a crucial step in ensuring that aggregated data can be correctly read by the system. In traditional storage models, the system reads metadata using a block-size aligned read mechanism. This means that after each metadata read, it automatically aligns to the ZNS SSD's block size before reading the next metadata record. However, when small files are aggregated, the original metadata structure is broken. The aggregated file contains metadata information from multiple small files, and the total length of this metadata may not perfectly match the ZNS SSD's block size. If the traditional block-size aligned method is still used, read misalignment issues are highly likely to occur, either missing some metadata or reading irrelevant garbage data. This prevents the system from correctly identifying the various small files in the aggregated file, ultimately affecting data availability.

[0036] To address this issue, the metadata reconstruction module redesigns and optimizes the aggregated metadata structure through a tail identifier addition mechanism. Specifically, when aggregating metadata from multiple small files into a single entity, the module adds an alignment identifier field to the end of the aggregated metadata. This field records the relative position of each metadata entry to the next through specific binary encoding, thus determining whether to use an alignment-based or compact stacking approach during subsequent readings. That is, if the metadata tail identifier indicates alignment, the system reads it using the traditional block size alignment method; if the metadata uses a compact stacking approach, the system reads it precisely from the beginning of the next metadata entry based on the length information in the identifier field, avoiding misreading.

[0037] Meanwhile, to ensure that the identifier field is not misinterpreted as metadata content, the module also employs a dual protection mechanism of a special delimiter and a checksum. A system-preset special delimiter is added before the identifier field to clearly define its starting position; a checksum is added after the identifier field to verify its integrity. When the system reads the data, it first locates the identifier field using the special delimiter, and then verifies whether the identifier field has been tampered with or damaged using the checksum. Only after successful verification will the metadata be read according to the alignment of the identifier field record. This design not only completely solves the problem of misreading aggregated metadata but also ensures the reliability of the identifier field itself, further enhancing the data security and availability of the entire aggregation system.

[0038] Example 3

[0039] This invention also provides a novel data placement algorithm, aimed at reducing write pauses caused by physical partition lock contention during multi-threaded merging operations and garbage collection write amplification caused by degraded allocation. The system operation flow is as follows: Figure 3 As shown, the allocation algorithm mainly consists of four parts: ZNS SSD zone scoring, hybrid allocation based on lifecycle and scoring mechanisms, data temporary storage during contention, and cross-device data reading. The ZNS SSD zone scoring mechanism serves as the foundation, quantifying multi-dimensional parameters such as remaining partition space, write frequency, and historical contention rate to accurately match the appropriate physical partition when threads write data, reducing subsequent problems caused by unreasonable allocation from the source. On top of this, the hybrid allocation mechanism based on lifecycle and scoring mechanisms further optimizes the physical partition allocation logic. It dynamically adjusts the strategy by combining file lifecycle characteristics and Zone scoring results, making the allocation more refined and effectively alleviating the problem of excessive differences in file lifecycle distribution within a Zone, thereby reducing the write amplification effect during garbage collection. However, although the first two modules significantly reduce allocation problems, write pauses and Zone downgrading may still occur due to multiple threads contending for the same Zone. Therefore, a data temporary storage mechanism for lock contention scenarios has emerged. When lock contention occurs, the algorithm temporarily stores the data to be written in persistent memory (PM) to avoid thread blocking and significantly reduce the probability of write pauses. Furthermore, since SSTable files are distributed across different physical devices such as ZNS SSDs and PMs, the algorithm has redesigned the data reading process, enabling cross-device data access through a unified read interface while ensuring data consistency during the reading process, thus guaranteeing the integrity and reliability of the entire data placement and access process: The implementation steps of this example will be detailed below:

[0040] S1: Scoring process for each Zone of the Named Partition ZNS SSD. A background thread performs scoring operations at fixed intervals to ensure that the Zone score reflects its current status in real time. The scoring process considers four core parameters: 1. Physical partition usage: the ratio of occupied space to total capacity, ranging from 0 to 1. Lower usage indicates a stronger ability to accept new data. 2. Single lifecycle difference: calculating the absolute difference between the actual lifecycle of the last stored SSTable file in the Zone and the Zone's rated lifecycle, then normalizing to the 0-1 range. A smaller difference indicates a higher degree of matching between the Zone and the file lifecycle. 3. Average lifecycle difference: calculating the difference between the average lifecycle of all SSTable files in the Zone and the Zone's initial lifecycle, then normalizing. This parameter reflects the overall adaptability of file lifecycles within the Zone. 4. File density: the ratio of the number of currently stored SSTable files in the Zone to the maximum supported number, ranging from 0 to 1. Lower density indicates more abundant remaining storage resources in the Zone. The scoring formula is as follows:

[0041]

[0042] in, The score for Zone. The weights of the four parameters satisfy the following conditions: . This refers to the physical partition usage. This represents the normalized difference in a single lifecycle. This refers to the file quantity density. For physical partition usage, This represents the already used capacity of the Zone, where Z is the Zone partition. This represents the total capacity of the Zone. The lifecycle of the last time a file was written to the Zone. For the current actual lifecycle of the Zone, The difference between the maximum lifespan and the minimum lifespan. Store the average lifetime of all SSTable files for this zone. The number of files currently stored in the Zone. This is the maximum number of files that can be stored, calculated based on the standard SSTable file size and the Zone size.

[0043] S2: Based on S1, execute Figure 3 The steps.

[0044] Step 1: The thread reads the SSTable file of an LSM tree in the L2 layer and the two SSTables with the most overlapping keys in the L3 layer into memory.

[0045] Step 2: Merge the data in memory into one or more ordered SSTable files.

[0046] Step 3: Multiple merging threads need to write the merged result to the physical partition of the ZNS SSD. At this point, the designed allocation strategy algorithm is required.

[0047] Step 4: The thread first performs a rough lifecycle filtering: based on the expected storage duration of the SSTable file, the Zones are divided into three categories according to their rated lifecycles: "Short-lifecycle Zone," "Medium-lifecycle Zone," and "Long-lifecycle Zone," filtering out the set of Zones that match the current SSTable file's lifecycle type. Then, it enters the score-based precise matching stage: Prioritizing the allocation of high-scoring Zones to the filtered set. If a high-scoring Zone cannot be obtained from the filtered set, a fallback allocation mechanism is triggered to attempt to acquire locks on other lifecycle Zones. If the above operations successfully acquire the Zone lock, then proceed to Step 5;

[0048] Step 5: Write the data to the allocated Zone. If no suitable Zone is found within the lifetime, the inability to acquire the Zone lock indicates that lock contention has occurred. This is because thread blocking may occur due to Zone allocation conflicts during the writing process of the LSM tree's SSTable file.

[0049] Specifically, when the system allocates a ZNS SSD Zone to a thread, if the target Zone is already occupied by another thread, the current thread needs to enter a loop waiting state, continuously trying to acquire a lock for the appropriate Zone. This lock contention directly causes write operations to pause, making the thread stuck in the Zone acquisition phase and unable to advance subsequent tasks. To address this, the algorithm designs a "peak-shaping temporary storage strategy," which selects to execute another step, 6.

[0050] Step 6: When a thread fails to acquire the lock multiple times in a row, the system temporarily writes the SSTable file data that the thread is currently writing to to a dedicated cache pool in persistent memory (PM). The PM cache pool uses a "dual-trigger mechanism" for data persistence, as described in steps 7, 8, and 9. Specifically, this includes: starting a synchronization task at a fixed period, and triggering immediate synchronization when the cached data volume reaches a preset threshold. Both require attempting to acquire the corresponding Zone's lock first. If successful, the SSTable data in the PM is synchronized to the ZNS SSD physical partition; the only difference is the timing of the triggering.

[0051] Reducing thread contention for Zone locks by asynchronously storing PMs includes: when a thread writing to the SSTable file fails to acquire the lock for the target Zone, the loop waiting logic is terminated, and the data to be written is temporarily stored in a dedicated buffer of the persistent memory PM; trigger conditions are set, and a background thread monitors the trigger conditions in real time. When the trigger conditions are met, the background thread writes the data in the PM buffer in batches to the corresponding Zone. Trigger conditions include the lock of the corresponding Zone in the PM buffer being released or the amount of data in the PM buffer reaching a preset threshold.

[0052] S3: Ensuring cross-device data consistency during reads. Because some SSTables in the LSM tree are partially stored in ZNSSSD and partially in persistent memory (PM), reading data from these SSTables may require traversing two physical devices. To ensure data consistency and prevent erroneous reads, this invention employs two safeguards: first, version control, assigning a globally unique version number to each data block and comparing it with the version number recorded in the index table during reads; second, double verification, calculating the CRC checksum after reading data, comparing it with the checksum in the metadata, and then verifying whether the data length matches the expected length. If both checks pass, the data is returned. Furthermore, due to the different physical devices, a multi-threaded read-and-merge approach is used to improve data read speed.

[0053] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data storage method based on a partitioned namespace solid-state drive and persistent memory, characterized in that, include: Design the structure of metadata for small-granularity systems; Fine-grained system metadata is persistently stored by maintaining two alternating circular queues in the PM within the persistence layer, and flushed to ZNS SSDs. Set a read strategy to read data; redesign the physical partition zone allocation mechanism of ZNS SSD using a scoring mechanism; perform multi-threaded writing of data through the allocation mechanism and reduce the contention for zone locks between threads by using asynchronous storage PM.

2. The data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 1, characterized in that, The structure of the small-granularity system metadata includes: a header, a double circular queue structure, a PM pointer, a PM memory pool, and a tail, with the header and tail being reconstructed.

3. The data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 2, characterized in that, The header and tail are restructured, including adding a new length field to the header and adding a storage method for the next data in the tail field; the new length field contains the actual data length and the tail length; the newly added tail field determines whether to read the next data directly in sequence or by skipping the block size.

4. The data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 2, characterized in that, In the dual-circular queue structure, the two circular queues work alternately: when one queue accumulates data to its limit, a background thread submits the aggregated multiple small files to the ZNS SSD; the other queue continues to receive new requests, ensuring continuous system availability and write performance.

5. A data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 1, characterized in that, The read strategy includes: setting a threshold to determine if the file size is smaller than the physical block size of the ZNS SSD or the set threshold; selecting the circular queue being written to and determining if the current queue is full; for a full queue, a background thread will perform a ZNS SSD flush operation, while a foreground thread will switch to a ready empty queue for persistent storage; when reading data, it will determine whether the data is in persistent memory (PM) or in the ZNS SSD, and read the corresponding physical location based on the data's location.

6. The data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 1, characterized in that, The redesigned allocation mechanism for ZNS SSD physical partition zones using a scoring system includes: determining the lifecycle of the corresponding SSTable file based on the LSM tree hierarchy; initially filtering out ZNS SSD physical partition zones with the same lifecycle based on this lifecycle; traversing all filtered zones, selecting the zone with the highest score that is unlocked, and if not found, selecting a zone with a smaller lifecycle difference according to the degradation strategy; and updating the score of the allocated zone after file storage is completed.

7. A data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 6, characterized in that, The formula for scoring the assigned ZNS SSD physical partition Zone is as follows: ； in, The score for Zone The weights of the four parameters, For physical partition usage, This represents the already used capacity of the Zone, where Z is the Zone partition. This represents the total capacity of the Zone. For normalized single lifecycle differences, The lifecycle of the last time a file was written to the Zone. For the current actual lifecycle of the Zone, The difference between the maximum lifespan and the minimum lifespan. Store the average lifetime of all SSTable files for this zone. For file quantity density, The number of files currently stored in the Zone. This is the maximum number of files that can be stored, calculated based on the standard SSTable file size and the Zone size.

8. A data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 1, characterized in that, Reducing thread contention for Zone locks by asynchronously storing PM includes: when a thread writing to the SSTable file fails to acquire the lock of the target Zone, the loop waiting logic is terminated, and the data to be written is temporarily stored in the dedicated buffer of the persistent memory PM; a trigger condition is set, and a background thread monitors the trigger condition in real time. When the trigger condition is met, the background thread writes the data in the PM buffer in batches to the corresponding Zone.

9. A data storage method based on a partitioned namespace solid-state drive and persistent memory according to claim 8, characterized in that, The triggering conditions include the lock of the corresponding Zone in the PM buffer being released or the amount of data in the PM buffer reaching a preset threshold.

Images