Cache management device for partition namespace solid state disk
By combining LRU and adaptive caching strategies, the cache management of partitioned namespace SSDs is optimized, solving the problems of low garbage collection efficiency and poor read/write performance, and achieving more efficient hard drive operations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-14
- Publication Date
- 2026-04-03
AI Technical Summary
Existing cache replacement algorithms are difficult to manage efficiently on partition-namespace-oriented SSDs, resulting in low garbage collection efficiency and poor read/write performance.
By combining LRU and adaptive strategies, and maintaining the page usage frequency and access time through two pointers, a cache management device for partitioned namespace solid-state drives was designed. This device includes modules for data statistics, page management, replacement strategy, global index, and garbage collection. The cache replacement strategy is optimized to reduce the replacement of pages that have not been accessed for a long time.
It improves the efficiency of hard drive garbage collection, reduces the number of partitions to be reset, the number of pages to be rewritten, and the read/write time, thereby improving read/write performance.
Smart Images
Figure CN121785946A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of database technology, and more specifically to a cache management device for partitioned namespace solid-state drives. Background Technology
[0002] In recent years, many scholars have proposed various cache replacement strategies. These works can be broadly divided into two categories: traditional cache replacement algorithms and cache replacement algorithms for flash memory.
[0003] (1) Traditional cache replacement strategy
[0004] Traditional cache replacement strategies primarily focus on hit rate, with well-known algorithms including LRU, CLOCK, LRU-2, and ARC.
[0005] LRU (Least Recently Used) is a commonly used memory page replacement algorithm. It works by recording the last access time of a page and removing the page that has not been used for the longest time. Its advantage is constant time complexity, but its disadvantages are also significant. First, it only considers the recentity of page access, not the frequency of access, meaning that frequently accessed but recently unaccessed pages may still be removed by LRU. Second, LRU lacks scan resistance; when performing sequential page scans, it may pollute the buffer pool and remove pages with a higher re-reference probability, leading to performance degradation.
[0006] The CLOCK algorithm uses a circular buffer to organize pages in memory and a reference bit to determine which pages to replace when the buffer is full. When a page is referenced, its reference bit is set to 1. The algorithm iterates through the pages, setting the reference bit to 0 when it encounters a page with a reference bit of 1, and not replacing that page. When it encounters the first page with a reference bit of 0, it selects and replaces it. However, the CLOCK algorithm has some limitations: it does not consider the frequency of page references and lacks scan resistance.
[0007] LRFU is a combination of LRU and LFU algorithms. It additionally considers the impact of access frequency on page replacement, but parameters still need to be adjusted to balance the weights of time and frequency. The 2Q algorithm uses two separate queues, one for recently referenced pages and the other for frequently referenced pages, but the size of the two queues and the replacement strategy for each queue still need to be adjusted.
[0008] (2) Flash-based cache replacement algorithm
[0009] Cache replacement algorithms not only have adaptive optimizations for traditional databases, but also have many optimizations for databases that use flash memory as external storage, such as CFLRU, LRU-WSR, AD-LRU, etc.
[0010] CFLRU is an algorithm based on flash memory databases. It divides cached pages into two queues: clean pages (unmodified pages) and dirty pages (modified pages). Following the LRU strategy, it prioritizes replacing data in the clean page queue, replacing data in the dirty page queue only when all data in the clean page queue has been replaced. CFLRU can significantly reduce flash memory write cycles and extend flash memory lifespan, but it may cause dirty page data to reside in the cache for extended periods, affecting the hit rate.
[0011] LRU-WSR combines the LRU and WSR algorithms. Building upon LRU, it maintains a write sequence for dirty page cache blocks and prioritizes replacing pages in the write sequence with a high proportion of free space within the flash block, reducing the number of erase operations for subsequent blocks. While LRU-WSR optimizes flash block utilization and reduces erase overhead, it requires maintaining block-level metadata, increasing memory consumption. Summary of the Invention
[0012] This invention provides a cache management device for partitioned namespace solid-state drives to improve the garbage collection efficiency and read / write performance of the hard drive.
[0013] In a first aspect of the present invention, a cache management device for a partitioned namespace solid-state drive is provided, comprising: a data statistics module, a page management module, a replacement strategy module, a global index module, a garbage collection module, and a hard disk device management module;
[0014] The data statistics module is used to statistically analyze experimental data during the experiment.
[0015] The page management module is used to organize the mapping between the logical addresses and physical addresses of pages;
[0016] The replacement strategy module is used for designing cache replacement strategies;
[0017] The global index module is used to construct in-memory B+ tree indexes to facilitate data retrieval;
[0018] The garbage collection module is used for data management during the hard drive garbage collection process;
[0019] The hard disk device management module is used for the design and partition management of hard disk read / write interfaces.
[0020] In some optional embodiments of the present invention, the experimental data includes: the number of pages read from the hard disk, the time spent reading, the number of pages written, the time spent writing, the number of partitions reset and the number of pages rewritten during garbage collection, the cache hit rate, and the runtime.
[0021] In some optional embodiments of the present invention, the data statistics module is specifically used to record the number of reads and writes each time the hard disk performs a read / write operation, and to record the number of times the hard disk partitions are reset and the number of times pages are rewritten when the hard disk performs garbage collection.
[0022] In some optional embodiments of the present invention, the data statistics module is also used to set a metadata array for each partition of the hard disk to record the valid data pages and invalid data pages in the partition. If the value of the parameter in the array is 0, it means that the page is invalid data, and the value is 1, it means that the page is valid data.
[0023] In some optional embodiments of the present invention, the data statistics module is also used to record a timestamp at the start of the experiment, and then subtract the timestamp at the start of the experiment from the timestamp at the end of the experiment to obtain the cached runtime.
[0024] In some optional embodiments of the present invention, the page management module includes a dedicated read / write interface. When a page is written from the cache to the hard disk, the interface determines whether the write partition has enough space. If so, the page is written to the current partition; otherwise, a free partition is selected for writing.
[0025] In some optional embodiments of the present invention, the replacement strategy module is used to cache replacement algorithms that facilitate disk garbage collection and LRU for comparative experiments.
[0026] In some optional embodiments of the present invention, the global index module is specifically used to divide data nodes into internal nodes and leaf nodes. Internal nodes are stored in memory because they are frequently accessed, while leaf nodes are stored on the hard disk.
[0027] In some optional embodiments of the present invention, the garbage collection module is specifically used to rewrite valid data and reset partitions. When the hard disk space is full, the garbage collection module will prioritize the partition with the highest garbage rate for recycling. For valid data in the partition, it will be read into memory and rewritten to a new partition, while invalid data will be directly cleared.
[0028] In some optional embodiments of the present invention, the hard disk device management module is specifically used to determine whether the partition has enough space when data needs to be written to the partition. If there is enough space, the position of the write pointer in the partition is read, and then the write function is called to write data to the partition. Finally, the partition status is adjusted according to the space usage in the partition. If the partition is full, the partition status is set to full and the partition is added to the garbage collection queue. If the partition is not full, the original status of the partition is saved and it is added to the available queue.
[0029] The beneficial effects provided by this invention are:
[0030] The cache management device for partitioned namespace solid-state drives provided by this invention can improve the garbage collection efficiency and read / write performance of the hard drive. Under different loads, this strategy can reduce the number of partitions reset during the hard drive garbage collection process by more than 10%, reduce the number of pages rewritten during the hard drive garbage collection process by more than 25%, reduce the hard drive read / write time by more than 15%, and reduce the number of reads by 21% and the number of writes by 10%.
[0031] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0032] Figure 1 This is an architecture diagram of a cache management device for a partitioned namespace solid-state drive provided in an embodiment of the present invention;
[0033] Figure 2 A schematic diagram of the structure of a cache management device for a partitioned namespace solid-state drive provided in an embodiment of the present invention;
[0034] Figure 3 The flowchart of the insertion algorithm provided in the embodiment of the present invention;
[0035] Figure 4 A flowchart of the replacement algorithm provided in an embodiment of the present invention;
[0036] Figure 5 This is a schematic diagram of the hard disk garbage collection process provided in an embodiment of the present invention. Detailed Implementation
[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0038] The device in this invention combines LRU and adaptive strategies to achieve efficient cache management. It uses two pointers to maintain the usage frequency of each page in the cache: one records the page's access frequency, and the other records the time elapsed since the page's last access. The AD-LRU algorithm calculates a weight value based on page usage and access time, and then selects pages to be replaced according to the weight values. This allows for the replacement of more pages that have not been accessed for a long time and fewer active pages, thereby improving the performance of the database system.
[0039] The technical solution of this invention mainly includes a data statistics module, a page management module, a replacement strategy module, a global index module, a garbage collection module, and a hard disk device management module. The overall architecture is as follows: Figure 1 As shown.
[0040] The data statistics module is responsible for collecting various data during the experiment, including the number of pages read from the hard drive, the time spent reading, the number of pages written, the time spent writing, the number of partitions reset and pages rewritten during garbage collection, the cache hit rate, and the runtime. To accurately collect this data, this module records the number of reads and writes for each hard drive read / write operation, and the number of hard drive partition resets and page rewrites during garbage collection. To facilitate the counting of rewrites of valid data pages in a partition, this module sets up a metadata array for each partition to record valid and invalid data pages. A value of 0 in the array indicates that the page is invalid, and a value of 1 indicates that the page is valid. At the start of the experiment, this module records a timestamp, and then subtracts the start timestamp from the end timestamp to obtain the cache runtime.
[0041] Since hard drives lack an FTL (File Transfer Table) and users need to manually map data addresses, this solution employs a page management module to organize the mapping between logical and physical page addresses. This module uses a page table data structure to record the logical and physical addresses of pages on the hard drive. When a page is written to memory, a tag is applied. When reading a page, the page table is searched first. If the page is tagged, it is read directly from the cache. If the page is not tagged, its physical address is searched in the page table, then the page is read from the hard drive into the cache, and the tag is applied. If a page is replaced from the cache, the tag is removed. This module organizes the logical addresses of pages using a heap file structure. Each page on the hard drive has a tag indicating its file number and offset within the file. When a new page is created, the module constructs the tag and sets a variable to determine whether the page contains valid or invalid data. Furthermore, this module provides a dedicated read / write interface for pages. When writing a page from the cache to the hard drive, the interface checks if the partition has sufficient space. If so, the page is written to the current partition; otherwise, a free partition is selected.
[0042] The replacement strategy module is responsible for designing the cache replacement strategy, including a replacement algorithm that facilitates disk garbage collection and an LRU (Least Recently Used) algorithm for comparative experiments. For example... Figure 2As shown, this strategy divides the cache queue into a cold queue and a hot queue. The hot queue uses an LRU replacement pattern, while the cold queue prioritizes replacing pages with low update frequency. The length ratio of the hot and cold queues can be manually adjusted. Initially, data is cached in the hot queue. When the hot queue is full, the least recently accessed page is moved to the cold queue. When the cold queue is full, the page with the lowest update frequency is removed. The specific algorithm is as follows: Figure 3 , Figure 4 As shown.
[0043] The global index module constructs an in-memory B+ tree index to facilitate data retrieval. This module divides data nodes into internal nodes and leaf nodes. Internal nodes, which are frequently accessed, are stored in memory, while leaf nodes are stored on disk. At the beginning of the experiment, data is inserted into the index. When data needs to be read or updated, the process starts from the root node and continues until the corresponding leaf node is reached. This module has designed corresponding insertion, update, and search algorithms for the index. When inserting data, it first checks if the index is empty. If it is, a new root node is created, and the data is inserted into the root node. If the index is not empty, the process starts from the root node, finds the corresponding leaf node, and inserts the corresponding data. The update and search algorithms are similar to the insertion algorithms, requiring traversal from the root node to the corresponding leaf node, and then operations are performed on the data within each node.
[0044] The garbage collection module is responsible for data management during the hard drive garbage collection process, including rewriting valid data and resetting partitions. When the hard drive is full, this module will prioritize the partition with the highest garbage rate (the proportion of invalid data in the partition's size) for garbage collection. Figure 5 As shown, valid data in a partition will be read into memory and rewritten to a new partition, while invalid data will be directly cleared.
[0045] The hard disk device management module is responsible for the design of the hard disk read / write interface and partition management. This module calls the open-source operation library libzbd and designs a data structure for each partition to store metadata, including the partition's starting address, write pointer position, status information, garbage rate, and a bitmap of valid and invalid data. When data needs to be written to a partition, it checks if the partition has enough space. If so, it reads the write pointer position in the partition and calls the write function to write the data. Finally, it adjusts the partition's status based on the space usage. If the partition is full, it sets the partition status to full and adds the partition to the garbage collection queue. If the partition is not full, it saves the original status and adds it to the available queue. When data needs to be read from a partition, the read function is called. When a partition needs to be reset, the partition reset function is called, the partition metadata is updated, the garbage rate in the partition is cleared to zero, and the partition is added to the free queue.
[0046] Through experimental comparison, this cache replacement strategy can improve the garbage collection efficiency and read / write performance of hard disks compared to LRU. Under different loads, this strategy can reduce the number of partitions reset during the hard disk garbage collection process by more than 10%, reduce the number of pages rewritten during the hard disk garbage collection process by more than 25%, reduce the hard disk read / write time by more than 15%, and reduce the number of reads by 21% and the number of writes by 10%.
[0047] This index was implemented using a simulated partitioned namespace solid-state drive. The experimental server system was Ubuntu 22.04.2.LTS, Linux version 5.19.0-32-generic, and gcc version 9.4.0. The experiment used the null_blk simulator and libzbd operation library provided by Western Digital. Three different workloads were employed: write-dominant, read-dominant, and read-write balanced. The read-dominant workload consisted of sequences of read and update operations with a read-to-write ratio of 8:2, 100,000 key-value pairs, and 1 million total operations. The cache sizes were set to 1MB, 2MB, and 3MB respectively. The write-dominant workload had a read-to-write ratio of 2:8, and the mixed read-to-write workload had a read-to-write ratio of 5:5. All other conditions for these two workloads were consistent with the read-dominant workload. The simulated device used in the experiment contained 32 partitions, each 32MB in size, with a uniform block size of 4KB within each partition.
[0048] During the experiment, the number of disk read / write operations, the number of partitions reset, the number of pages rewritten, and the running time during garbage collection were compared between the proposed cache replacement strategy and the traditional LRU replacement strategy under different loads.
[0049] It should be understood that in the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this description, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate different embodiments or examples described in this specification, as well as some features of different embodiments or examples.
[0050] Of course, those skilled in the art can make various corresponding changes and modifications based on the present invention without departing from its spirit and essence, but such changes and modifications should all fall within the protection scope of the claims of the present invention.
Claims
1. A cache management device for partitioned namespace solid-state drives, characterized in that, include: The module includes data statistics, page management, replacement strategy, global index, garbage collection, and hard disk device management. The data statistics module is used to statistically analyze experimental data during the experiment. The page management module is used to organize the mapping between the logical addresses and physical addresses of pages; The replacement strategy module is used for designing cache replacement strategies; The global index module is used to construct in-memory B+ tree indexes to facilitate data retrieval; The garbage collection module is used for data management during the hard drive garbage collection process; The hard disk device management module is used for the design and partition management of hard disk read / write interfaces.
2. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The experimental data includes: the number of pages read from the hard drive, the time spent reading, the number of pages written, the time spent writing, the number of partitions reset and the number of pages rewritten during garbage collection, the cache hit rate, and the runtime.
3. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The data statistics module is specifically used to record the number of reads and writes each time the hard drive performs a read or write operation, and to record the number of hard drive partition resets and page rewrites during hard drive garbage collection.
4. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The data statistics module is also used to set up a metadata array for each partition of the hard drive to record the valid and invalid data pages in the partition. If the value of the parameter in the array is 0, it means that the page is invalid data, and the value is 1, which means that the page is valid data.
5. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The data statistics module is also used to record a timestamp at the start of the experiment, and then subtract the start timestamp from the end timestamp to obtain the cached runtime.
6. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The page management module includes dedicated read and write interfaces. When a page is written from the cache to the hard drive, the interface will determine whether the write partition has enough space. If it does, the page will be written to the current partition; otherwise, a free partition will be selected for writing.
7. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The replacement strategy module is used to cache replacement algorithms that facilitate disk garbage collection and LRU for comparative experiments.
8. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The global index module is specifically used to divide data nodes into internal nodes and leaf nodes. Internal nodes are stored in memory because they are frequently accessed, while leaf nodes are stored on the hard disk.
9. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The garbage collection module is specifically used to rewrite valid data and reset partitions. When the hard drive space is full, the garbage collection module will prioritize the partition with the highest garbage rate for recycling. For valid data in the partition, it will be read into memory and rewritten to the new partition, while invalid data will be directly deleted.
10. The cache management device for partitioned namespace solid-state drives according to claim 1, characterized in that, The hard disk device management module is specifically used to determine whether the partition has enough space when data needs to be written to it. If there is enough space, it reads the position of the write pointer in the partition, then calls the write function to write data to the partition. Finally, it adjusts the partition's state according to the space usage in the partition. If the partition is full, it sets the partition state to full and adds the partition to the garbage collection queue. If the partition is not full, it saves the original state of the partition and adds it to the available queue.