Anti-cache indexing method for mixed storage of nonvolatile memory and solid state disk

Abstract

The invention discloses an anti-cache indexing method for mixed storage of a nonvolatile memory and a solid state disk, and belongs to the technical field of database indexing. Comprising the steps that a hierarchical index architecture of hybrid storage is constructed, and the hierarchical index architecture comprises a nonvolatile memory layer and a solid state disk layer; dividing index nodes in the nonvolatile memory layer into internal nodes and leaf nodes; separating the metadata of the leaf node from the load, storing the metadata in the nonvolatile memory layer as an anchor, and unifying the address pointer formats of the nonvolatile memory layer and the solid state disk layer; and judging the cold and hot states of the leaf nodes according to the leaf node access frequency and the dynamic expiration time strategy, and dynamically managing the attribution of the leaf nodes between two layers of storage and the migration between different storage layers in combination with the candidate queues. According to the method, the nonvolatile memory and the solid state disk are combined, the durability and low delay of the nonvolatile memory and the low cost and large capacity of the solid state disk are utilized, the scene of mass data is met, meanwhile, the resource utilization rate is increased, and the cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of database indexing technology, and more specifically to an anti-caching indexing method using a hybrid storage of non-volatile memory and solid-state drive. Background Technology

[0002] In database systems, indexes, as a core technology, have a profound and significant impact on improving system performance and efficiency. The rational design and application of indexes are key to ensuring the efficient operation of storage systems. On the one hand, with the rapid development of informatization and digitalization, the data processing demands faced by databases are constantly increasing, and the rapid expansion of massive amounts of data makes the role of indexes increasingly prominent. On the other hand, the bottlenecks of traditional DRAM technology in capacity expansion and cost control are gradually becoming apparent, making it difficult to keep up with the ever-increasing memory demands.

[0003] The rise of non-volatile memory technologies, such as phase-change memory (PCM) and 3DXPoint, has provided new possibilities for building efficient indexes in NVM. Intel Optane, for example, not only meets the needs of large-capacity storage but also offers advantages such as byte addressing, low latency, and no data loss during power outages, effectively compensating for the high cost and small capacity of DRAM. However, as data volume increases, the space occupied by indexes also increases, still facing the challenge of insufficient NVM resources. When dealing with massive amounts of data, database indexes face the following challenges in efficiently utilizing limited NVM resources: Existing non-volatile memory space management mechanisms are designed for general space management scenarios, thus consuming a large amount of DRAM during runtime. Furthermore, the need to manage non-volatile memory fragmentation prevents the effective reuse of freed space, resulting in significant resource waste. Current methods for identifying hot and cold index data operate at the page level (i.e., multiple nodes reside on the same page, and the node types within the page may differ), leading to inaccurate identification of hot and cold index data. For example, a page with a few hot data nodes may be classified as a cold page, causing frequent swapping of large amounts of data and resulting in substantial I / O overhead.

[0004] Therefore, how to solve the problems in the existing technology is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention proposes an anti-caching indexing method for hybrid storage of non-volatile memory and solid-state drive to overcome or at least partially solve the above problems; To achieve the above objectives, the present invention adopts the following technical solution: This invention provides an anti-caching indexing method for hybrid storage of non-volatile memory and solid-state drive, comprising the following steps: S1. Construct a hybrid storage hierarchical index architecture, which includes a non-volatile memory layer and a solid-state drive layer. The non-volatile memory layer is used to store frequently accessed hot index nodes, and the solid-state drive layer is used to store infrequently accessed cold index nodes. S2. Divide the index nodes in the non-volatile memory layer into internal nodes and leaf nodes, where the leaf nodes include metadata and payload. S3. Separate the metadata of leaf nodes from the payload, store the metadata in the non-volatile memory layer as an anchor, and unify the address pointer format of the non-volatile memory layer and the solid-state drive layer. S4. Based on the anchor points in the non-volatile memory layer, access characteristics are obtained. At the same time, the hot and cold status of leaf nodes is judged according to the access frequency of leaf nodes and the dynamic expiration time strategy. Combined with the candidate queue, the affiliation of leaf nodes between the two storage layers and the migration between different storage layers are dynamically managed.

[0006] Furthermore, the hierarchical index architecture adopts a space management mechanism, dividing the non-volatile memory layer into different types of segments according to the node type of the index. Within each segment, a partition management strategy is adopted. Within each partition, space is allocated using offsets and released using logical deletion. Log information of the corresponding operations is recorded in the metadata of the partition. The solid-state drive (SSD) is divided into several files of the same size, managed using a bitmap stored in a non-volatile memory layer, and status bits are set for SSD layer blocks. The SSD layers are allocated using a producer-consumer model.

[0007] Furthermore, the producer-consumer model includes: Producer thread: Obtains the offset of the free block by accessing the bitmap, sets the block state to the first state, and puts it at the tail of the producer-consumer queue; Consumer thread: The thread requesting space retrieves the offset of the head of the producer-consumer queue and modifies the corresponding block state to the second state after writing the data; When freeing up space, the state of the corresponding block is changed to the third state. The first state indicates that the block has been allocated but not yet used; the second state indicates that the block has been used; and the third state indicates that the block is idle.

[0008] Furthermore, the non-volatile memory layer is segmented, and each segment is allocated a node space of a certain type; The segment offset is used to locate the position of the inode within the segment. The nextOffset variable exists in the partition to mark the next free position in the partition. During logical deletion, the corresponding offset is linked to the free list in the partition. Logs record key steps when allocating and releasing space within a partition.

[0009] Furthermore, when allocating non-volatile memory layer space, the type of index data node to be allocated is determined; First, check if there is any free space in the free list. If not, allocate the unused space, return nextOffset, and move it to the next free position. If allocation is not possible, check if there is any logically freed space in the free list. If it exists, retrieve the offset of the end of the list and encode the segment number and offset into a unified address pointer. If it does not exist, allocate a new non-volatile memory segment. When freeing up space, the corresponding non-volatile memory segment number and offset are obtained by resolving the unified address pointer, and logical release is performed, linking the offset corresponding to the free space to the free list.

[0010] Furthermore, the unified address pointer structure consists of n bits, where the (n-1)th bit is used to distinguish between internal nodes and leaf nodes, the (n-2)th bit is used to distinguish the storage level to which the node belongs, the (n-4)th to (n-16)th bits represent the non-volatile memory segment number or solid-state drive file number, and the (n-17)th to 0th bits represent the offset.

[0011] Furthermore, based on the access feature data stored in the leaf node metadata, the leaf nodes are identified as hot or cold nodes. The access characteristic data stored in the leaf node metadata includes the most recent access time, the number of accesses, and address pointer information; The most recent access time is the time of the most recent access to the leaf payload node, the number of accesses is the number of times the leaf payload node was accessed in NVM, and the average access frequency of the leaf payload node is the ratio of the total number of accesses to the number of leaf payload nodes in the non-volatile memory layer.

[0012] Furthermore, a fine-grained cache management algorithm is adopted; Set up a candidate queue to store nodes that are to be migrated to the solid-state drive layer; When the set threshold for non-volatile memory layer space usage is reached, a traversal thread is activated to scan all leaf metadata parts, and leaf payload nodes that match cold data are added to the candidate queue as candidate nodes. If a leaf payload node that is being accessed is encountered, it is skipped.

[0013] Furthermore, while starting the traversal thread, a migration thread is activated to process the nodes in the candidate node queue. Candidate nodes are taken out one by one from the head of the queue, and their hot and cold status is judged again. If they still meet the cold data condition, they are migrated to the solid-state drive layer, and the corresponding non-volatile memory layer space is released, and the state of the corresponding block in the solid-state drive layer space is changed. When a leaf payload node in the solid-state drive layer is accessed, it is migrated to the non-volatile memory layer, and the corresponding metadata and the state of its corresponding solid-state drive layer block are modified.

[0014] As can be seen from the above technical solution, compared with the prior art, the present invention discloses an anti-caching indexing method for hybrid storage of non-volatile memory and solid-state drive, which has the following beneficial effects: 1. This invention combines non-volatile memory and solid-state drives (SSDs), utilizing the persistence and low latency of non-volatile memory and the low cost and large capacity of SSDs to expand the index capacity of non-volatile memory, making it suitable for scenarios with massive amounts of data, while improving resource utilization and reducing costs.

[0015] 2. This invention proposes a fine-grained space management mechanism for non-volatile memory. Non-volatile memory is divided into different types of segments according to the index node type. A partition management strategy is adopted within the segment. Space is allocated using offsets and released using logical deletion within the partition, so as to efficiently allocate and release non-volatile memory. Log information of the corresponding operations is recorded in the metadata of the partition to prevent space leakage after failure.

[0016] 3. This invention separates the metadata of the index leaf node from the payload, stores the metadata in non-volatile memory as an anchor, and unifies the address pointer format of non-volatile memory and solid-state drive, realizing fast node location and transparent access across storage layers, improving index query efficiency and simplifying system implementation complexity.

[0017] 4. This invention proposes a fine-grained cache management algorithm. It determines the hot / cold status of leaf nodes based on the access frequency of index leaf nodes and the dynamic expiration time strategy. It also combines a candidate queue to dynamically manage the affiliation of leaf payload nodes between two storage layers and the migration between different storage layers. This cache management algorithm can more accurately determine the hot / cold status of data payload nodes and determine their affiliation to the storage layer, reducing the amount of data swapped in and out, thereby reducing computational overhead. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a flowchart of the decaching indexing method provided in the embodiments of the present invention; Figure 2 This is an overall architecture diagram of the non-volatile memory index provided in this embodiment of the invention; Figure 3 This is a flowchart illustrating the allocation and release of non-volatile memory space provided in this embodiment of the invention. Figure 4 This is a flowchart illustrating the allocation and release of solid-state drive space provided in an embodiment of the present invention. Figure 5 This is a diagram of the unified address pointer structure provided in the embodiments of the present invention; Figure 6 This is a flowchart of the cache replacement process provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention discloses an anti-caching indexing method for hybrid storage of non-volatile memory and solid-state drive; such as... Figure 1 As shown, it includes the following steps: S1. Construct a hybrid storage hierarchical index architecture, which includes a non-volatile memory layer and a solid-state drive layer. The non-volatile memory layer is used to store frequently accessed hot index nodes, and the solid-state drive layer is used to store infrequently accessed cold index nodes. S2. Divide the index nodes in the non-volatile memory layer into internal nodes and leaf nodes, wherein the leaf nodes include metadata and payload. S3. Separate the metadata of leaf nodes from the payload, store the metadata in the non-volatile memory layer as an anchor, and unify the address pointer format of the non-volatile memory layer and the solid-state drive layer. S4. Based on the anchor points in the non-volatile memory layer, access characteristics are obtained. At the same time, the hot and cold status of leaf nodes is judged according to the access frequency of leaf nodes and the dynamic expiration time strategy. Combined with the candidate queue, the affiliation of leaf nodes between the two storage layers and the migration between different storage layers are dynamically managed.

[0022] This invention combines non-volatile memory and solid-state drives (SSDs), leveraging the persistence and low latency of non-volatile memory and the low cost and large capacity of SSDs to expand the index capacity of non-volatile memory, making it suitable for scenarios with massive amounts of data, while improving resource utilization and reducing costs.

[0023] The above methods will now be described in detail. like Figure 2As shown, during index construction, nodes are allocated and data is stored in the NVM layer. At the same time, the nodes are organized into a structured form with high query performance to achieve fast data retrieval. As the amount of index data continues to grow, when the storage capacity of NVM approaches saturation, a data migration mechanism is activated: cold leaf nodes with low access frequency in NVM are migrated to the solid-state drive layer. At the same time, in order to maintain the overall access performance of the index, only leaf payload nodes are migrated to SSD, while all internal nodes remain in NVM. In SSD, storage space is allocated for the migrated cold data nodes. The node location is efficiently managed through metadata information such as bitmaps stored in NVM. A unified address pointer is used to ensure that the index can quickly determine the storage layer and physical location of the node, thereby supporting fast and transparent access to cross-layer data. Cold-load nodes that have been migrated to SSDs may still be accessed again. When this happens, the system triggers a reverse migration mechanism to migrate the corresponding nodes back from SSDs to the NVM layer to ensure subsequent access performance.

[0024] The non-volatile memory space management mechanism adopts a segmented management strategy. Within each segment, a partitioning strategy is used to improve concurrency. NVM allocation and reclamation are performed using offsets combined with logical deletion within the partitions. To prevent space leakage after a failure, logs are logged before allocation and release operations, and only nodes of the same type are allowed to be allocated within the same segment. Compared with existing third-party general-purpose non-volatile memory allocation libraries, this method effectively reduces the overhead of memory management metadata and overcomes the inherent limitations of general-purpose memory allocators in effectively reusing and releasing memory space due to adapting to multiple scenarios, significantly improving the utilization efficiency and space reusability of NVM.

[0025] In terms of implementation, the system records the starting virtual address allocated to each segment. By combining this starting address with the offset within the segment, the physical location of the data node can be accurately located. To further improve concurrency performance and avoid a single segment becoming a bottleneck for thread contention during allocation and release, a segment partitioning strategy is adopted. A segment is divided into multiple partitions, each maintaining independent metadata information to manage its internal space state, thereby enabling multi-threaded parallel allocation and release operations. Partition metadata information is uniformly stored in the segment header. The metadata content includes a partition lock for concurrency control, the offset of the next free position within the partition, a partition free list for storing released space in the partition, and a log of staged operation types for recovery. The global number of allocation nodes is used to quickly determine whether the space of the segment is exhausted, and the free list is operated on by recording the tail index.

[0026] To ensure that the allocator does not deadlock due to incorrect lock states before the failure during fault recovery, the metadata of each partition is traversed during restart, and the lock state is reset to unlocked. The log is divided into two main categories: allocation and release. Before allocation and release, the offset and the number of free nodes in the free list are recorded. When critical steps are executed, such as after modifying the free list, a completion flag is recorded. This is used to restore the space to its original correct state during recovery log traversal, avoiding NVM leaks. A separate set of free segments is maintained for each segment type.

[0027] After completing the creation or release of space within a segment, the system will add the segment number to the corresponding type of free segment set, effectively avoiding the problem that the system will always create a new segment unconditionally when the current segment space is insufficient.

[0028] like Figure 3 As shown, the specific process of NVM space allocation is as follows: After receiving a memory allocation request, the allocator first determines the starting partition to try: it adds the partition number of the last accessed partition to a maximum prime number less than the total number of partitions, and then takes the modulo of the total number of partitions to obtain the first partition number to try. Apply a mutex lock to the selected partition to implement concurrent access control; Record operation logs, and update the logs in subsequent execution steps; First, check the length of the free list of the partition to determine if there are any allocable nodes. If there are, return the offset of the corresponding node from the list, decrement the index pointing to the end of the list by 1, and record the corresponding operation log. If there are no available nodes in the free list of the current partition, query the next free offset recorded in the partition and determine whether the offset does not exceed the maximum range of the current partition: if it does not exceed the limit, return the offset and update the free offset pointer in the partition to the starting position of the next node, and record the relevant log; otherwise, the allocation fails. Release the lock on the current partition. If the allocation is successful, mark it as completed in the log and return the allocated offset; if the allocation fails, determine whether there is no available space in the current segment by checking the number of global allocation nodes for the current segment: if there is insufficient space, the allocator will allocate a new segment and return to step 1 to retry the allocation until the allocation is successful, and then update the number of global allocation nodes.

[0029] The specific process for releasing NVM space is as follows: Release space request, and parse out the partition corresponding to the space that needs to be released; Apply a mutex lock to the partition to implement concurrent access control; Record operation logs, and update the logs in subsequent execution steps; Record the tail index of the free list, write the release offset at the tail of the free list, increment the tail index by 1, update the global allocation node count of the current segment, and add the current segment number to the corresponding free segment set. Release the mutex lock on the current partition.

[0030] SSD space management is built on top of a traditional file system, and to address its shortcomings in fine-grained space management, a bitmap mechanism is introduced to manage the state of storage blocks within each SSD file in a granular manner. Specifically, a two-bit encoded bitmap identifies three states for each block: "00" indicates free, "01" indicates allocated but not yet used, and "11" indicates used. The bitmap is the core metadata structure in SSD space allocation.

[0031] To improve access and update efficiency, bitmaps are partitioned to support concurrent operations, and the state of blocks within each partition is quickly determined and modified through efficient bitwise operations. Each partition is equipped with a partition lock, which, in addition to providing concurrency control, also integrates generational information, i.e., the number of times the current indexing system has been started. When locking, the generational information within the lock is first checked to see if it matches the current index generation; if they do not match, it is determined to be the first access after a restart, and the corresponding recovery logic must be executed.

[0032] When the index restarts, the error correction and recovery of the bitmap state are completed during the index log replay process. Therefore, during this recovery process, it is only necessary to reset all blocks in the current partition with a state of "01" to "00" and synchronously update the generation information in the lock. To optimize allocation performance, a producer-consumer pattern is adopted, that is, a separate dedicated allocation producer thread is set up to scan and pre-allocate free SSD blocks, construct them into address pointers of a uniform format, and insert them into the tail of the free block queue.

[0033] The maximum length of the queue is configurable, and the status of blocks inserted into the queue is marked as "01". When a consumer thread that needs to allocate an SSD block arrives, it can directly obtain the pre-constructed address pointer from the head of the queue and modify the block's status to "11", which greatly reduces the real-time allocation latency.

[0034] In addition, similar to non-volatile memory management mechanisms, the SSD layer also maintains a set of free files. Since the nodes migrated to SSDs contain only a single type, only a unified set of free files needs to be maintained for efficient management of SSD file reuse and allocation.

[0035] The specific process of SSD space allocation and release is as follows: Figure 4 As shown: The specific process for allocating SSD space is as follows, and this process loops continuously: Check if the queue is full. If not, determine the first partition to try: add the partition number of the last accessed partition to the largest prime number less than the total number of partitions, and then take the modulo of the total number of partitions to get the first partition number to try. Apply a mutex lock to the selected partition to implement concurrent access control; Find the free blocks in the current partition. If they exist, add them to the tail of the queue; otherwise, return to step 1. Change the status of the corresponding block to "01"; Unlock the corresponding partition.

[0036] The specific process for freeing up SSD space is as follows: The partition number and block number are obtained by resolving the unified address pointer, and the corresponding partition is locked. Change the status of the corresponding block to "00"; Unlock the corresponding partition.

[0037] A standardized address pointer structure, such as Figure 5 As shown: First, the metadata of the index leaf nodes is separated from the payload. The metadata is stored in NVM as an anchor. Based on this, a unified address pointer is proposed. This structure is 64 bits long, where bit 63 is used to distinguish between internal nodes and leaf nodes, bit 62 is used to distinguish the storage level to which the node belongs ("0" indicates non-volatile memory level, "1" indicates solid-state drive level), bits 60 to 48 represent the non-volatile memory segment number or solid-state drive file number, and bits 47 to 0 represent the offset. First, the storage level is determined, and then the segment or file to which the node belongs is determined. SSD files are allocated with the same type of node, so the block number can be calculated from the offset.

[0038] The fine-grained cache management strategy combines the access frequency of leaf nodes and dynamic expiration time to determine whether a payload node should belong to the NVM layer or the SSD layer. It also uses a candidate strategy to put nodes that are initially identified as cold into a candidate queue. Only when a node is identified as cold again in the candidate queue is it considered a true cold node. At the same time, it manages the migration of cold and hot data between the two layers.

[0039] In the specific implementation, each leaf metadata node records its most recent access timestamp and access count, which serves as the basis for determining its hot / cold status. Leaf nodes have a dynamically set expiration time, which is updated every minute. Its value is estimated based on the rate of access to leaf nodes in the current minute to obtain an estimate of the time required to access all leaf nodes, and the expiration time is limited to a minimum of 360 seconds and a maximum of 720 seconds.

[0040] A monitoring thread is configured to acquire key information such as NVM usage, total access count of leaf nodes, and access rate in real time. Simultaneously, a traversal thread is configured to scan the leaf nodes in the NVM, recording the start timestamp at the beginning of the traversal. During the traversal, the system determines the "hot" or "cold" status of each node: if a node has expired or its access count is lower than the current average access count, the node is added to a candidate queue.

[0041] The migration thread continuously monitors the candidate queue and removes nodes for secondary screening. Only when a node simultaneously meets the conditions of being "expired" and having "access counts lower than the average access count" is its load node migrated to the SSD layer. To improve the accuracy of hot / cold node determination and reduce jitter caused by frequent migrations, a two-stage screening mechanism is employed.

[0042] In the expiration time determination, the time difference between the start times of two traversals is considered as a complete time period. If the most recently accessed timestamp of a leaf node plus its expiration time is earlier than 80% of the time points in this time period, then the node is determined to be an expired node. Specifically, in the first traversal, the time difference is the interval from the initial time 0 to the current time.

[0043] The specific processes of the traversal thread and migration thread in cache replacement are as follows: Figure 6 As shown: When the monitoring thread detects that the non-volatile memory usage has reached a preset threshold, it wakes up the traversal thread to begin traversing the leaf nodes stored in NVM. The traversal thread attempts to lock the leaf node. If locking fails, it indicates that the node is being accessed, and it is classified as a hot data node and its processing is skipped. If locking is successful, it further checks whether the node meets one of the following conditions: the node has expired, or its access frequency is lower than the current system's average access frequency. Meeting either condition classifies it as a cold data node. All identified cold data nodes are added to the candidate cold data node queue for further processing.

[0044] The specific process of the migration-disk write thread is as follows: A node is retrieved from the head of the candidate cold data node queue and subjected to a secondary evaluation. Only when a node simultaneously meets both the conditions of "expired" and "access frequency lower than average access frequency" is it finally confirmed as a cold data node. To allocate SSD storage space for the confirmed cold data node, retrieve a pre-formatted SSD block with an address pointer ready from the head of the pre-allocation queue of the SSD space manager. Migrate the data from the cold data nodes to the allocated SSD space and change the status of the corresponding SSD blocks to "11". Simultaneously, record detailed migration logs, including migration stage identifiers and SSD space address pointers, to ensure system recovery in the event of a failure. After the migration is complete, release the NVM space originally occupied by the cold data node.

[0045] The specific process of migration-callback thread is as follows: When accessing a data node, first determine whether the target leaf node has been migrated to the SSD layer; If the node has indeed been migrated to SSD, a callback process is triggered to reallocate space for the node in the NVM layer; Migrate node data from the SSD space back to the newly allocated NVM space; Once the data callback is complete, immediately release the SSD space previously occupied by the node, that is, change the status of the corresponding SSD block to "00".

[0046] This invention proposes a fine-grained space management mechanism for non-volatile memory. Non-volatile memory is divided into different types of segments according to the node type of the index. A partitioning management strategy is adopted within each segment, using offset allocation and logical deletion to release space within each partition. This ensures efficient allocation and release of non-volatile memory. Log information of corresponding operations is recorded in the metadata of each partition to prevent space leakage after a failure. Simultaneously, the metadata of the index leaf nodes is separated from the payload, with the metadata stored in the NVM as an anchor. The NVM and SSD address pointer formats are unified, enabling fast node location and transparent access across storage layers, improving index query efficiency and simplifying system implementation complexity.

[0047] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0048] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A decaching indexing method using hybrid storage of non-volatile memory and solid-state drive, characterized in that, Includes the following steps: S1. Construct a hybrid storage hierarchical index architecture, which includes a non-volatile memory layer and a solid-state drive layer. The non-volatile memory layer is used to store frequently accessed hot index nodes, and the solid-state drive layer is used to store infrequently accessed cold index nodes. S2. Divide the index nodes in the non-volatile memory layer into internal nodes and leaf nodes, wherein the leaf nodes include metadata and payload. S3. Separate the metadata of leaf nodes from the payload, store the metadata in the non-volatile memory layer as an anchor, and unify the address pointer format of the non-volatile memory layer and the solid-state drive layer. S4. Based on the anchor points in the non-volatile memory layer, access characteristics are obtained. At the same time, the hot and cold status of leaf nodes is judged according to the access frequency of leaf nodes and the dynamic expiration time strategy. Combined with the candidate queue, the affiliation of leaf nodes between the two storage layers and the migration between different storage layers are dynamically managed.

2. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 1, characterized in that, The hierarchical index architecture adopts a space management mechanism, which divides the non-volatile memory layer into different types of segments according to the node type of the index. Within each segment, a partition management strategy is adopted. Within each partition, space is allocated using offsets and released using logical deletion. Log information of the corresponding operations is recorded in the metadata of the partition. The solid-state drive (SSD) is divided into several files of the same size, managed using a bitmap stored in a non-volatile memory layer, and status bits are set for SSD layer blocks. The SSD layers are allocated using a producer-consumer model.

3. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 2, characterized in that, The producer-consumer model includes: Producer thread: Obtains the offset of the free block by accessing the bitmap, sets the block state to the first state, and puts it at the tail of the producer-consumer queue; Consumer thread: The thread requesting space retrieves the offset of the head of the producer-consumer queue and modifies the corresponding block state to the second state after writing the data; When freeing up space, the state of the corresponding block is changed to the third state. The first state indicates that the block has been allocated but not yet used; the second state indicates that the block has been used; and the third state indicates that the block is idle.

4. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 3, characterized in that, The non-volatile memory layer is segmented, including: Each segment is allocated one type of node space; The segment offset is used to locate the position of the inode within the segment. The nextOffset variable exists in the partition to mark the next free position in the partition. During logical deletion, the corresponding offset is linked to the free list in the partition. Logs record key steps when allocating and releasing space within a partition.

5. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 4, characterized in that, Allocating non-volatile memory layer space includes: determining the type of index data nodes to be allocated; First, check if there is any free space in the free list. If not, allocate the unused space, return nextOffset, and move it to the next free position. If allocation is not possible, check if there is any logically freed space in the free list. If it exists, retrieve the offset of the end of the list and encode the segment number and offset into a unified address pointer. If it does not exist, allocate a new non-volatile memory segment. When freeing up space, the corresponding non-volatile memory segment number and offset are obtained by resolving the unified address pointer, and logical release is performed, linking the offset corresponding to the free space to the free list.

6. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 5, characterized in that, The unified address pointer includes: The unified address pointer structure consists of n bits, where the (n-1)th bit is used to distinguish between internal nodes and leaf nodes, the (n-2)th bit is used to distinguish the storage level to which the node belongs, the (n-4)th to (n-16)th bits represent the non-volatile memory segment number or solid-state drive file number, and the (n-17)th to 0th bits represent the offset.

7. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 6, characterized in that, The determination of the hot and cold status of leaf nodes includes: identifying and determining the hot and cold status of data leaf nodes based on the access characteristics stored in the leaf node metadata; The access characteristic data stored in the leaf node metadata includes the most recent access time, the number of accesses, and address pointer information; The most recent access time is the time of the most recent access to the leaf payload node, the number of accesses is the number of times the leaf payload node was accessed in NVM, and the average access frequency of the leaf payload node is the ratio of the total number of accesses to the number of leaf payload nodes in the non-volatile memory layer.

8. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 7, characterized in that, The hierarchical index architecture employs a fine-grained cache management algorithm; Set up a candidate queue to store nodes that are to be migrated to the solid-state drive layer; When the set threshold for non-volatile memory layer space usage is reached, a traversal thread is activated to scan all leaf metadata parts, and leaf payload nodes that match cold data are added to the candidate queue as candidate nodes. If a leaf payload node that is being accessed is encountered, it is skipped.

9. The decaching indexing method for hybrid storage of non-volatile memory and solid-state drive as described in claim 8, characterized in that, While starting the traversal thread, a migration thread is started to process the nodes in the candidate node queue. Candidate nodes are taken out one by one from the head of the queue, and their hot and cold status is judged again. If they still meet the cold data condition, they are migrated to the solid-state drive layer, and the corresponding non-volatile memory layer space is released, and the state of the corresponding block in the solid-state drive layer space is changed. When a leaf payload node in the solid-state drive layer is accessed, it is migrated to the non-volatile memory layer, and the corresponding metadata and the state of its corresponding solid-state drive layer block are modified.

Images