A b+ tree index construction method based on zns ssd for dynamic placement of hot and cold data

Abstract

The application relates to a B+ tree index construction method based on ZNS SSD cold and hot data dynamic placement, which comprises a persistent memory PM, a dynamic random access memory DRAM and a ZNS SSD hard disk; the persistent memory PM and the dynamic random access memory DRAM are directly connected with a CPU memory bus and have the ability of being accessed by the CPU by bytes; internal nodes of the B+ tree are stored in the dynamic random access memory DRAM, wherein the internal nodes only store index information for positioning leaf nodes; pseudo leaf nodes of the B+ tree are stored in the persistent memory PM, wherein the pseudo leaf nodes only store key index information of the leaf nodes; leaf nodes for actually storing key value pairs in the B+ tree are divided into hot leaf nodes and cold leaf nodes according to data access frequencies, wherein the hot leaf nodes are stored in the persistent memory PM, and the cold leaf nodes are stored in the ZNS SSD hard disk. The application utilizes the DRAM to quickly position the leaf nodes, reduces query time, stores the hot leaf nodes in the PM to meet real-time business read-write requirements, stores the hot data in the high-speed DRAM and the PM, reduces random read-write delay and performs well in big data analysis and other scenes.

Description

Technical Field

[0001] This invention belongs to the field of data storage, and in particular relates to a B+ tree index construction method for dynamic placement of cold and hot data based on ZNS SSD, which is suitable for building high-performance, low-power storage systems. Background Technology

[0002] With the rapid development of information technology, data volume is growing exponentially, and various applications are placing higher demands on the performance, cost, and management efficiency of storage systems. Traditional storage architectures are gradually showing their inadequacy in meeting these challenges.

[0003] In terms of storage performance, traditional hard drives (HDDs) are relatively slow in read and write speeds due to their mechanical structure limitations, especially during random read and write operations. High seek times and latency severely impact system response speed. For example, in database systems, frequent random read and write operations are common scenarios, and the performance bottleneck of traditional HDDs leads to inefficient data querying and writing, failing to meet the demands of real-time business. While solid-state drives (SSDs) have significantly improved storage performance, with the continuous increase in data volume and the ever-increasing demands for read and write speeds from applications, a single SSD storage device is insufficient to fully meet high-performance requirements. Furthermore, in large-scale data storage environments, the overall read and write performance of the system depends not only on the storage device itself but also on the organization and distribution of data within the storage system. For the widely used B+ tree index structure, if all nodes are stored on traditional storage devices, the significant differences in access frequency among different nodes during data retrieval and updates can prevent overall performance from reaching its optimal level.

[0004] Storage resource utilization is also a key issue. Different types of storage media vary significantly in cost and performance. Dynamic Random Access Memory (DRAM) and Persistent Memory (PM) offer excellent read / write performance and can quickly respond to CPU access requests, but they are relatively expensive, and capacity expansion is also costly. While ZNS SSDs (Partition Namespace SSDs) are cheaper and have larger storage capacities, their performance lags behind DRAM and PM. In practical applications, storing all data in high-performance DRAM or PM without considering data characteristics will significantly increase storage costs; conversely, storing all data on low-performance media like ZNS SSDs will fail to meet the performance requirements for fast access to hot data, leading to wasted storage resources and a decline in overall system performance. Therefore, how to achieve efficient utilization of storage resources while ensuring system performance, and balance storage costs and performance, has become an urgent problem to be solved.

[0005] From a data management perspective, data access patterns are dynamic in practical applications. In many business scenarios, some data is frequently accessed over a period of time, while other data is accessed less frequently. For example, in e-commerce platform order data, recent order data is frequently queried and processed, and is considered "hot data"; while older order data is accessed less frequently, and is considered "cold data." Traditional storage management methods often lack effective mechanisms to cope with dynamic changes in data access frequency and cannot automatically adjust storage strategies based on changes in data popularity. This results in a situation where it is difficult to meet the rapid access needs of hot data while also failing to make reasonable use of storage resources to store cold data. Furthermore, for widely used data structures like B+ tree indexes, traditional index maintenance methods are inefficient when data volume changes dynamically and data access patterns constantly evolve. During index update, insertion, and deletion operations, imbalances in the index structure can easily occur, thus affecting data access performance.

[0006] In summary, existing storage technologies have many problems in terms of performance, storage resource utilization, and data management. There is an urgent need for an innovative storage architecture and data management method to solve these problems and meet the ever-increasing demand for data storage and processing. Summary of the Invention

[0007] To address the problems existing in the background technology, achieve optimized data storage and efficient access, and solve the problems of low read performance, high write overhead, and poor adaptability in the prior art, this invention provides a B+ tree index construction method based on ZNS SSD for dynamic placement of cold and hot data, including: persistent memory PM, dynamic random access memory DRAM, and ZNS SSD hard disk;

[0008] Both persistent memory (PM) and dynamic random access memory (DRAM) are directly connected to the CPU memory bus and have the ability to be accessed by the CPU byte by byte.

[0009] The internal nodes of a B+ tree are stored in dynamic random access memory (DRAM). The internal nodes only store index information used to locate leaf nodes.

[0010] The pseudo-leaf nodes of the B+ tree are stored in persistent memory (PM). The pseudo-leaf nodes only store the key index information of the leaf nodes.

[0011] In a B+ tree, the leaf nodes that actually store key-value pairs are divided into hot leaf nodes and cold leaf nodes based on the frequency of data access. Hot leaf nodes are stored in persistent memory (PM), while cold leaf nodes are stored on ZNS SSD hard disks.

[0012] The present invention has at least the following beneficial effects

[0013] This invention stores the internal nodes of the B+ tree in Dynamic Random Access Memory (DRAM). Due to its high read / write speed, leaf nodes can be quickly located, significantly reducing the time overhead during data queries. Hot leaf nodes are stored in persistent memory (PM), enabling frequently accessed data to be read and written quickly, effectively meeting the stringent requirements of real-time applications for data read / write speeds, such as the rapid response to frequent queries and updates in database systems. Traditional storage devices suffer from high latency due to mechanical limitations during random read / write operations. This solution, by storing hot data in high-speed DRAM and PM, greatly reduces latency during random read / write operations, effectively improving random read / write performance. This allows the system to operate efficiently even when faced with a large number of random read / write operations, demonstrating excellent performance in big data analysis scenarios involving frequent random read / write operations. Attached Figure Description

[0014] Figure 1 This is a diagram of the overall architecture of DPZB+tree in this invention;

[0015] Figure 2 This is a schematic diagram illustrating the identification of hot leaf nodes in this invention;

[0016] Figure 3 This is a schematic diagram illustrating the replacement of the hot leaf node in this invention;

[0017] Figure 4 This is a schematic diagram of node splitting in this invention. Detailed Implementation

[0018] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0019] Please see Figure 1 This invention provides a B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data, including: persistent memory (PM), dynamic random access memory (DRAM), and ZNS SSD hard disk;

[0020] Both persistent memory (PM) and dynamic random access memory (DRAM) are directly connected to the CPU memory bus and have the ability to be accessed by the CPU byte by byte.

[0021] The internal nodes of a B+ tree are stored in dynamic random access memory (DRAM). The internal nodes only store index information used to locate leaf nodes.

[0022] The pseudo-leaf nodes of the B+ tree are stored in persistent memory (PM). The pseudo-leaf nodes only store the key index information of the leaf nodes.

[0023] In a B+ tree, the leaf nodes that actually store key-value pairs are divided into hot leaf nodes and cold leaf nodes based on the frequency of data access. Hot leaf nodes are stored in persistent memory (PM), while cold leaf nodes are stored on ZNS SSD hard disks.

[0024] In this embodiment, we propose an overall system architecture for a B+ tree index with dynamic placement of hot and cold data based on a DRAM-PM-ZNS SSD hybrid storage architecture, as follows: Figure 1 As shown, in our design, the DPZB+ tree is divided into three layers: internal nodes stored in DRAM, pseudo-leaf nodes (leaf node index information) stored in PM, hot leaf nodes stored in PM, and cold leaf nodes stored in ZNS SSDs. In this storage architecture, persistent memory and DRAM are directly connected to the CPU memory bus, and both can be accessed by the CPU byte-by-byte. In the computer system, ZNS SSDs are generally connected to the storage controller (such as the storage controller integrated in the southbridge chip on the motherboard), and the storage controller then communicates with the CPU. Small-capacity persistent memory is mainly used to cache critical metadata and hot-access data of the storage system. This can effectively utilize the byte-by-byte addressing characteristic of persistent memory to perform differential modification of metadata, reduce write amplification, and at the same time, utilize its in-place update characteristic to block the chain recursive updates between nodes of different layers of the B+ tree. ZNS SSDs serve as the main medium to store most of the infrequently accessed data.

[0025] The three-layer structure of DPZB+tree

[0026] Internal Node Layer: Internal nodes are stored in DRAM. DRAM's fast read and write speeds make leaf node searches extremely efficient. Simultaneously, DRAM's in-place update capability prevents cascading updates, significantly reducing write amplification. Internal nodes only store index information used to locate leaf nodes, playing a crucial guiding role in the data search process. For example, in a B+ tree with a large amount of data, internal nodes can quickly narrow the search range to specific leaf node branches, greatly improving search efficiency.

[0027] Pseudo-leaf node layer: Only the key index information of leaf nodes is stored in PM, i.e., pseudo-leaf nodes. The advantage of this design is that when the system loses power, the entire tree can be reconstructed based on the relevant information of the head and leaf nodes, and the usage of PM can be effectively controlled.

[0028] Cold and hot data storage layer: The leaf nodes that actually store key-value pairs are stored in PM and ZNSSSD respectively based on their hot / cold attributes. Hot nodes are stored in PM to prevent write amplification and garbage collection issues caused by frequent small-granular updates on ZNS SSD, ensuring efficient read and write of hot data. Cold nodes are stored on ZNS SSD because they have lower requirements for write operation frequency, fully leveraging the advantages of ZNS SSD's large-capacity storage while reducing storage costs.

[0029] During data access, there is close data interaction among the three types of storage hardware. When the CPU needs to access data, it first searches for relevant internal node index information in DRAM to determine the range of leaf nodes where the data might reside. Then, based on the index information of the pseudo-leaf nodes stored in the PM, it further determines whether the data is stored in a hot node in the PM or a cold node in the ZNS SSD. If it is hot node data, it is read directly from the PM; if it is cold node data, it is read from the ZNS SSD through the storage controller.

[0030] During data writing, hot data is written to the PM and cold data is written to the ZNS SSD based on the data's hot / cold attribute. Simultaneously, during data operations, such as B+ tree structure adjustments (splitting, merging, etc.), DRAM, PM, and ZNS SSD need to work together to ensure data consistency and integrity. For example, when a leaf node splits, it may be necessary to update the index information of the internal node in DRAM, update the relevant information of the pseudo-leaf node in PM, and store the new data in the corresponding PM or ZNS SSD. These three hardware components in the DRAM-PM-ZNS SSD hybrid storage architecture achieve efficient data storage and access through different connection methods with the CPU, clear division of storage functions, and close data interaction and collaboration, meeting the comprehensive requirements of modern computer systems for storage performance, capacity, and cost.

[0031] Preferably, the key index information of the leaf node includes: minimum node value, hot node flag, parent node pointer, number of keys, left sibling node pointer, right sibling node pointer, pointer to persistent memory, pointer to ZNSSSD, and access timestamp; wherein, the minimum node value is used to reconstruct the B+ tree structure; the hot node flag is used to indicate whether the leaf node is a hot leaf node. If it is marked as a hot leaf node, the pointer to persistent memory (PM) is used to query data from persistent memory (PM) during access, otherwise the pointer to ZNSSSD is used to query data from ZNSSSD; the parent node pointer is used to record the position information of the leaf node's parent node in the B+ tree, which allows for convenient tracing and adjustment of the B+ tree structure upwards; the number of keys is used to record the number of keys contained in the leaf node; the left and right sibling node pointers are used to connect adjacent leaf nodes at the same level, forming an ordered linked list structure among the leaf nodes; and the access timestamp is used to record the time of the most recent read / write operation of the leaf node.

[0032] In this embodiment, the pseudo-leaf nodes are stored in PM (persistent memory), mainly containing the key index information of the leaf nodes, specifically including the following parts:

[0033] Minimum node value: This value is used to reconstruct the tree structure during a normal system restart, avoiding a full system scan and improving system recovery efficiency. The minimum node value allows for quick determination of the approximate position and order of nodes within the tree.

[0034] Hot Node Flag: This flag indicates whether the corresponding leaf node is a hot node. When the system accesses a leaf node, it uses this flag to determine whether to use a pointer to the PM (if it's a hot node) or a pointer to the ZNS SSD (if it's a cold node), thus enabling quick determination and accurate access to the location of hot and cold data storage.

[0035] Parent node pointer: Records the position of the parent node of the pseudo-leaf node, which helps maintain the hierarchical structure of the B+ tree and facilitates tracing upwards and adjusting the tree structure when performing data insertion, deletion and other operations.

[0036] Key count: Records the number of keys (keys in key-value pairs) contained in the leaf node, providing important information for data management and operations, such as when determining whether a node is full or needs to be split or merged.

[0037] Left and right sibling pointers: These connect adjacent leaf nodes at the same level, forming an ordered linked list structure among the leaf nodes. This is crucial for operations such as range queries, allowing for rapid traversal of adjacent leaf nodes to retrieve data that meets the given conditions.

[0038] A pointer to persistent memory: When a leaf node is a hot node, this pointer points to the actual key-value pair data stored in PM for fast reading and writing of hot data.

[0039] Pointer to ZNS SSD: When the leaf node is a cold node, this pointer points to the actual key-value pair data stored in the ZNS SSD, used to access cold data stored on the large-capacity ZNS SSD.

[0040] Access timestamp: Records the time of the most recent write operation performed on this node. When PM space is insufficient, the system will replace the hottest node (i.e., the least active) based on the access timestamp to ensure that PM always caches the most currently hot data. Read operations do not modify the access timestamp.

[0041] The hot and cold data storage layer is the leaf node layer that actually stores key-value pairs. Based on the hot / cold attribute of the data, it stores it in PM and ZNS SSD respectively:

[0042] Data stored in PM primarily consists of hot node data, i.e., data that is frequently accessed (frequently read or written). Storing hot data in PM leverages its fast read / write speeds and byte-by-byte addressing capabilities to quickly respond to data access requests and reduce latency. It also avoids the write amplification and excessive garbage collection issues that can occur with frequent small-granular updates of hot data on ZNS SSDs, thus improving the overall performance of the storage system.

[0043] Data stored in ZNS SSDs primarily consists of cold node data, i.e., data that is rarely accessed. ZNS SSDs are characterized by large capacity and relatively low cost, making them suitable for storing large amounts of infrequently accessed data. Storing cold data in ZNS SSDs fully leverages their large-capacity storage advantage while reducing storage costs.

[0044] In summary, the pseudo-leaf node layer mainly stores key information used for indexing and managing leaf nodes, while the cold and hot data storage layer stores the actual key-value pairs in appropriate storage media according to the data access frequency characteristics.

[0045] Please see Figure 2B+ tree write operations are highly random, resulting in hot nodes exhibiting timeliness and dynamism. Failure to identify and properly place currently written hot nodes can lead to severe write amplification, degrading storage system performance. In read-write intensive applications, request operations over a period often follow specific distribution characteristics, such as the Zipfian distribution, where most operations are concentrated on a small portion of data. Therefore, the DPZB+tree proposed in this invention employs a clustering algorithm. Before executing read / write operations, it clusters the key-value information corresponding to all current read / write requests, finding the key at the cluster center. This key has the shortest average distance to all other keys, making it the hot key in the current read / write sequence. The node corresponding to this key will handle most of the read / write requests in subsequent operations. The specific operation is as follows:

[0046] Preferably, a hot / cold identification strategy for read / write requests is set to identify the hot leaf nodes corresponding to the read / write request sequence. The hot / cold identification strategy for read / write requests includes:

[0047] S101: Obtain the current M read / write request sequences, where each request in the read / write request sequence contains key-value information of key-value pair data that needs to be processed;

[0048] S102: Based on the principle of temporal locality, select the key values ​​of K requests from M read and write request sequences as the initial center point with a step size of M / K;

[0049] S103: According to the principle of proximity, each request in the read / write request sequence is assigned to the nearest center point based on the key value difference, forming K clusters;

[0050] d(k i k j )=|k i -k j |,i=1,2,…,M,j=1,2,…,K)

[0051] Where d(k) i k j ) represents the key value k of the i-th request in the read / write request sequence. i The key value k of the j-th center point j The gap;

[0052] S104: For each cluster, according to the formula Calculate the new centroids for each cluster, where G j Let j represent the j-th cluster, and k represent the k-th cluster. mj G represents the j-th cluster. j The key value of the m-th request in the data. This indicates that for the j-th cluster G jThe average of the key values ​​of all requests is calculated.

[0053] S105: Repeat steps S103 to S105 until the center points converge, then stop clustering; the key values ​​of the K center points obtained at this time are the key values ​​of the hot leaf nodes of the current read / write request sequence.

[0054] In this embodiment, after finding the K hot keys to be written through the above steps, the leaf nodes containing these K hot keys are the hot nodes of the current write sequence. Before performing specific operations, it is necessary to ensure that these leaf nodes have been stored as hot nodes in persistent memory. If the B+ tree is empty, the first leaf node will be created directly in persistent memory, and the current request sequence will be written to persistent memory. If the tree is not empty, the hot keys are first found through clustering algorithm, and then the leaf nodes corresponding to the hot keys are found by searching the B+ tree. After that, the hot node flag bool hot of the leaf node is used to determine whether it has been stored as a hot node in persistent memory. If it is not, it needs to be migrated from the ZNS where cold data is placed to the persistent memory where hot data is placed. At this time, it is first determined whether there is space in persistent memory to allocate to the leaf nodes that need to be migrated. If there is insufficient space, the hot node replacement algorithm needs to be executed first to select the node with the most recent access timestamp among the hot nodes and write it back to the ZNS SSD. By identifying hot keys and storing their corresponding hot nodes in persistent memory, the write amplification problem caused by hot nodes on ZNS SSDs is avoided, improving the storage system's efficiency in handling frequently accessed data. This fully utilizes the high-speed read and write characteristics of persistent memory and enhances the overall performance of the storage system.

[0055] Please see Figure 3 Preferably, after identifying the hot leaf nodes, the current read / write request sequence is processed as follows:

[0056] By searching the key values ​​of the hot leaf nodes in the current read / write request sequence, the corresponding leaf node is found in the B+ tree. It is then determined whether the leaf node is a cold node. If it is a cold node and there is no space in the persistent memory (PM), the node with the furthest most recent access timestamp among the hot leaf nodes is selected and written back to the ZNS SSD to free up space to write the leaf node to the persistent memory (PM). If it is a cold node but there is storage space in the persistent memory (PM), the leaf node is migrated from the ZNS SSD to the persistent memory (PM). If the leaf node is not a cold node, no operation is performed.

[0057] In this embodiment, since the small-capacity persistent memory is only used to cache leaf node index information and store some hot nodes, the number of nodes it can cache simultaneously has an upper limit. When the number of cached nodes reaches the upper limit, the persistent cache cannot accommodate new nodes, which will have a significant impact on the performance of the storage system. Therefore, we store node access timestamp information in the pseudo-leaf nodes as a standard for judging the coldness or hotness of leaf nodes. Initially, there are no cached nodes in the persistent cache, that is, the cache node set H is an empty set. Whenever a new cache node n is added to the set H or a read / write operation is performed on cache node n, the access timestamp Ta of n is set to the current timestamp Tc. When the number of elements in H reaches the upper limit, the persistent cache cannot accommodate new nodes. At this time, the node that is accessed furthest from the current timestamp is calculated as a candidate cold node. Finally, the cold node is flushed back to the ZNS SSD, and its cache space is replaced by a new hot node. Through the above method, the B+ tree nodes stored in the persistent memory are rationally organized, reducing the node write amplification rate.

[0058] Specifically, for leaf nodes, we use K-means clustering to identify K hot write requests. The K nodes corresponding to these K hot requests will handle most of the subsequent writes. Therefore, to reduce cascading updates and garbage collection caused by frequent writes on the ZNS SSD, we need to dynamically move the non-hot nodes from these K nodes to persistent memory as hot nodes. When persistent memory is insufficient, we need to implement a node replacement strategy to replace the least active hot nodes. For example... Figure 3 As shown, according to the algorithm, we obtain the hot key as 8. By searching, we find that leaf node 2, where the hot key resides, is currently stored as a cold node on the ZNS SSD. Therefore, we need to migrate node 2 entirely to persistent memory as a hot node before writing. However, persistent memory is already full. Therefore, we need to traverse the dynamic array of hot node pointers, find the coldest leaf node among the hot nodes (i.e., the leaf node whose access timestamp is furthest from the current timestamp, node 1), copy node 2 to the ZNS SSD, and modify its access timestamp to the current access timestamp. Simultaneously, we also copy node 1 to the ZNS SSD.

[0059] Preferably, the internal node is made to be a multiple of 64 bytes by adding an appropriate number of reserved fields, so that it can fully adapt to the granularity of cache read and write.

[0060] Preferably, the data structure size of the hot leaf node and the cold leaf node is uniformly adjusted to 4096 bytes, so that whether the leaf node is stored as a hot node in persistent memory or as a cold node in ZNS SSD, space fragmentation can be avoided and additional read and write operations can be reduced, thereby improving the performance of the entire storage system.

[0061] In this embodiment, the structural design of the leaf nodes in the B+ tree is crucial, directly affecting query efficiency, space utilization, the complexity of insertion and deletion operations, the performance of range queries, and memory and disk optimization. A reasonable design can significantly improve the overall performance of the B+ tree. Therefore, this invention, taking into account the characteristics of various hardware components, designs a B+ tree node suitable for the DRAM-PM-ZNS SSD storage architecture.

[0062] Given that the cache read / write granularity is 64 bytes, if the size of the data structure is not an integer multiple of 64 bytes, free space fragmentation may occur in the cache, thus affecting cache utilization and efficiency. Furthermore, data structures whose size does not match the cache read / write granularity may require multiple read / write operations to complete, as a cache line (64 bytes) may contain parts of other data structures, necessitating additional read / write operations and reducing efficiency. To address this, we added a reserved field to the internal nodes stored in DRAM to align their size with the cache read / write granularity, effectively avoiding space fragmentation and improving read / write efficiency. The pseudo-leaf node's data structure is set to 64 bytes. Since the cache read / write granularity is 64 bytes, setting the pseudo-leaf node's data structure to 64 bytes better adapts to the cache's read / write granularity. When the pseudo-leaf node's data needs to be read or written in the cache, because its size matches the cache's read / write granularity, it can be completely read and written by the cache, avoiding free space fragmentation and improving cache utilization and read / write efficiency. At the same time, it also reduces multiple read and write operations caused by the mismatch between data size and cache read and write granularity, thus reducing system overhead.

[0063] Meanwhile, the read / write granularity of persistent memory is 256 bytes, while that of ZNS SSD is 4096 bytes. To optimize performance, we adjusted the data structure size of the leaf nodes to 4096 bytes, ensuring that whether stored as a hot node in persistent memory or a cold node in ZNS SSD, space fragmentation and additional read / write operations are avoided, thereby improving the overall performance of the storage system.

[0064] While optimizing the storage structure, efficiently handling node splitting and merging operations is also a crucial issue. Leaf node splitting and merging operations in B+ trees play a vital role in ensuring tree balance, optimizing storage space utilization, improving query and write efficiency, supporting efficient range queries, reducing disk I / O operations, and enhancing system stability. A good splitting and merging strategy can significantly improve the overall performance of the storage system, especially in applications that require handling large-scale data and frequent updates and queries.

[0065] Please see Figure 4 While optimizing the storage structure, efficiently handling node splitting and merging operations is also a crucial issue. Leaf node splitting and merging operations in B+ trees play a vital role in ensuring tree balance, optimizing storage space utilization, improving query and write efficiency, supporting efficient range queries, reducing disk I / O operations, and enhancing system stability. A good splitting and merging strategy can significantly improve the overall performance of the storage system, especially in applications that require handling large-scale data and frequent updates and queries.

[0066] Preferably, when a node split occurs and the persistent memory (PM) is sufficient to store the new hot leaf node, regardless of whether the leaf node to be split is a cold or hot node, the new node after the split is stored in the persistent memory (PM). When the persistent memory storage space is full and a hot node splits, the hot node with the furthest access timestamp in the persistent memory is found and flushed back to the ZNS SSD to free up the persistent memory space. Then, the new node after the split is stored in the persistent memory. When the persistent memory storage space is full and a cold node splits, the two leaf nodes with the furthest and second furthest access timestamps in the persistent memory are found and flushed back to the ZNS SSD. After freeing up enough space, the new node after the split is stored in the persistent memory.

[0067] In this embodiment, regarding node splitting, the present invention considers the principle of spatial locality, ensuring that regardless of whether the leaf node to be split is a cold node or a hot node, it should be stored as a hot node in persistent memory after the split. This is because node splitting usually implies a large number of write operations have occurred; therefore, after the split, all nodes should be stored as hot nodes in persistent memory to maximize support for data access. The specific process is as follows: Figure 4 As shown. Initially, the node corresponding to the first piece of data we write will be stored as a hot node in persistent memory. Furthermore, if persistent memory has enough space to store a new hot node, when a node split occurs, all new nodes should be stored in persistent memory to improve access efficiency. When persistent memory is full, if a node split occurs, for a hot node split, we only need to find the node with the furthest access timestamp and flush it back to the ZNS SSD to free up persistent memory space. For a cold node split, we need to find two nodes (the furthest and second furthest access timestamps) and flush them back to the ZNS SSD.

[0068] Preferably, when the number of key-value pairs in a leaf node is less than N / 2 of its maximum storage capacity after a deletion operation, a node merging operation is triggered. If the leaf node has no left sibling node, it is merged with its right sibling node. If the leaf node has no right sibling node, it is merged with its left sibling node. If the leaf node has both a left and a right sibling node, and either the left or right sibling node is a hot leaf node, the leaf node is preferentially merged with the hot leaf node. If the leaf node has both a left and a right sibling node, and both the left and right sibling nodes are either hot or cold leaf nodes, the leaf node is merged with its left sibling node by default.

[0069] In this embodiment, the core idea of ​​DPZB+tree for node merging is to maximize the utilization efficiency of persistent memory. Specifically, when the number of key-value pairs in a leaf node is less than N / 2 of its maximum storage capacity after a deletion operation, node merging is triggered. At this time, priority is given to merging with the hottest nodes among the left and right sibling nodes to make full use of the hottest nodes in persistent memory and improve the access efficiency and storage performance after merging.

[0070] In summary, DPZB+tree significantly improves the performance of ZNS SSD-based systems, reduces tail latency, and minimizes write amplification through a series of optimization measures. First, DPZB+tree employs a B+ tree index structure, which outperforms LSM-tree under read-intensive loads. Because the leaf nodes of DPZB+tree are ordered and linked, the Scan operation can efficiently traverse data sequentially, thus quickly completing data retrieval and processing. Second, DPZB+tree avoids the reliance on merge operations in LSM-trees to free up space and improve query speed, avoiding the performance overhead of multiple merges. Therefore, in terms of long-tail latency, DPZB+tree is far superior to LSM-tree. Furthermore, DPZB+tree uses the K-means algorithm for hot and cold data identification, and under sequential write workloads, its read and write speeds are significantly better than other data structures such as LSM-tree. DPZB+tree also achieves fast recovery operations by persistently storing pseudo-leaf nodes in memory. When a system failure occurs, DPZB+tree can complete the recovery process in milliseconds because it reduces the recovery time and I / O overhead by storing important data in persistent memory.

[0071] This invention, DPZB+tree, employs a DRAM-PM-ZNS SSD hybrid storage architecture. By storing the index information of leaf nodes in persistent memory (PM), it effectively solves the additional overhead caused by B+ tree cascading updates. Secondly, DPZB+tree designs a low-overhead one-dimensional K-means algorithm to identify currently hot-request nodes for writing. By storing these hot nodes in PM, it reduces the write overhead caused by frequent access to hot nodes, thereby improving the overall system performance. Simultaneously, to address the limitations of PM capacity, DPZB+tree designs a hot node replacement algorithm. By dynamically replacing hot and cold nodes based on the access timestamps of leaf nodes, it optimizes PM utilization efficiency. Finally, DPZB+tree optimizes the leaf node structure design and splitting / merging algorithms, fully utilizing the characteristics of small-capacity PMs to effectively improve the performance of the storage system.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A B+ tree index construction method based on ZNS SSD hot and cold data dynamic placement, characterized by, Comprise: Persistent memory PM, dynamic random access memory DRAM and ZNS SSD hard disk; Persistent memory PM and dynamic random access memory DRAM are directly connected to CPU memory bus, and have the ability to be accessed by CPU byte by byte; The internal nodes of the B+ tree are stored in dynamic random access memory DRAM, wherein the internal nodes only store index information for locating leaf nodes; The pseudo-leaf nodes of the B+ tree are stored in persistent memory PM, wherein the pseudo-leaf nodes only store key index information of the leaf nodes; The key index information of the leaf nodes includes: node minimum value, hot node flag bit, parent node pointer, key quantity, left brother node pointer, right brother node pointer, pointer to persistent memory, pointer to ZNS SSD and access timestamp; The leaf nodes that really store key value pairs in the B+ tree are divided into hot leaf nodes and cold leaf nodes according to data access frequency, wherein the hot leaf nodes are stored in persistent memory PM and the cold leaf nodes are stored in ZNS SSD hard disk.

2. The B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data according to claim 1, characterized in that, The node minimum value is used to reconstruct the B+ tree structure; The hot node flag bit is used to mark whether the leaf node is a hot leaf node, if it is marked as a hot leaf node, the pointer to persistent memory PM is used to query data from persistent memory PM when accessing, otherwise the pointer to ZNS SSD is used to query data from ZNS SSD; The parent node pointer is used to record the parent node position information of the leaf node in the B+ tree, and the structure of the B+ tree can be easily traced upwards and adjusted through the parent node pointer; The key quantity is used to record the number of keys contained in the leaf node; The left brother node pointer and the right brother node pointer are used to connect adjacent leaf nodes at the same level, so that an ordered linked list structure is formed between the leaf nodes; The access timestamp is used to record the time of the last read / write operation of the leaf node.

3. The B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data according to claim 1, characterized in that, A read / write request cold and hot identification strategy is set to identify the hot leaf nodes corresponding to the read / write request sequence, and the read / write request cold and hot identification strategy comprises: S101: Obtain M current read / write request sequences, wherein each request in the read / write request sequence contains key value information of key value pair data to be processed; S102: Based on the principle of temporal locality, select K keys of requests as initial center points from M read / write request sequences with M / K as step; S103: According to the nearest principle, attribute each request in the read / write request sequence to the nearest center point according to the key value gap to form K clusters; wherein, represents a gap between a key value of an i-th request in a read-write request sequence and a key value of an i-th center point of the read-write request sequence.​​ S104: For each cluster, compute new center point according to formula for each cluster, where denotes the jth cluster, denotes the mth request's key value in the jth cluster denotes the average of all requests' key values in the jth cluster denotes the average of all requests' key values in the jth cluster denotes the average of all requests' key values in the jth cluster S105: Continuously repeat steps S103-S105 until the center point converges, stop clustering; At this time, the key values of the K center points obtained are the key values of the hot leaf nodes of the current read / write request sequence.

4. The B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data according to claim 3, characterized in that, After identifying the hot leaf nodes, the current read / write request sequence is processed, and the process is as follows: The key value of the hot leaf node in the current read-write request sequence is searched in the B+ tree to find the corresponding leaf node, and it is judged whether the leaf node is a cold node. If it is a cold node and the persistent memory PM has no space to store, the node with the farthest access timestamp in the hot leaf node is selected to be written back to the ZNS SSD hard disk to free up space to write the leaf node to the persistent memory PM. If it is a cold node but the persistent memory PM has storage space, the leaf node is migrated from the ZNS SSD hard disk to the persistent memory PM. If the leaf node is not a cold node, no operation is performed.

5. The B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data according to claim 3, characterized in that, The internal node is added with a proper amount of reserved field to make the internal node size an integer multiple of 64 bytes, so as to ensure that it can be completely adapted to the cache read-write granularity.

6. The B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data according to claim 3, characterized in that, The data structure size of the hot leaf node and the cold leaf node is uniformly adjusted to 4096 bytes, so that the leaf node can avoid space fragmentation and reduce additional read-write operations whether it is stored in the persistent memory as a hot node or stored in the ZNS SSD as a cold node, thereby improving the performance of the entire storage system.

7. The B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data according to claim 3, characterized in that, When node splitting occurs and the persistent memory PM is sufficient to store the new hot leaf node, whether the leaf node to be split is a cold node or a hot node, the new node after splitting is stored in the persistent memory PM; When the persistent memory storage space is full and the hot node splits, find the hot node with the farthest access timestamp in the persistent memory and flush it back to the ZNS SSD to release the persistent memory space, and then store the new node after splitting in the persistent memory; when the persistent memory storage space is full and the cold node splits, find the two leaf nodes with the farthest and second farthest access timestamps in the persistent memory, and flush them back to the ZNS SSD to release enough space, and then store the new node after splitting in the persistent memory.

8. The B+ tree index construction method based on ZNS SSD for dynamic placement of hot and cold data according to claim 3, characterized in that, When the leaf node is deleted, the number of key-value pairs is less than N / 2 of its maximum storage capacity, triggering node merging operation; If the leaf node has no left sibling node, it is merged with the right sibling node; if the leaf node has no right sibling node, it is merged with the left sibling node; if the leaf node has both left and right sibling nodes, and the left or right sibling node is a hot leaf node, the leaf node is preferentially merged with the hot leaf node; if the leaf node has both left and right sibling nodes, and the left and right sibling nodes are hot leaf nodes or cold leaf nodes, the leaf node is merged with the left sibling node by default.

Images