A skip list structure based on persistent memory and its access method

Abstract

The present invention discloses a skip list structure based on persistent memory and an access method thereof, allocating persistent memory to the bottom-level linked list of the skip list, and persisting the bottom-level linked list of the skip list; designing a skip list structure optimized for CPU cache; inserting data into the skip list based on persistent memory; deleting data from the skip list based on persistent memory; and managing the free space of persistent memory. Through the present invention, when inserting, searching, and deleting operations are performed in the skip list based on persistent memory, the CPU cache and persistent memory hardware characteristics can be fully utilized, thereby improving the performance of the operation.

Description

Technical Field

[0001] The present invention relates to the technical field of persistent memory, and in particular to a skip list structure based on persistent memory and an access method thereof. Background Art

[0002] Persistent Memory: Persistent Memory is a new type of memory-level storage hardware that supports byte-granular addressing and can store data persistently. It also has the characteristics of low read and write latency, high bandwidth, and large capacity. Persistent memory devices not only have read and write performance similar to dynamic random access memory (DRAM), with a read and write latency of less than 1 microsecond, but also have the ability to persist data like traditional storage devices such as disks, and will not lose data when power is off. The emergence of persistent memory has led researchers to design storage structures based on persistent memory, such as indexes in database systems, to ensure data persistence while improving the performance of database systems.

[0003] A skip list is an ordered linked list that can perform binary search. It adds multiple levels of indexes to the original ordered linked list and performs fast search on the indexes through binary search. Skip lists are widely used as memory indexes for databases due to their good insertion and deletion performance. Modern computer systems use CPU cache to speed up read and write access to data and instructions, and the CPU cache has a great impact on program performance. However, since skip lists use linked lists to store data, the data addresses between linked list nodes are jumpy, and existing skip list designs cannot fully utilize modern CPU caches.

[0004] Persistent memory devices are internally read and stored in 4 cache lines (256 bytes) of granularity. A small write-merge buffer is used to avoid write amplification, because the transfer size between persistent memory and the CPU is the same as DRAM, 64 bytes. Therefore, it is necessary to optimize for 256 bytes, and peak throughput can only be achieved when using multiples of the block size. Summary of the invention

[0005] The purpose of the present invention is to provide a skip table structure based on persistent memory and an access method thereof in view of the deficiencies in the prior art. The skip table structure uses a hybrid main memory data structure of persistent memory and volatile memory to solve the problem of low CPU cache utilization of the persistent skip table data structure.

[0006] The objective of the present invention is achieved through the following technical solution: a skip list structure based on persistent memory, storing the bottom-level linked list data of the skip list in persistent memory, and storing the remaining upper-level linked lists in volatile memory, thereby forming a skip list structure based on persistent memory.

[0007] Furthermore, the nodes of the bottom-level linked list of the skip list are stored in a block that can accommodate multiple node spaces.

[0008] Furthermore, the layout of the bottom-level linked list of the skip list on the persistent memory is divided into five parts, namely: a metadata area, a bitmap area, a log area, an idle queue area and a data area.

[0009] Furthermore, the metadata area is used to record the size of a single data, the starting address of the bitmap area, log area, free queue area and data area, the size of the metadata area, bitmap area, log area, free queue area and data area, and metadata of the block size.

[0010] Furthermore, the bitmap area is used to mark whether each data of each block in the data area is used.

[0011] Furthermore, the log area stores operation logs for ensuring data consistency during failure recovery.

[0012] Furthermore, the idle queue area is used to manage data blocks that are marked as idle after data is deleted.

[0013] Furthermore, the data area includes a plurality of data blocks, each data block includes a plurality of data; the data area includes used files and unused files; and the boundary between the used files and the unused files is identified by a pointer unused_ptr.

[0014] A method for accessing a skip list structure based on a persistent memory according to any one of the above, comprising the steps of inserting data, deleting data and managing free space;

[0015] The step of inserting data is as follows: first, find the first node in the bottom linked list that is smaller than the inserted data through the linked list on the volatile memory, and judge whether the block to which it belongs has free space according to the bitmap bit corresponding to the first node that is smaller than the inserted data; if so, select the block to which it belongs to prepare to insert the data, otherwise a new block needs to be allocated; the new block is preferentially obtained from the idle queue area, if there is a free block in the idle queue area, it is taken out and the next read position of the idle queue area is updated, if there is no free block in the idle queue area, a new block is allocated from the idle queue area; after the block is allocated, the data is inserted into the allocated block, and the corresponding bit of the bitmap is set to 1; then the pointer of the data is updated to complete the data insertion operation;

[0016] The step of deleting data is as follows: searching for the address of the data to be deleted in the jump table, if not found, then ending the operation; if found, then deleting it on the block to which the data belongs, and setting the corresponding bit on the bitmap to 0; judging whether all the data in the block to which the data belongs has been cleared, if so, then adding the block to which the data belongs to the free queue area, otherwise ending the operation;

[0017] The free space management includes a free block queue and unused portions of a data area;

[0018] The steps to obtain a free block are: first determine whether the free queue area is empty. If it is not empty, take a free block from the free queue area and return it. If it is empty, get a free block from the unused part of the data area, and then update the unused_ptr pointer to point to the starting position of the latest unused part.

[0019] The beneficial effects of the present invention are as follows: the structural design of the present invention enables the persistent skip list to fully utilize the CPU cache during insertion and search, can effectively reduce the delay of the persistent skip list insertion and search operations, and will increase the bandwidth for accessing the persistent memory device. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a schematic diagram of the data structure of the jump table of the present invention;

[0022] Figure 2 A schematic diagram of the layout of the bottom-level linked list of the jump list of the present invention on the persistent memory;

[0023] Figure 3 It is a schematic diagram of the structure of the data block of the present invention;

[0024] Figure 4 It is a flow chart of a method for inserting data into a jump table according to the present invention;

[0025] Figure 5 It is a flow chart of a method for deleting data on a jump table according to the present invention;

[0026] Figure 6 is a schematic diagram of free space management of the present invention; DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of structures and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0028] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0030] like Figure 1 As shown, a skip list structure based on persistent memory stores the bottom-level linked list data of the skip list in the persistent memory, and the remaining upper-level linked lists are stored in the volatile memory, thereby forming a skip list structure based on persistent memory.

[0031] The nodes of the bottom-level linked list of the skip list are stored in a block that can accommodate multiple node spaces;

[0032] In one embodiment, the nodes of the bottom-level linked list of the skip list are stored in a 256-byte block; it is understandable that they may also be stored in a 128-byte, 512-byte or other block.

[0033] Specifically, the skip list has 9 key / value pairs, which are stored in order of key size. The node with key 1 points to the node with key 2, the node with key 2 points to the node with key 3, and so on, to obtain an ordered linked list. The bottom-level linked list is stored on the persistent memory device, so that data can be saved when the device is powered off, while the volatile memory DRAM is based on the index generated by the sequential linked list of the persistent memory, which is used to quickly index the nodes of the bottom-level ordered linked list.

[0034] like Figure 2 As shown, the layout of the bottom-level linked list of a skip list based on persistent memory on the persistent memory is divided into five parts, namely: metadata area, bitmap area, log area, free queue area, and data area.

[0035] The metadata records the size of a single data, the starting address of other zones, the size of each zone, the block size and other necessary metadata. Specifically, in the embodiment of the present invention, the metadata zone starts at position 0, is 4KB in size, and contains the following metadata: the key size is 8 bytes, the value size is 16 bytes, and the next pointer is 8 bytes; the bitmap zone size is 1GB; the log zone size is 16GB; the idle queue zone size is 16GB; and the data zone size is 256GB. The starting position of each zone can be calculated based on the starting position and size of the previous zone.

[0036] The bitmap area is used to mark whether the data in each block of the data area is used.

[0037] One bit in the bitmap represents one data, and the bits from left to right correspond to the data from left to right in the data area. If the corresponding bitmap bit is 0, it means that the address is free, and if the corresponding bitmap bit is 1, it means that the address has been used. When all bits of the bitmap are 1, it means that there is no free space in the block; when all bits of the bitmap are 0, it means that the block is a free block. Specifically, assuming there is a 256-Byte block, which can store 8 32-Byte nodes, then the corresponding bitmap is 8 bits, occupying 1 Byte of space. Assume that the binary representation of the bitmap is 00110001, it means that the 3rd, 4th, and 8th positions of the block have been used, and the other positions are free.

[0038] The log area stores operation logs to ensure data consistency during failure recovery.

[0039] The free queue area is used to manage data blocks that are marked free after data is deleted.

[0040] The data area contains multiple data blocks, and each data block contains multiple data.

[0041] like Figure 3 As shown, each data block contains multiple data, and the data is composed of a structure such as [key, value, next pointer]. The organization of the data block is: [key1, value1, next pointer 1; key2, value2, next pointer 2; key3, value3, next pointer 3...]. In this embodiment, the size of the data block is 256 bytes, the size of the key is 8 bytes, the size of the value is 16 bytes, and the size of the next pointer is 8 bytes. Therefore, the size of a single data is 32 bytes, and a data block contains 8 data.

[0042] like Figure 4 As shown, the steps of inserting data on the jump table of the present invention are:

[0043] First, find the first node in the bottom linked list that is smaller than the data to be inserted through the linked list on the volatile memory, and determine whether the block to which it belongs has free space according to the bitmap corresponding to the data node. If there is free space, then select the block to insert the data. Specifically, if the binary value of the bitmap corresponding to the block to which the node belongs is 00111100, it means that there are 4 free spaces, namely the 1st, 2nd, 7th, and 8th positions.

[0044] If there is no free space, a new block is allocated. The block to be allocated is first obtained from the free queue area. If there is a free block in the free queue area, it is taken out and the next read position of the free queue area is updated. If the free queue area is empty, a new block is allocated from an unused file. The unused file belongs to the data area. In the data area, the boundary between the used and unused files is marked by a pointer unused_ptr.

[0045] After selecting a block, insert the data into the block and set the corresponding bit in the bitmap to 1. Then update the data pointer to complete the insertion operation.

[0046] like Figure 5 As shown, the steps of deleting data on the jump table of the present invention are:

[0047] Search the address of the data to be deleted in the jump table. If not found, end the operation. If found, delete it in the block to which the data belongs and set the corresponding bit in the bitmap to 0. Determine whether all the data in the block to which it belongs has been cleared. If so, add the block to the free queue area, otherwise end the operation.

[0048] like Figure 6 As shown, the free space management of the embodiment of the present invention is divided into two parts, one part is the free block queue, and the other part is the unused part of the data area. An unused_ptr pointer points to the used part and the unused part of the data area.

[0049] The steps to get a free block are:

[0050] Determine whether the free queue area is empty. If the free queue area is not empty, take a free block from the free queue area and return it. If the free queue area is empty, get a block of space from the unused part of the data area, and then update the unused_ptr pointer to point to the starting position of the latest unused part.

[0051] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0052] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

[0053] Those skilled in the art will readily appreciate other embodiments of the present application after considering the description and practicing the contents disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application. The description and examples are intended to be exemplary only.

[0054] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for accessing a skip list structure based on persistent memory, It is characterized in that The bottom-level linked list data of the skip list is stored in the persistent memory, and the rest of the upper-level linked lists are stored in the volatile memory, forming a skip list structure based on the persistent memory; The layout of the bottom-level linked list of the skip list on the persistent memory is divided into five parts, namely: metadata area, bitmap area, log area, free queue area and data area; The metadata area is used to record the size of a single data, the starting address of the bitmap area, log area, free queue area and data area, the size of the metadata area, bitmap area, log area, free queue area and data area, and the metadata of the block size; The access method includes the steps of inserting data, deleting data and managing free space; The step of inserting data is: firstly, searching the first node in the bottom linked list that is smaller than the inserted data through the linked list on the volatile memory, and judging whether the block to which the node belongs has free space according to the bitmap bit corresponding to the first node that is smaller than the inserted data; If there is, the block to which the data belongs is selected to be inserted, otherwise a new block needs to be allocated; the new block is obtained from the free queue area first, if there is a free block in the free queue area, it is taken out and the next read position of the free queue area is updated, if there is no free block in the free queue area, a new block is allocated from the free queue area; after the block is allocated, the data is inserted into the allocated block, and the corresponding bit of the bitmap is set to 1; then the data pointer is updated to complete the data insertion operation; The step of deleting data is as follows: searching for the address of the data to be deleted in the jump list, if not found, then ending the operation; if found, then deleting it on the block to which the data belongs, and setting the corresponding bit on the bitmap to 0; judging whether all the data in the block to which the data belongs has been cleared, if so, then adding the block to which the data belongs to the free queue area, otherwise ending the operation; The free space management includes a free block queue and unused portions of a data area; The steps to obtain a free block are: first determine whether the free queue area is empty. If it is not empty, take a free block from the free queue area and return it. If it is empty, get a free block from the unused part of the data area, and then update the unused_ptr pointer to point to the starting position of the latest unused part.

2. The method for accessing a skip list structure based on persistent memory according to claim 1, It is characterized in that The nodes of the bottom-level linked list of the skip list are stored in a block that can accommodate multiple node spaces.

3. The method for accessing a skip list structure based on persistent memory according to claim 1, It is characterized in that The bitmap area is used to mark whether each data of each block in the data area is used.

4. The method for accessing a skip list structure based on persistent memory according to claim 1, It is characterized in that The log area stores operation logs to ensure data consistency during failure recovery.

5. The method for accessing a skip list structure based on persistent memory according to claim 1, It is characterized in that The idle queue area is used to manage data blocks that are marked as idle after data is deleted.

6. The method for accessing a skip list structure based on persistent memory according to claim 1, It is characterized in that The data area includes a plurality of data blocks, each of which includes a plurality of data; the data area includes used files and unused files; the boundary between the used files and the unused files is identified by a pointer unused_ptr.

Images