Index operation method, electronic equipment and readable storage medium

Abstract

The invention discloses an index operation method, electronic equipment and a readable storage medium, and belongs to the field of computers. In the application, an index comprises an index entry and metadata of the index entry, the index entry is stored in a persistent memory, the metadata is stored in a memory, and the operation method of the index comprises: obtaining a target operation request; obtaining target metadata from metadata stored in a memory based on the target operation request; obtaining a target index entry from index entries stored in a persistent memory based on the target metadata; and executing the target operation on the target index entry. The metadata does not need to be acquired from the persistent memory, so that the read-write overhead for reading and writing the metadata in the persistent memory is reduced, and the search performance of the index entry is improved.

Description

Technical Field

[0001] This application belongs to the field of computers, and particularly relates to an operation method for an index, an electronic device, and a readable storage medium. Background Art

[0002] In the field of computers, persistent memory (PMEM) has both the persistence (i.e., non-volatility) of external memory and an access rate close to that of memory, and can significantly improve data access efficiency. Therefore, using persistent memory to replace external memory such as traditional disks and redesigning the index of the database accordingly has important application and practical value for database acceleration.

[0003] In related technologies, in order to improve the search performance of an index, metadata of index entries is generally set in the index, and the search for index entries is accelerated through the metadata. In order to ensure the consistency between the metadata and the index entries during a power outage, generally both the index entries and the metadata need to be maintained in persistent memory.

[0004] However, in this method, the metadata occupies a large amount of overhead in persistent memory, affecting the search performance of the index. For example, when using the index to process different types of requests, in addition to performing read and write operations on the persistent memory for the index entries, separate read and write operations on the persistent memory for the metadata are also required. The metadata occupies a large amount of overhead in the read and write persistent memory, affecting the search performance of the index. Summary of the Invention

[0005] Embodiments of this application provide an operation method for an index, an electronic device, and a readable storage medium to solve the problem in related technologies that the metadata of index entries occupies a large amount of overhead in persistent memory, affecting the search performance of the index.

[0006] In a first aspect, embodiments of this application provide an operation method for an index. The index includes index entries and metadata of the index entries. The index entries are stored in persistent memory, and the metadata is stored in memory. The method includes:

[0007] Obtain a target operation request;

[0008] Based on the target operation request, obtain target metadata from the metadata stored in memory;

[0009] Based on the target metadata, obtain target index entries from the index entries stored in persistent memory;

[0010] Perform a target operation on the target index entries.

[0011] In a second aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory. The memory stores a program or instructions that can run on the processor. When the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0012] In a third aspect, an embodiment of the present application provides a readable storage medium, on which a program or instructions are stored. When the program or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which when executed by a processor implements the method described in the first aspect.

[0014] In the embodiment of the present application, the index includes index entries and metadata of the index entries. The index entries are stored in persistent memory, and the metadata is stored in memory. By obtaining a target operation request; based on the target operation request, obtaining target metadata from the metadata stored in memory; based on the target metadata, obtaining a target index entry from the index entries stored in persistent memory; and performing a target operation on the target index entry. In this way, compared with the related art in which the metadata of the index entries is stored in persistent memory, since the metadata of the index entries is stored in memory in the present application, when processing the target operation request, the target metadata is directly obtained from memory without obtaining the target metadata from persistent memory, reducing the read / write overhead of reading and writing the target metadata in persistent memory and improving the lookup performance of the index entries. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic flowchart of an operation method of an index provided by an embodiment of the present application;

[0016] Figure 2 It is a schematic flowchart of another operation method of an index provided by an embodiment of the present application;

[0017] Figure 3 It is a schematic flowchart of a write operation of an index provided by an embodiment of the present application;

[0018] Figure 4-1 It is a schematic flowchart of reconstructing target metadata in an index provided by an embodiment of the present application;

[0019] Figure 4-2 It is a schematic flowchart of another reconstruction of target metadata in an index provided by an embodiment of the present application;

[0020] Figure 4-3 It is a schematic flowchart of yet another reconstruction of target metadata in an index provided by an embodiment of the present application;

[0021] Figure 5 Schematic flowchart of another indexing operation method provided by an embodiment of the present application;

[0022] Figure 6-1 Schematic structural diagram of a B+ tree index provided by an embodiment of the present application;

[0023] Figure 6-2 Schematic flowchart of a write operation of a B+ tree index provided by an embodiment of the present application;

[0024] Figure 6-3 Schematic flowchart of a single-point query operation of a B+ tree index provided by an embodiment of the present application;

[0025] Figure 6-4 Schematic flowchart of a range query operation of a B+ tree index provided by an embodiment of the present application;

[0026] Figure 6-5 Schematic flowchart of a deletion operation of a B+ tree index provided by an embodiment of the present application;

[0027] Figure 7 Schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0028] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, rather than all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. generally belong to the same category, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0030] B+ tree index is one of the most common index structures in relational databases. Due to its excellent performance and range query capabilities, it is widely used in databases and big data systems to accelerate database query performance. Most traditional indexes are persisted using disks, writing index data in volatile memory to non-volatile disk storage. However, compared with memory, disks have lower bandwidth and larger latency, and there is also the problem of write amplification. Since indexes are accessed frequently, their performance can affect the overall performance of the database system.

[0031] With the booming development of new NVM (Non-Volatile Memory) technology, a type of random-access, non-volatile memory has emerged. Non-volatile means that data will not disappear even when the power is off. Therefore, data can be retained even in the event of a computer power failure, system crash, or normal shutdown. Due to its non-volatility and compatibility with traditional DRAM interfaces, the memory based on NVM is also called persistent memory (PMEM). PMEM combines the persistence of traditional hard disks and the access rate close to that of memory, which can significantly improve data access efficiency. Therefore, redesigning database indexes using persistent memory has important application and practical value for database acceleration.

[0032] Traditional B+ tree indexes cannot directly use persistent memory by replacing hardware because B+ trees are disk-optimized structures that use a page size of 4K (4 * 1024 = 4096B) to convert random disk writes into sequential writes to improve performance. However, persistent memory itself has some characteristics, such as 256B internal blocks and support for byte-addressability. The design of traditional B+ trees cannot fully utilize the hardware performance of PMEM, resulting in a significant read-write amplification problem and wasting the bandwidth of PMEM hardware instead.

[0033] To ensure crash consistency and improve the performance of index item lookup, bitmap and fingerprint technologies are adopted in related technologies and placed in the first 64B address space of the leaf nodes of the B+ tree index. Each time an index item is written to a leaf node, it is first necessary to ensure that the data is flushed into persistent memory, and then the bitmap of the leaf node needs to be updated. If it is an insert operation, the fingerprint of the leaf node also needs to be updated. To ensure the crash consistency of the index structure, the update of the metadata of the leaf node (such as the bitmap and fingerprint) often requires a separate PMEM write. Otherwise, after the machine power-off, the metadata of the index item will be lost, resulting in data inconsistency. In scenarios with frequent updates, such metadata often occupies a relatively large PMEM write overhead, affecting the overall performance of the index.

[0034] Based on this, in order to reduce the read and write overhead of reading and writing the metadata of index items in persistent memory, an embodiment of the present application provides an index operation method. Compared with the related technology of maintaining the metadata of index items in persistent memory, the embodiment of the present application maintains the metadata of index items (such as bitmaps and fingerprints) in memory, which can avoid reading and writing metadata in persistent memory, thereby reducing the read and write overhead of persistent memory and improving the search performance of index items.

[0035] In addition, in order to ensure consistency during crashes, an embodiment of the present application provides an inert persistence strategy, which records verification information in index items through atomic writes. Therefore, when metadata is lost due to a crash, the metadata is reconstructed in memory based on the verification information of the index items. Compared with related technologies, there is no need to synchronize persistent metadata, thereby reducing additional persistence overhead.

[0036] The following is a detailed description of the index operation method provided in the embodiment of the present application through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0037] With the emergence of new storage devices such as persistent memory, new requirements are put forward for the architectural design of traditional database systems. Persistent memory can replace memory or solid-state drives because it has a bandwidth close to that of memory, higher storage density and lower storage cost per unit capacity. As an important component to accelerate database query performance, database indexes are currently mostly persisted using disks, and the performance of disks is poor, which will greatly reduce the overall performance of the system. Therefore, the use of persistent memory to replace disks and optimize database systems has been put into practical application. Based on this, in specific application scenarios, the index operation method provided in the embodiment of the present application can use persistent memory to redesign the database index, maintain the index items of the database index in persistent memory, and significantly improve data access efficiency.

[0038] Figure 1 A schematic flowchart of an index operation method provided in an embodiment of the present application.

[0039] like Figure 1 As shown, an embodiment of the present application provides an index operation method, which may include:

[0040] Step 110: Obtain a target operation request;

[0041] Step 120: based on the target operation request, obtaining target metadata from metadata stored in the memory;

[0042] Step 130: Obtain a target index item from the index items stored in the persistent memory based on the target metadata;

[0043] Step 140: Perform a target operation on the target index entry.

[0044] In the embodiments of the present application, the index includes index entries and metadata of the index entries. The index entries are stored in persistent memory, and the metadata is stored in memory.

[0045] Among them, an index is a data structure used to accelerate the query speed of a database. Each item in the index table is called an index entry, and an index entry includes a key-value pair (index key and index value). The metadata of the index entry is information used to describe the index entry. The metadata of the index entry may include a bitmap and a fingerprint. The bitmap is used to indicate whether the index entry is occupied, and the fingerprint is used to accelerate the search for the index entry.

[0046] Among them, persistent memory is a randomly accessible, non-volatile memory. Non-volatile means that data will not disappear even when the power is off. Therefore, persistent memory can still retain data in the case of a computer power-off, system crash, and normal shutdown. Persistent memory combines the non-volatility of traditional external storage and an access rate close to that of traditional memory. The access rate of persistent memory is greater than that of external storage, which can significantly improve data access efficiency.

[0047] Among them, memory is a volatile memory, and the data stored in it will disappear when the power is off. The access rate of memory is generally slightly greater than that of persistent memory and much greater than that of external storage.

[0048] Among them, the index entries are stored in persistent memory, which can accelerate the query speed of the database and significantly improve data access efficiency.

[0049] Among them, the metadata of the index entry is stored in memory, which can reduce the read and write overhead of reading and writing metadata in persistent memory and improve the search performance of the index entry.

[0050] In step 110, the target operation request may be an operation request input by the user. The target operation request may include, but is not limited to, at least one of the following: a write operation request, a read operation request, and a delete request. Among them, the write operation corresponding to the write operation request may be an index entry update operation or an index entry insertion operation. The read operation corresponding to the read operation request may be a single-point query operation or a range query operation. The delete request corresponds to a delete operation.

[0051] In step 120, based on the target operation request, obtain the target metadata from the metadata stored in memory. Among them, the target operation request carries a target index key. In the embodiments of the present application, the target metadata can be obtained from the metadata stored in memory based on the target index key carried by the target operation request. Among them, the target metadata may be the metadata associated with the target index key in the metadata stored in memory.

[0052] In step 130, based on the target metadata, obtain the target index entry from the index entries stored in the persistent memory. Among them, the target metadata can be the metadata associated with the target index entry, and the target metadata can accelerate the acquisition of the target index entry.

[0053] In step 140, the target operation is the operation corresponding to the target operation request. The target operation may include, but is not limited to: write operation, read operation, and delete operation. Among them, the write operation may be an index entry update operation or an index entry insertion operation. The read operation may be a single-point query operation or a range query operation.

[0054] It can be understood that compared with the related art where both the index entry and the metadata of the index entry are stored in the persistent memory, in the embodiment of the present application, the metadata of the index entry is stored in the memory, and there is no need to store the metadata of the index entry in the persistent memory. When processing the target operation request, directly obtain the target metadata from the metadata stored in the memory, without obtaining the target metadata from the persistent memory, reducing the read and write overhead of reading and writing the target metadata in the persistent memory, and improving the search performance of the index entry.

[0055] According to the index operation method provided by the embodiment of the present application, the index includes an index entry and the metadata of the index entry. The index entry is stored in the persistent memory, and the metadata is stored in the memory. By obtaining the target operation request; based on the target operation request, obtain the target metadata from the metadata stored in the memory; based on the target metadata, obtain the target index entry from the index entries stored in the persistent memory; perform the target operation on the target index entry. In this way, compared with the related art where the metadata of the index entry is stored in the persistent memory, since the metadata of the index entry is stored in the memory in the present application, when processing the target operation request, directly obtain the target metadata from the memory, without obtaining the target metadata from the persistent memory, reducing the read and write overhead of reading and writing the target metadata in the persistent memory, and improving the search performance of the index entry.

[0056] In a specific embodiment, the target metadata may include a fingerprint array, and the fingerprint array can be used to accelerate the search for index entries. In order to reduce the read overhead of reading the fingerprint array in the persistent memory, the index operation method provided by the embodiment of the present application may include:

[0057] Step 210: Obtain the target operation request, where the target operation request carries a target index key;

[0058] Step 220: Based on the target index key carried in the target operation request, determine the fingerprint array associated with the target index key from the metadata stored in the memory;

[0059] Step 230: When there is a target fingerprint value in the fingerprint array, based on the mapping relationship between N fingerprint values in the fingerprint array and N index entries, obtain a target index entry corresponding to the target fingerprint value from the N index entries stored in the persistent memory;

[0060] Step 240: Perform a target operation on the target index entry, the target operation.

[0061] Among them, step 210 can be a sub-step of step 110.

[0062] In step 210, the target operation request may include, but is not limited to, at least one of the following: a write operation request, a read operation request, and a delete request.

[0063] Among them, step 220 can be a sub-step of step 120. The target metadata may include a fingerprint array, and the fingerprint array is stored in memory.

[0064] In step 220, based on the target index key carried in the target operation request, determine the fingerprint array associated with the target index key from the metadata stored in memory.

[0065] Among them, in the embodiment of the present application, the fingerprint array associated with the target index key is directly obtained from memory without obtaining the fingerprint array from the persistent memory.

[0066] Among them, the fingerprint array obtained in the embodiment of the present application can be used later to determine the target index entry.

[0067] Among them, step 230 can be a sub-step of step 130. The fingerprint array includes N fingerprint values, and the N fingerprint values in the fingerprint array are used to determine N index entries; N is an integer greater than 1.

[0068] In step 230, obtain the target fingerprint value based on the target index key carried in the target operation request. When there is a target fingerprint value in the fingerprint array, based on the mapping relationship between N fingerprint values in the fingerprint array and N index entries, obtain the target index entry corresponding to the target fingerprint value from the N index entries stored in the persistent memory.

[0069] Among them, the target fingerprint value is obtained based on the target index key carried in the target operation request. For example, the target fingerprint value is calculated using the target index key and a preset hash function.

[0070] In this way, in the embodiment of the present application, the target fingerprint value is first obtained through the target index key carried by the target operation request, and it is determined whether there is a fingerprint value in the fingerprint array that is the same as the target fingerprint value. When there is a fingerprint value in the fingerprint array that is the same as the target fingerprint value, based on the mapping relationship between the N fingerprint values in the fingerprint array and the N index entries, the target index entry corresponding to the target fingerprint value is obtained from the N index entries stored in the persistent memory, thereby accelerating the search for the target index entry through the fingerprint array.

[0071] Among them, step 240 may be a sub-step of step 140.

[0072] In step 240, the target operation is the operation corresponding to the target operation request, and the target operation may include, but is not limited to: write operation, read operation, and delete operation. Among them, when there is a target fingerprint value in the fingerprint array, the write operation may be an index entry update operation. Among them, when there is a target fingerprint value in the fingerprint array, the read operation may be a single-point query operation or a range query operation.

[0073] In this way, when processing the target operation request in the embodiment of the present application, the fingerprint array associated with the target index key is directly obtained from the memory, without obtaining the fingerprint array from the persistent memory, reducing the read and write overhead of reading and writing the fingerprint array in the persistent memory, and improving the search performance of the index entry.

[0074] In a specific example, taking the index as a B+ tree index, in order to quickly obtain the required fingerprint array, in the above step 220, based on the target index key carried by the target operation request, determining the fingerprint array associated with the target index key from the metadata stored in the memory may include:

[0075] Based on the target index key carried by the target operation request, determine the target node of the index;

[0076] Based on the mapping relationship between the target node and the fingerprint array, determine the fingerprint array corresponding to the target node from the metadata stored in the memory.

[0077] Among them, in the application scenario where the index is a B+ tree index, the index includes multiple leaf nodes, each leaf node includes multiple index entries, and the index key ranges of each leaf node are different. The fingerprint array of each leaf node is used to describe the multiple index entries in the leaf node.

[0078] In step 220, in the embodiment of the present application, the target node of the index may be first determined from multiple leaf nodes based on the target index key carried by the target operation request, and then the fingerprint array corresponding to the target node is determined.

[0079] Among them, the target node may be a leaf node associated with the target index entry, and the index key of the target index entry may be within the index key range of the target node.

[0080] In this way, the embodiment of the present application can first determine the target node, and then determine the fingerprint array corresponding to the target node, without traversing the fingerprint arrays of multiple leaf nodes to obtain the required fingerprint array from multiple fingerprint arrays, which can accelerate the acquisition of the required fingerprint array.

[0081] In another specific embodiment, in the case where the target fingerprint value does not exist in the fingerprint array, the write operation may be an index entry insertion operation. The target metadata may further include a bitmap, and the bitmap may be used to accelerate the index entry insertion operation.

[0082] In order to reduce the read overhead of reading the fingerprint array and the bitmap in the persistent memory, the operation method of the index provided by the embodiment of the present application may include:

[0083] Step 310: Obtain a target operation request, where the target operation request carries a target index key;

[0084] Step 320: Based on the target index key carried in the target operation request, determine a fingerprint array and a bitmap associated with the target index key from the metadata stored in the memory;

[0085] Step 330: In the case where the target fingerprint value exists in the fingerprint array, based on the mapping relationship between N fingerprint values and N index entries in the fingerprint array, obtain a target index entry corresponding to the target fingerprint value from the N index entries stored in the persistent memory;

[0086] Step 340: In the case where the target fingerprint value does not exist in the fingerprint array, determine an unoccupied index entry based on the bitmap; determine a target index entry from the unoccupied index entries;

[0087] Step 350: Perform a write operation on the target index entry.

[0088] Among them, step 310 may be a sub-step of step 110. In step 310, the target operation request may include a write operation request.

[0089] Among them, step 320 may be a sub-step of step 120. The target metadata may include a fingerprint array and a bitmap, and the fingerprint array and the bitmap are stored in the memory.

[0090] In step 320, based on the target index key carried in the target operation request, determine a fingerprint array associated with the target index key from the metadata stored in the memory.

[0091] In step 320, based on the target index key carried in the target operation request, a bitmap associated with the target index key is determined from the metadata stored in the memory.

[0092] Wherein, in the embodiments of the present application, the fingerprint array and the bitmap associated with the target index key are directly obtained from the memory, without obtaining the fingerprint array and the bitmap from the persistent memory.

[0093] Wherein, step 330 may be a sub-step of step 130. The fingerprint array includes N fingerprint values, and the N fingerprint values in the fingerprint array are used to determine N index entries; N is an integer greater than 1.

[0094] In step 330, a target fingerprint value is obtained based on the target index key carried in the target operation request. When the target fingerprint value exists in the fingerprint array, based on the mapping relationship between the N fingerprint values and the N index entries in the fingerprint array, the target index entry corresponding to the target fingerprint value is obtained from the N index entries stored in the persistent memory.

[0095] Wherein, the target fingerprint value is obtained based on the target index key carried in the target operation request. For example, the target fingerprint value is calculated by using the target index key and a preset hash function.

[0096] Wherein, obtaining the target index entry corresponding to the target fingerprint value from the N index entries stored in the persistent memory can be understood as obtaining the index entry to be updated. The subsequent write operation can be understood as an index entry update operation.

[0097] Wherein, step 340 may be a sub-step of step 130. The bitmap is used to indicate whether the index entry is occupied. In step 340, when the target fingerprint value does not exist in the fingerprint array, the unoccupied index entry is determined based on the bitmap; the target index entry is determined from the unoccupied index entries.

[0098] Wherein, determining the target index entry from the unoccupied index entries can be understood as inserting a target index entry, and the subsequent write operation can be understood as an index entry insertion operation.

[0099] Wherein, step 350 may be a sub-step of step 140. In step 350, the target operation is a write operation corresponding to the write operation request. Wherein, when the target fingerprint value exists in the fingerprint array, the write operation may be an index entry update operation. When the target fingerprint value does not exist in the fingerprint array, the write operation may be an index entry insertion operation.

[0100] In this way, in the embodiments of the present application, when processing the write operation request, the fingerprint array and the bitmap associated with the target index key are directly obtained from the memory, without obtaining the fingerprint array and the bitmap from the persistent memory, reducing the read and write overhead of reading and writing the fingerprint array and the bitmap in the persistent memory, and improving the search performance of the index entry.

[0101] To ensure crash consistency, in the case where metadata is lost due to a crash failure, the embodiments of the present application can also reconstruct the metadata based on the index entries in the persistent memory, without synchronizing the persistent metadata, reducing the additional persistent overhead. The following is an example:

[0102] In a specific embodiment, such as Figure 4-1 shown, the embodiments of the present application provide an operation method for an index, and the operation method for the index may include:

[0103] Step 410: Obtain a target operation request;

[0104] Step 420: Based on the target operation request, obtain target metadata from the metadata stored in the memory;

[0105] Step 430: Based on the target metadata, obtain a target index entry from the index entries stored in the persistent memory;

[0106] Step 440: Perform a target operation on the target index entry;

[0107] Step 450: In the case where the target metadata is lost, based on the association relationship between the target metadata and M index entries, obtain the M index entries from the persistent memory;

[0108] Step 460: Based on the M index entries, reconstruct the target metadata in the memory.

[0109] Among them, the specific content of steps 410-step 440 can refer to the specific content of steps 110-step 140, which will not be elaborated here.

[0110] Among them, in steps 450-step 460, there are at least M index entries, and the target metadata is associated with the M index entries; M is an integer greater than 1;

[0111] In the case where the target metadata is lost, based on the association relationship between the target metadata and M index entries, obtain the M index entries from the persistent memory; based on the M index entries, reconstruct the target metadata in the memory.

[0112] In this way, the embodiments of the present application can reconstruct the target metadata in the memory based on the index entries in the persistent memory, ensuring the consistency between the index entries in the persistent memory and the target metadata in the memory.

[0113] Among them, in a specific example, in the case where the target metadata includes a fingerprint array, the fingerprint array can be reconstructed in the memory based on the index entries in the persistent memory. For example, such as Figure 4-2As shown in the figure, in the above step 460, reconstructing the target metadata in the memory based on the M index entries may include:

[0114] Step 4601: Obtain M index keys of the M index entries from the persistent memory;

[0115] Step 4602: Determine M fingerprint values based on the M index keys of the M index entries;

[0116] Step 4603: Reconstruct a fingerprint array in the memory based on the M fingerprint values.

[0117] Wherein, there are at least M index entries, and the fingerprint array is associated with the M index entries; M is an integer greater than 1.

[0118] Wherein, each of the M index entries includes an index key.

[0119] Wherein, for each index key, a fingerprint value can be calculated by using the index key and a preset hash function. Furthermore, M fingerprint values are determined based on the M index keys of the M index entries. A fingerprint array is reconstructed in the memory based on the M fingerprint values. Wherein, the M fingerprint values in the fingerprint array are used to determine the M index entries.

[0120] In this way, in the scenario where the fingerprint array is lost due to a crash failure, the embodiment of the present application can reconstruct the fingerprint array in the memory based on the index entries in the persistent memory, ensuring the consistency between the index entries in the persistent memory and the fingerprint array in the memory.

[0121] In a specific example, in order to accurately reconstruct the fingerprint array, the embodiment of the present application can set a verification field in the index entry, and determine whether the index entry is valid (for example, determine whether the index entry is successfully persisted) through the verification field, and then accurately reconstruct the fingerprint array.

[0122] For example, in the above step 4601, obtaining the M index keys of the M index entries from the persistent memory may include:

[0123] When there is persistent verification information in the verification field of each of the M index entries, obtain the M index keys of the M index entries from the persistent memory.

[0124] Wherein, each of the M index entries further includes a verification field.

[0125] For each of the M index entries, when there is persistent verification information in the verification field of the index entry, it indicates that the index entry is valid, and the index key of the index entry can be obtained for the reconstruction of the fingerprint value; when there is no persistent verification information in the verification field of the index entry, it indicates that the index entry is invalid, and there is no need to obtain the index key of the index entry for the reconstruction of the fingerprint value.

[0126] Based on this, when there is persistent verification information in the verification field of each of the M index entries, the M index keys of the M index entries can be obtained from the persistent memory for the subsequent reconstruction of the M fingerprint values.

[0127] In other embodiments, when there is persistent verification information in the verification fields of K (K is less than M) index entries among the M index entries, the K index keys of the K index entries can be obtained from the persistent memory for the subsequent reconstruction of the K fingerprint values. Based on the K index keys of the K index entries, K fingerprint values are determined; based on the K fingerprint values, a fingerprint array is reconstructed in the memory.

[0128] In this way, in the scenario where the fingerprint array is lost due to a crash failure, the embodiments of the present application can determine whether an index entry is valid based on the verification field of the index entry, and based on the valid index entries in the persistent memory, reconstruct the fingerprint array in the memory, thereby accurately reconstructing the fingerprint array and ensuring the consistency between the valid index entries in the persistent memory and the fingerprint array in the memory.

[0129] In another specific embodiment, when the target metadata further includes a bitmap, the bitmap can be reconstructed in the memory based on the valid index entries in the persistent memory. For example, Figure 4-3 As shown, in step 460 above, reconstructing the target metadata in the memory based on the M index entries may include:

[0130] Step 4604: For the i-th index entry among the M index entries, when there is persistent verification information in the verification field of the i-th index entry, set the i-th bit corresponding to the i-th index entry in the bitmap to a first value, where the first value indicates that the i-th index entry is occupied;

[0131] Step 4605: When there is no persistent verification information in the verification field of the i-th index entry, set the i-th bit corresponding to the i-th index entry in the bitmap to a second value, where the second value indicates that the i-th index entry is not occupied; i is a positive integer less than or equal to M;

[0132] Step 4606: Based on the M index entries, obtain the M bits in the bitmap.

[0133] Among them, there are at least M index entries, and the bitmap is associated with the M index entries; M is an integer greater than 1.

[0134] Among them, each of the M index entries includes a verification field; there are M bits in the bitmap, and the M bits are used to indicate whether the M index entries are occupied.

[0135] For example, the first value can be "1" and the second value can be "0". For the i-th index entry among the M index entries, when there is persistent verification information in the verification field of the i-th index entry, it indicates that the i-th index entry is valid (occupied), and the i-th bit among the M bits of the bitmap is set to "1", and "1" indicates that the i-th index entry is occupied.

[0136] Or, when there is no persistent verification information in the verification field of the i-th index entry, it indicates that the i-th index entry is invalid (not occupied), and the i-th bit among the M bits of the bitmap is set to "0", and "0" indicates that the i-th index entry is not occupied.

[0137] And so on, based on the verification fields of the M index entries, the M bits corresponding to the M index entries are set to achieve the reconstruction of the bitmap.

[0138] When the target metadata further includes a bitmap, the bitmap can be reconstructed in the memory based on the valid index entries in the persistent memory.

[0139] In this way, in the scenario where the bitmap is lost due to a crash failure, the embodiments of the present application can determine whether an index entry is occupied based on the verification field of the index entry, reconstruct the bitmap in the memory, and then accurately reconstruct the bitmap to ensure the consistency between the valid index entries in the persistent memory and the bitmap in the memory.

[0140] It should be noted that, in order to ensure that the verification field can accurately determine whether an index entry is successfully persisted, the target index entry can include a verification field and a data field; the verification field and the data field are written through an atomic write method.

[0141] In this way, since the embodiments of the present application can write the verification field and the data field as a whole at one time through the atomic write method, and the verification field and the data field will not be partially written, the reliability and accuracy of the verification field and the data field can be ensured in application scenarios such as power failure, system crash, and normal shutdown.

[0142] For example, when the target operation request includes a write operation request and the target operation includes a write operation, in step 140 above, performing the target operation on the target index entry may include:

[0143] Write the target index key carried in the write operation request into the index key of the target index entry in an atomic write manner;

[0144] Write the set persistent verification information into the verification field of the target index entry and write the target data carried in the write operation request into the data field of the target index entry in an atomic write manner.

[0145] For example, the target index entry may include an index key, a verification field, and a data field. The index key is 64 bits, the verification field is 16 bits, the data field is 48 bits, and the atomic write size is 64 bits.

[0146] In the embodiments of the present application, the target index key carried in the write operation request can be written into the 64-bit index key in an atomic write manner of 64-bit size.

[0147] Moreover, in the embodiments of the present application, the set persistent verification information can be written into the 16-bit verification field and the target data carried in the write operation request can be written into the 48-bit data field in an atomic write manner of 64-bit size.

[0148] Among them, for example, the persistent verification information can be 16 bits all "1".

[0149] In this way, during the process of performing a write operation on the target index entry, the verification field and the data field are written as a whole at one time in an atomic write manner, ensuring the reliability and accuracy of the written verification information and the target data.

[0150] For another example, in the case where the target operation request includes a deletion request and the target operation includes a deletion operation, in step 140 above, performing the target operation on the target index entry may include:

[0151] Write the marking information into the verification field of the target index entry in an atomic write manner, where the marking information is used to indicate that the target index entry is in a deleted state.

[0152] For example, the target index entry may include an index key, a verification field, and a data field. The index key is 64 bits, the verification field is 16 bits, the data field is 48 bits, and the atomic write size is 64 bits.

[0153] In the embodiments of the present application, the marking information can be written into the 16-bit verification field in an atomic write manner of 64-bit size. Among them, the marking information is different from the persistent verification information. For example, the marking information can be 16 bits all "0".

[0154] In this way, during the process of performing a deletion operation on a target index entry, instead of immediately deleting the key-value pair of the target index entry and reclaiming space, a mark deletion method can be adopted. The mark information is written into the verification field of the target index entry in an atomic write manner to mark and delete the target index entry.

[0155] In addition, there is a need to update metadata after an index entry insertion operation or deletion operation. During the process of updating metadata, the present application can also reduce the write overhead of updating metadata in persistent memory. In a specific embodiment, in order to reduce the write overhead of updating metadata in persistent memory during the process of updating metadata, as Figure 5 shown, the operation method of the index provided by the embodiment of the present application may include:

[0156] Step 510: Obtain a target operation request;

[0157] Step 520: Based on the target operation request, obtain target metadata from the metadata stored in memory;

[0158] Step 530: Based on the target metadata, obtain a target index entry from the index entries stored in persistent memory;

[0159] Step 540: Perform a target operation on the target index entry;

[0160] Step 550: After performing the target operation on the target index entry, update the target metadata; the target operation includes a deletion operation or a write operation when the target index entry is an unoccupied index entry.

[0161] Among them, the specific content of steps 510 - 540 can refer to the specific content of steps 110 - 140, which will not be elaborated here.

[0162] In step 550, the write operation when the target index entry is an unoccupied index entry can be understood as an index entry insertion operation. After performing the index entry insertion operation, both the bitmap and fingerprint of the target index entry need to be updated.

[0163] In step 550, after performing the deletion operation on the target index entry, both the bitmap and fingerprint of the target index entry need to be updated.

[0164] In this way, after performing the index entry insertion operation or deletion operation, the embodiment of the present application updates the target metadata. Compared with the related art in which the metadata of the index entry is stored in persistent memory, since the target metadata in the present application is stored in memory, the target metadata can be directly updated in memory without writing new target metadata in persistent memory, reducing the write overhead of updating the target metadata in persistent memory and improving the search performance of the index entry.

[0165] In a specific embodiment, after performing the index item insertion operation, in the above step 550, updating the target metadata may include:

[0166] After performing a write operation on an unoccupied index item, based on the mapping relationship between the M bit positions and the M index items, modifying the value of a first target bit position in the bitmap to a first value; the first target bit position is the bit position in the bitmap corresponding to the target index item, and the first value indicates that the target index item has been occupied;

[0167] Adding a target fingerprint value for the target index item to the fingerprint array.

[0168] Wherein, the target metadata includes a bitmap and a fingerprint array; there are M bit positions in the bitmap, and the M bit positions are used to indicate whether M index items located on the persistent memory are occupied, and M is an integer greater than 1; the target operation includes a write operation on an unoccupied index item, and the target index item is an unoccupied index item among the M index items.

[0169] For example, in the case where the target operation includes a write operation on an unoccupied index item, modifying the bit position in the bitmap corresponding to the target index item from "0" to "1", and "1" indicates that the target index item has been occupied. And, in the case where the target operation includes a write operation on an unoccupied index item, adding a target fingerprint value for the target index item to the fingerprint array, and the target fingerprint value can be obtained based on the target index key of the target index item.

[0170] In this way, after the embodiment of the present application performs the index item insertion operation, the bitmap and the fingerprint array are updated. Compared with the related art in which the metadata of the index item is stored in the persistent memory, since the bitmap and the fingerprint array in the present application are stored in the memory, the bitmap and the fingerprint array can be directly updated in the memory without updating the bitmap and the fingerprint array in the persistent memory, reducing the write overhead of updating the bitmap and the fingerprint array in the persistent memory and improving the insertion performance of the index item.

[0171] In another specific embodiment, after performing the index item deletion operation, in the above step 550, updating the target metadata may include:

[0172] After performing a write operation on an unoccupied index item, based on the mapping relationship between the M bit positions and the M index items, modifying the value of a first target bit position in the bitmap to a first value; the first target bit position is the bit position in the bitmap corresponding to the target index item, and the first value indicates that the target index item has been occupied;

[0173] Add a target fingerprint value for the target index entry in the fingerprint array.

[0174] Among them, the target metadata includes a bitmap and a fingerprint array; there are M bit positions in the bitmap, and the M bit positions are used to indicate whether M index entries located on the persistent memory are occupied, and M is an integer greater than 1; the target operation includes a write operation on the unoccupied index entry, and the target index entry is an unoccupied index entry among the M index entries.

[0175] For example, in the case where the target operation includes a deletion operation, change the bit position corresponding to the target index entry in the bitmap from "1" to "0", where "0" indicates that the target index entry is unoccupied, and delete the target fingerprint value of the target index entry in the fingerprint array.

[0176] In this way, after the deletion operation of the index entry is executed in the embodiment of the present application, the bitmap and the fingerprint array are updated. Compared with the related technology in which the metadata of the index entry is stored in the persistent memory, since the bitmap and the fingerprint array in the present application are stored in the memory, the bitmap and the fingerprint array can be directly updated in the memory without updating the bitmap and the fingerprint array in the persistent memory, reducing the write overhead of updating the bitmap and the fingerprint array in the persistent memory and improving the search performance of the index entry.

[0177] In practical applications, taking the B+ tree index as an example, the operation method of the index provided by the embodiment of the present application is described by way of example.

[0178] As Figure 6-1 shown, the structure of the B+ tree index includes internal nodes and leaf nodes. Among them, the internal nodes include root nodes and intermediate nodes, which are stored in the memory, for example, stored in DRAM (Dynamic Random Access Memory). The leaf nodes are stored in the persistent memory (PMEM).

[0179] In the embodiment of the present application, the size of each leaf node can be fixed at 256B, so as to adapt to the characteristic that the internal data transfer size of PMEM is 256B, give full play to the bandwidth of PMEM, and reduce the performance loss caused by read / write amplification.

[0180] Among them, for a leaf node (such as the target node to be operated), M index entries are stored in the persistent memory, and the target metadata of the M index entries is transferred from the PMEM to the memory for storage.

[0181] Among them, the first 16B of the leaf node is the target metadata, and the embodiment of the present application transfers the target metadata to the memory for storage.

[0182] Among them, the target metadata may include a fingerprint array. The fingerprint array includes multiple fingerprint values, each of which may occupy 1B of space. Each fingerprint value can be calculated from an 8B index key through a hash function. Each target operation first compares the target fingerprint value calculated based on the target index key with the fingerprint array. When there is the same fingerprint value in the fingerprint array, the specific record information is compared. It can be seen that the fingerprint array in the memory improves the query performance of the index.

[0183] Among them, in addition to the fingerprint array, the target metadata may also include a bitmap, which is used to record whether an index entry is occupied. "1" indicates that the corresponding index entry is occupied, and "0" indicates that the corresponding index entry is not occupied (is free). After the index entry is persisted, the bit corresponding to the index entry in the bitmap can be modified to "1" in an atomic write manner, indicating that the corresponding index entry is occupied. When a failure is recovered, data consistency can be ensured by checking the bitmap.

[0184] Among them, the embodiment of the present application may adopt a metadata management strategy of lazy persistence. The metadata of the index entries of the leaf nodes (such as bitmaps and fingerprints) are all stored in the memory and do not need to be persisted, and can be recovered after a power failure.

[0185] In order to be able to recover the bitmap and fingerprints in the memory, the embodiment of the present application makes a special design for the value pointer of the index entry. For example, modern 64-bit processors actually do not use the upper 16 bits, and the upper 16 bits are generally filled with all 0s, and only the lower 48 bits are used to represent the memory address space, and the maximum addressable space is 256TB. Refer to Figure 6-1 , and the embodiment of the present application further decomposes the 8B (8 * 8bit = 64bit) value pointer into a 16bit valid field and a 48bit pointer field. Also, since the atomic write size supported by PMEM is 8B, persistent instructions such as the clwb instruction and the sfence instruction can be used to persist the data to PMEM. PMEM supports atomic writes of 8B data and there will be no situation of partial data writing. When a failure occurs, what may be stored in PMEM is either the new 8B word or the original 8B word, and 8B data cannot be partially written. This is the PMEM atomic write (NVM Atomic Write, NAW) mechanism. Using the NAW mechanism, the embodiment of the present application can determine whether the index entry has been successfully persisted by determining whether the verification field (i.e., the first 16 bits of the value pointer) is all 1. When a failure is recovered, after scanning the index entries, the bitmap and fingerprints can be easily reconstructed, so there is no need for additional persistence overhead, and at the same time, it provides crash consistency guarantee.

[0186] Generally speaking, the embodiments of the present application maintain the metadata of index entries (such as bitmaps and fingerprints) in memory, which can avoid reading and writing metadata in PMEM, thereby reducing the read and write overhead of PMEM and improving the lookup performance of index entries.

[0187] The embodiments of the present application regard the operations of reading / writing index entries as one operation request. The specific request types may include write requests (updating index entries or inserting index entries), read requests (single-point query requests, range query requests), and delete requests. When the B+ tree index of the embodiments of the present application processes different types of requests, it can be implemented through different processes, which will be illustrated by examples below.

[0188] Figure 6-2 FIG. is a schematic flowchart of a write operation of a B+ tree index provided by an embodiment of the present application.

[0189] When the B+ tree index is initialized, the leaf nodes are all reset to 0, indicating that no index entries are inserted. As Figure 6-2 shown, the following is the write operation processing flow of the B+ tree index provided by the embodiment of the present application:

[0190] First, obtain the write request input by the user. The write request carries the target index key;

[0191] Then, calculate the target fingerprint value using the target index key; and determine whether there is a fingerprint value in the fingerprint array that is the same as the target fingerprint value; if it exists, the original index entry can be updated, and if it does not exist, a new index entry is inserted.

[0192] When there is no fingerprint value in the fingerprint array that is the same as the target fingerprint value, an index entry can be inserted into the leaf node. Specifically, the free position (i.e., the unoccupied index entry) can be obtained by querying the bitmap first. After obtaining the insertable position, an index entry is inserted into the leaf node. Then, the key-value pair (index key and index value) is written into the index entry and the persistent verification field; finally, the bitmap and fingerprint are modified.

[0193] Among them, in the embodiments of the present application, the value of the index entry is a value pointer pointing to the PMEM data. The first 16 bits of the value pointer are the verification field, and the last 48 bits are the PMEM address pointing to the data. To ensure the atomicity of index entry insertion, this solution uses the NAW atomic write mechanism. The verification field of the inserted index entry will be set to 1 because 8B bytes will not be partially written, and the value pointer will definitely be atomically written. By determining whether the verification field is all 1, it can be known whether this write operation is successful. After the index entry is successfully written, the embodiments of the present application can also update the bitmap and fingerprint in memory. The specific update operation is to modify the corresponding bit in the bitmap to "1" for this index entry and add the target fingerprint value corresponding to this index entry to the fingerprint array.

[0194] In the case where there is a fingerprint value identical to the target fingerprint value in the fingerprint array, the index entry position can be determined in the leaf node, and key-value pairs (index key and index value) can be written in the index entry and the verification field can be persisted.

[0195] In this way, in the embodiments of the present application, the bitmap of the index entry and the fingerprint array are stored in the memory, rather than storing the bitmap of the index entry and the fingerprint array in the persistent memory. When processing a write request, the bitmap and the fingerprint array are directly obtained from the metadata stored in the memory, without obtaining the bitmap and the fingerprint array from the persistent memory, reducing the read overhead of reading the bitmap and the fingerprint array in the persistent memory and improving the search performance of the index entry. Moreover, after performing a write operation on the newly inserted index entry, the bitmap and the fingerprint array are directly modified from the metadata stored in the memory, without modifying the bitmap and the fingerprint array in the persistent memory, reducing the write overhead of updating the bitmap and the fingerprint array in the persistent memory and improving the search performance of the index entry.

[0196] Figure 6-3 FIG. is a schematic flowchart of a single-point query operation of a B+ tree index provided by an embodiment of the present application.

[0197] In a single-point query, fingerprints are mainly used to accelerate data query. As Figure 6-3 shown, FIG. is a single-point query processing flow of a B+ tree index provided by an embodiment of the present application.

[0198] As Figure 6-3 shown, first, a single-point query request input by the user is obtained, and the single-point query request carries a target index key; then, the target fingerprint value is calculated using the target index key; and it is determined whether there is a fingerprint value identical to the target fingerprint value in the fingerprint array; if there is no fingerprint value identical to the target fingerprint value in the fingerprint array, it indicates that there is no key-value pair to be queried in the leaf node, and the returned result is non-existent; if there is a fingerprint value identical to the target fingerprint value in the fingerprint array, it indicates that the queried key value exists. Since the order of the fingerprint array is consistent with the position of the index entry in the leaf node, the subscript of the fingerprint array is the position of the key-value pair in the leaf node. At this time, the key-value pair data can be read and returned according to the position of the key value in the leaf node.

[0199] In this way, in the embodiments of the present application, the fingerprint array of the index entry is stored in the memory, rather than storing the fingerprint array of the index entry in the persistent memory. When processing a single-point query request, the fingerprint array is directly obtained from the metadata stored in the memory, without obtaining the fingerprint array from the persistent memory, reducing the read overhead of reading the fingerprint array in the persistent memory and improving the search performance of the index entry.

[0200] Figure 6-4 FIG. is a schematic flowchart of a range query operation of a B+ tree index provided by an embodiment of the present application.

[0201] As Figure 6-4 shown, first, obtain the range query request input by the user. The range query request carries an index key range, and the index key range includes the index key of the left boundary and the index key of the right boundary; in a range query, it is necessary to clarify the left and right boundary values of the range query first. When querying, first determine the position of the index key of the left boundary in the leaf node. If the index key of the left boundary exists in the index, use the index key of the left boundary as the starting index key for the range query and determine the position of the starting index key in the leaf node. Otherwise, if the index key of the left boundary does not exist in the index, continue to search backward from this position until the first index key not less than the index key of the left boundary is found, and use this index key as the starting index key for the range query and determine the position of the starting index key in the leaf node.

[0202] Then, starting from the position of the starting index key, traverse and access all subsequent positions, add the data to the result set, and continue to search until the index key found is greater than the index key of the right boundary or after traversing all positions, then return the result set of the range query. Determine whether the result set is an empty set. If the result set is an empty set, it means that this range does not exist in the index, and the returned result is empty.

[0203] In this way, in the embodiment of the present application, the fingerprint array of the index item is stored in the memory, rather than storing the fingerprint array of the index item in the persistent memory. When processing the range query request, directly obtain the fingerprint array from the metadata stored in the memory, without obtaining the fingerprint array from the persistent memory, reducing the read overhead of reading the fingerprint array in the persistent memory and improving the search performance of the index item.

[0204] Figure 6-5 FIG. is a schematic flowchart of a deletion operation of a B+ tree index provided by an embodiment of the present application.

[0205] As Figure 6-5 shown, FIG. is a processing flow of a deletion operation of a B+ tree index provided by an embodiment of the present application.

[0206] First, obtain the deletion request input by the user. The deletion request carries the target index key; then, calculate the target fingerprint value using the target index key; and determine whether there is a fingerprint value in the fingerprint array that is the same as the target fingerprint value; if there is no fingerprint value in the fingerprint array that is the same as the target fingerprint value, it means that there is no key-value pair to be deleted in the leaf node, and the returned result is non-existent; if there is a fingerprint value in the fingerprint array that is the same as the target fingerprint value, it means that the index item to be deleted exists. Since the order of the fingerprint array is the same as the position of the index item in the leaf node, the subscript of the fingerprint array is the position of the key-value pair in the leaf node. The present application finds the position of the index item in the leaf node and deletes the key-value data according to the position of the index item in the leaf node.

[0207] Among them, the specific deletion operation can be carried out by means of marked deletion, and the key-value recycling space will not be immediately deleted. The specific method is to set the verification field of the value pointer to 0, and then use the NAW atomic write mechanism to write back the new index pointer to mark and delete the index entry.

[0208] Finally, after the deletion operation is executed, the bitmap and fingerprint are modified. The specific modification operation is to modify the bit corresponding to the index entry in the bitmap to "0", and delete the target fingerprint value corresponding to the index entry in the fingerprint array.

[0209] In this way, in the embodiment of the present application, the bitmap and fingerprint array of the index entry are stored in the memory, rather than storing the bitmap and fingerprint array of the index entry in the persistent memory. When processing the deletion request, the fingerprint array is directly obtained from the metadata stored in the memory, without obtaining the fingerprint array from the persistent memory, reducing the read overhead of reading the fingerprint array in the persistent memory and improving the search performance of the index entry. Moreover, after the deletion operation of the index entry is executed, the bitmap and fingerprint array are directly modified from the metadata stored in the memory, without modifying the bitmap and fingerprint array from the persistent memory, reducing the write overhead of updating the bitmap and fingerprint array in the persistent memory and improving the search performance of the index entry.

[0210] Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0211] As Figure 7 shown, an embodiment of the present application further provides an electronic device 700, including a processor 701 and a memory 702. The memory 702 stores a program or instruction that can run on the processor 701. When the program or instruction is executed by the processor 701, each step of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described in detail here.

[0212] It should be noted that the electronic device in the embodiment of the present application includes a memory and PMEM hardware. The memory is used to store index entries, and the PMEM hardware is used to store metadata of index entries.

[0213] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0214] The embodiments of the present application further provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each step of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here.

[0215] Among them, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk, or an optical disc, etc.

[0216] The embodiments of the present application further provide a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run a program or instruction to implement each process of the above method embodiment, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here.

[0217] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-on-chip, a system chip, a chip system, or a system-on-chip, etc.

[0218] The embodiments of the present application provide a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement each process of the above method embodiment, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here.

[0219] Another embodiment of the present application provides a computer program product, including a computer program, which when executed by a processor, implements each process of the method embodiment as described above and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0220] It should be noted that in this article, the terms "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described method may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0221] Through the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better embodiment. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present application.

[0222] The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Those of ordinary skill in the art, under the inspiration of the present application and without departing from the purpose of the present application and the scope protected by the claims, can also make many forms, all of which are within the protection scope of the present application.

Claims

1. A method for operating an index, the index including index entries and metadata of the index entries, characterized in that, The index items are stored in persistent memory, and the metadata is stored in memory. The method includes: Obtaining a target operation request; Based on the target operation request, obtaining target metadata from the metadata stored in the memory; Based on the target metadata, obtaining a target index item from the index items stored in the persistent memory; Performing a target operation on the target index item.

2. The method according to claim 1, wherein The target metadata includes a fingerprint array, and the target operation request carries a target index key; The obtaining target metadata from the metadata stored in the memory based on the target operation request includes: Based on the target index key carried in the target operation request, determining a fingerprint array associated with the target index key from the metadata stored in the memory.

3. The method according to claim 2, characterized in that, The determining a fingerprint array associated with the target index key from the metadata stored in the memory based on the target index key carried in the target operation request includes: Based on the target index key carried in the target operation request, determining a target node of the index; Based on the mapping relationship between the target node and the fingerprint array, determining a fingerprint array corresponding to the target node from the metadata stored in the memory.

4. The method according to claim 1, wherein The target operation request carries a target index key, the target metadata includes a fingerprint array, and N fingerprint values in the fingerprint array are used to determine N index items; N is an integer greater than 1; The obtaining a target index item from the index items stored in the persistent memory based on the target metadata includes: When there is a target fingerprint value in the fingerprint array, based on the mapping relationship between the N fingerprint values and the N index items in the fingerprint array, obtaining a target index item corresponding to the target fingerprint value from the N index items stored in the persistent memory; Wherein, the target fingerprint value is obtained based on the target index key carried in the target operation request.

5. The method according to claim 4, characterized in that, The target metadata further includes a bitmap, and the bitmap is used to indicate whether an index item is occupied; The obtaining a target index item from the index items stored in the persistent memory based on the target metadata further includes: When the target fingerprint value does not exist in the fingerprint array, determining an unoccupied index item based on the bitmap; Determining a target index item from the unoccupied index items.

6. The method according to claim 1, wherein There are M index items, and the target metadata is associated with the M index items; M is an integer greater than 1; The method further includes: When the target metadata is lost, based on the association relationship between the target metadata and the M index items, obtaining the M index items from the persistent memory; Based on the M index items, reconstructing the target metadata in the memory.

7. The method according to claim 6, wherein Each of the M index items includes an index key; the target metadata includes a fingerprint array; The reconstructing the target metadata in the memory based on the M index items includes: Obtaining M index keys of the M index items from the persistent memory; Based on the M index keys of the M index items, determining M fingerprint values; Based on the M fingerprint values, reconstruct the fingerprint array in the memory; wherein, the M fingerprint values in the fingerprint array are used to determine M index entries.

8. The method according to claim 7, characterized in that Each of the M index entries further includes a verification field; The obtaining the M index keys of the M index entries from the persistent memory includes: When there is persistent verification information in the verification field of each of the M index entries, obtaining the M index keys of the M index entries from the persistent memory.

9. The method according to claim 6, characterized in that, Each of the M index entries includes a verification field; the target metadata includes a bitmap; there are M bit positions in the bitmap, and the M bit positions are used to indicate whether the M index entries are occupied; The reconstructing the target metadata in the memory based on the M index entries includes: For the i-th index entry among the M index entries, when there is persistent verification information in the verification field of the i-th index entry, setting the i-th bit position corresponding to the i-th index entry in the bitmap to a first value, and the first value indicates that the i-th index entry is occupied; When there is no persistent verification information in the verification field of the i-th index entry, setting the i-th bit position corresponding to the i-th index entry in the bitmap to a second value, and the second value indicates that the i-th index entry is not occupied; i is a positive integer less than or equal to M; Based on the M index entries, obtaining the M bit positions in the bitmap.

10. The method according to claim 1, characterized in that, The target index entry includes a verification field and a data field; The verification field and the data field are written by an atomic write method.

11. The method according to claim 10, characterized in that, The target index entry further includes an index key; the target operation request includes a write operation request, and the target operation includes a write operation; the performing the target operation on the target index entry includes: By an atomic write method, writing the target index key carried in the write operation request into the index key of the target index entry; By an atomic write method, writing the set persistent verification information into the verification field of the target index entry, and writing the target data carried in the write operation request into the data field of the target index entry.

12. The method according to claim 10, characterized in that The target operation request includes a deletion request; the performing the target operation on the target index entry includes: By an atomic write method, writing marking information into the verification field of the target index entry, and the marking information is used to indicate that the target index entry is in a deleted state.

13. The method according to claim 1, characterized in that The target operation includes a deletion operation or a write operation when the target index entry is an unoccupied index entry; after performing the target operation on the target index entry, the method further includes: Updating the target metadata.

14. The method according to claim 13, characterized in that, The target metadata includes a bitmap and a fingerprint array; there are M bit positions in the bitmap, and the M bit positions are used to indicate whether M index entries located on the persistent memory are occupied, and M is an integer greater than 1; the target operation includes a write operation on an unoccupied index entry, and the target index entry is an unoccupied index entry among the M index entries; The updating the target metadata includes: After performing a write operation on an unoccupied index entry, based on the mapping relationship between the M bit positions and the M index entries, modify the value of a first target bit position in the bitmap to a first value; the first target bit position is the bit position in the bitmap corresponding to the target index entry, and the first value indicates that the target index entry is occupied; Add a target fingerprint value for the target index entry to the fingerprint array.

15. The method according to claim 13, wherein The target metadata includes a bitmap and a fingerprint array; there are M bit positions in the bitmap, and the M bit positions are used to indicate whether M index entries located on the persistent memory are occupied, where M is an integer greater than 1; the target operation includes a delete operation; The updating of the target metadata includes: After performing a delete operation on the target index entry, based on the mapping relationship between the M bit positions and the M index entries, modify the value of a second target bit position in the bitmap to a second value; the second target bit position is the bit position in the bitmap corresponding to the target index entry, and the second value indicates that the target index entry is unoccupied; Delete the target fingerprint value for the target index entry from the fingerprint array.

16. An electronic device, characterized in that, Comprising a processor and a memory, the memory stores a program or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method according to any one of claims 1 to 15 are implemented.

17. A readable storage medium, characterized in that, A program or instructions are stored on the readable storage medium, and when the program or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 15 are implemented.

18. A computer program product, characterized in that, Comprising a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images