A method and apparatus for deleting an index item in memory

Abstract

A method for deleting index entries in memory includes: selecting a target storage unit to be evicted from a plurality of storage units (410); reading all index entries in the target storage unit (420), wherein all index entries in the target storage unit are stored contiguously in the target storage unit; deleting all read index entries from the memory index table (430); and marking the target storage unit as free (440). This method can read index entries corresponding to multiple data cached in memory in a single I / O operation, thereby deleting identical index entries from the memory index table.

Description

Technical Field

[0001] This application relates to the field of storage, and more specifically, to a method and apparatus for deleting index entries in memory. Background Technology

[0002] Software (e.g., a file system) can send instructions to the processor to read and write data from memory. An index table is stored in memory to facilitate looking up data stored in memory.

[0003] If there is insufficient cache space in the memory, the data stored in the memory cache can be deleted, and the index entries in the memory index table can be removed.

[0004] In existing technologies, multiple I / O operations are required to read the index entries corresponding to multiple data cached in memory and delete the same index entries in the memory index table. This results in a large number of memory reads, which affects the overall system performance.

[0005] Therefore, how to reduce the number of memory reads and improve system performance during the process of deleting index entries in the memory index table has become an urgent problem to be solved. Summary of the Invention

[0006] This application provides a method and apparatus for deleting index entries in memory. Since all index entries in the target storage unit are stored contiguously rather than scattered, multiple index entries corresponding to cached data in memory can be read in a single IO operation, and the same index entries can be deleted at once in the memory index table, thus improving deletion efficiency.

[0007] In a first aspect, a method for deleting index entries in memory is provided. This method is applied to a storage manager, which includes memory and communicates with a first memory. The first memory stores multiple storage units, each storage unit including multiple data blocks and an index entry corresponding to each data block. The memory stores an index table, which records the index entries corresponding to the data blocks of the multiple storage units. The method includes: selecting a target storage unit to be evicted from the multiple storage units; reading all index entries in the target storage unit, wherein all index entries in the target storage unit are stored contiguously in the target storage unit; deleting all read index entries from the memory index table; and marking the target storage unit as free.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the storage manager communicates with the second memory, and before deleting all index entries read from the memory index table, the method further includes: storing a plurality of data blocks in the target storage unit to the second memory.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, a starting address and a length are used to read all index entries in the target storage unit at once, wherein the starting address is the starting address of all index entries in the target storage unit, and the length is the total length of all index entries in the target storage unit.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the index table in memory includes multiple members, and the members contain an index entry corresponding to each of the multiple data blocks.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the first memory records information about the plurality of storage cells, the information including the number of storage cells and / or the number of storage cells in an idle state.

[0012] In this embodiment of the application, the information of multiple storage units recorded in the first memory can facilitate the storage manager to manage the storage information.

[0013] Secondly, an apparatus for deleting index entries in memory is provided, applied to a storage manager, the storage manager including memory, the storage manager communicating with a first memory, the first memory recording multiple storage units, each storage unit including multiple data blocks and an index entry corresponding to each data block in the multiple data blocks, the memory storing an index table, the index table recording the index entries corresponding to the data blocks of the multiple storage units, comprising: a selection module, used to select a target storage unit to be evicted from the multiple storage units; a reading module, used to read all index entries in the target storage unit, wherein all index entries in the target storage unit are stored consecutively in the target storage unit; a deletion module, used to delete all read index entries from the memory index table; and a processing module, used to mark the target storage unit as free.

[0014] In conjunction with the second aspect, in some implementations of the second aspect, the storage manager communicates with the second memory, and the apparatus further includes: a storage module for storing multiple data blocks in the target storage unit to the second memory.

[0015] In conjunction with the second aspect, in some implementations of the second aspect, the reading module is specifically used to: read all index entries in the target storage unit at once using a starting address and a length, wherein the starting address is the starting address of all index entries in the target storage unit, and the length is the total length of all index entries in the target storage unit.

[0016] In conjunction with the second aspect, in some implementations of the second aspect, the index table in memory includes multiple members, each member containing an index entry corresponding to each of the multiple data blocks.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, the first memory records information about the plurality of storage cells, the information including the number of storage cells and / or the number of storage cells in an idle state.

[0018] In this embodiment of the application, the information of multiple storage units recorded in the first memory can facilitate the storage manager to manage the storage information.

[0019] Thirdly, a storage manager is provided, comprising a processor, memory, and a first memory. The first memory stores a plurality of storage units, each storage unit including a plurality of data blocks and an index entry corresponding to each data block. The memory stores an index table, which stores the index entries corresponding to the data blocks of the plurality of storage units. The memory stores a program, and the processor executes the following by running the program: selecting a target storage unit to be evicted from the plurality of storage units; reading all index entries in the target storage unit, wherein all index entries in the target storage unit are stored contiguously in the target storage unit; deleting all read index entries from the memory index table; and marking the target storage unit as free.

[0020] Fourthly, a computer program product is provided, the computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the above aspects.

[0021] Fifthly, a computer-readable medium is provided that stores program code, which, when run on a computer, causes the computer to perform the methods described in the preceding aspects. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a possible storage structure applied to an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of a possible cache management scheme for an SSD 130.

[0024] Figure 3 This is a schematic structural diagram of a system architecture 300 provided in an embodiment of this application.

[0025] Figure 4This is a schematic flowchart illustrating a method for deleting index entries in memory, as provided in an embodiment of this application.

[0026] Figure 5 This is a schematic flowchart illustrating a cache recovery method provided in an embodiment of this application.

[0027] Figure 6 This is a schematic structural diagram of a DRAM 120 provided in an embodiment of this application.

[0028] Figure 7 This is a schematic flowchart illustrating a method for caching write data, as provided in an embodiment of this application.

[0029] Figure 8 This is a schematic structural diagram of a hash table in a DRAM 120 provided in an embodiment of this application.

[0030] Figure 9 This is a schematic flowchart illustrating a method for caching write data, as provided in an embodiment of this application.

[0031] Figure 10 This application provides an embodiment of an apparatus 1000 for deleting index entries in memory. Detailed Implementation

[0032] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0033] Figure 1 This is a schematic diagram of a possible storage structure applied to an embodiment of this application. The storage structure may include: a processor 110, a dynamic random access memory (DRAM) 120, a solid state disk (SSD) 130, and a hard disk drive (HDD) 140.

[0034] DRAM 120 can be used as a level 1 cache, while SSD 130 has read / write latency between DRAM 120 and SSD 130 and can be used as a level 2 cache.

[0035] Software (e.g., a file system) can send commands via processor 110 to read and write data from DRAM 120. SSD 130 can provide read (get), write (put), and delete interfaces for DRAM 120. After the software sends a read request, it can read data from the cache. If no data is found in the DRAM 120 cache, the software can call the get interface to read data from the SSD 130 cache. If no data is found, the software can also read data from HDD 140. After the software sends a write request, it can write data to the DRAM 120 cache. If the DRAM 120 cache is full, the software can call the put interface to write the data in the DRAM 120 cache to SSD 130. If the SSD 130 cache is full, the software can also write the data from SSD 130 to HDD 140.

[0036] The cache management of the SSD 130 can be divided into two parts: the data layout on the SSD 130 and the memory index structure in the DRAM 120. During read or write operations, the software can look up data on the SSD 130 using the memory index in the DRAM 120. The following section combines... Figure 2 This paper provides a detailed description of the specific implementation process of cache management for the SSD 130.

[0037] Figure 2 This is a schematic diagram of a possible cache management scheme for an SSD 130. Figure 2 It may include memory index tables in SSD 130 and DRAM 120.

[0038] See Figure 2 The cache management system can write data (value) to the SSD 130 and create an index table in the DRAM 120. This index table can include multiple index entries, each containing a key and an offset. Data stored on the SSD 130 can be quickly located based on the key or offset in the created index table. Here, the key can be understood as the location of the data value on the HDD 140, and the offset can be understood as the offset of the data value on the SSD 130.

[0039] As an example, when the cache management system writes data (put), it first looks up the index table in DRAM 120 to determine whether the write operation is an update operation or a new data write operation. If an index entry matching the key to be written is found in the index table of DRAM 120, it means that data is already stored on that key in HDD 140, indicating that the write operation is an update operation. If no index entry matching the key to be written is found in the index table of DRAM 120, it means that data is not stored on that key in HDD 140, and the data can be written to SSD 130. Then, the index entry corresponding to the data is added to the index table of DRAM 120.

[0040] As another example, when the cache management system reads data (gets), it first looks up the index table in DRAM 120. If an index entry matching the key of the data to be read is found, the data can be read from SSD 130 based on the offset. If no index entry matching the key is found, the read fails.

[0041] As another example, the cache management system can delete data stored on the SSD 130 when the cache space is insufficient. It can then set the storage space occupied by that data on the SSD 130 to free, allowing the cache management system to write data to that storage space when needed.

[0042] Specifically, the cache management system can read the index entries corresponding to the data stored in SSD 130 and delete the same index entries in DRAM 120. This ensures that when the cache management system writes data, if it does not find an index entry with the same key as the data to be written in the index table of DRAM 120, it can write the data to SSD 130 and then add the corresponding index entry to the index table of DRAM 120.

[0043] In the prior art, in the index table of DRAM 120, each index entry stores the hash value of the key (hereinafter referred to as hashkey) and offset, and the data block containing the key and value (hereinafter referred to as KV pair) can be stored in SSD 130. In the data layout of SSD 130, an index area is allocated on SSD 130 to store the corresponding index table in DRAM 120, and the KV pairs are stored in the data area of ​​SSD 130.

[0044] When the SSD 130 cache space is insufficient, existing technologies require multiple input / output (IO) operations to read the corresponding index entries from the SSD 130's index area into the DRAM 120 during the process of evicting multiple key-value pairs stored in the SSD 130. Then, all read index entries can be traversed to delete the corresponding index entries from the DRAM 120's index table.

[0045] In the existing technology, when the SSD 130 cache space is insufficient, multiple read I / O operations with the SSD 130 are required before multiple corresponding index entries in the DRAM 120 index table can be deleted, thus affecting its deletion performance.

[0046] This application provides a method for deleting index entries in memory. The method is applied to a storage manager, which includes memory and communicates with a first memory. This method can delete multiple corresponding index entries in the index table of DRAM 120 after a single read I / O, thereby improving performance.

[0047] The following combination Figure 3 Taking SSD 130 as the first memory as an example, the system architecture provided in this application will be described in detail.

[0048] Figure 3 This is a schematic structural diagram of a system architecture 300 provided in an embodiment of this application. The system architecture 300 may include DRAM 120 and SSD 130.

[0049] DRAM 120: Can be used as a level 1 cache and can communicate with SSD 130. DRAM 120 can store an index table, which records the index entries corresponding to data blocks (corresponding to KV pairs above) of multiple storage units in SSD 130. A storage unit is a segment of data space that stores multiple data items according to certain rules.

[0050] SSD 130: can be divided into a superblock area of ​​310 and a data area of ​​320.

[0051] Data area 320: Can be managed in fixed granularity. As an example, data stored in SSD 130 can be managed in slabs as storage units. For example, slab 321, slab 322, slab 323, and slab 324. Each slab can store management information (e.g., ...). Figure 3 In the middle, head 325 is stored in slab 322), and multiple consecutive index entries are stored in a centralized manner (e.g. Figure 3In slab 322, index entries 326-327 are stored, along with multiple key-value pairs (e.g., ...). Figure 3 In slab 322, KV pairs 328-329 are stored.

[0052] It should be understood that head 325 can record necessary management information, such as the slab number.

[0053] Superblock 310: Can record relevant information about SSD 130, which may include, but is not limited to: the total number of slabs in data area 320, the number of free slabs in data area 320, and the number of full slabs in data area 320.

[0054] It should be noted that an empty slab can be understood as one in which all key-value pairs are not stored. A full slab can be understood as one in which all key-value pairs are stored.

[0055] Figure 4 This is a schematic flowchart illustrating a method for deleting index entries in memory, as provided in an embodiment of this application. Figure 4 The method shown may include steps 410-440, which are described in detail below.

[0056] Step 410: Select the target storage unit to be eliminated from the plurality of storage units.

[0057] In this embodiment of the application, when the number of free storage units in the SSD 130 is less than a certain preset threshold, the storage unit replacement process can be triggered.

[0058] This application does not specifically limit the implementation method of selecting the target storage unit to be evicted from multiple storage units. As an example, the Least Recently Used (LRU) algorithm can be used to remove data blocks that are in SSD 130 but are not frequently used, so that the memory occupied by the data block can be reused to load other data blocks.

[0059] Step 420: Read all index entries in the target storage unit.

[0060] See Figure 3In this embodiment, the index entries corresponding to all data blocks in the storage unit are stored consecutively within that storage unit. This embodiment allows for the reading of all index entries in a storage unit at once. As an example, in a read request, all index entries stored in a storage unit can be read at once based on the starting address and the length. The starting address can be the address of the first index entry in the storage unit, and the length can be the total length of all index entries stored in the storage unit.

[0061] Step 430: Delete all index entries read from the memory index table.

[0062] In this embodiment, after reading all the index entries stored in the storage unit at once, all the index entries can be traversed, and the corresponding index entries in the memory DRAM 120 can be deleted.

[0063] Step 440: Mark the target storage unit as free.

[0064] In this embodiment, after deleting the corresponding index entry in the DRAM 120, the target storage cell can be marked as free. When a write request is received, the data can be written to the target storage cell, and the index entry corresponding to the written data can be added to the index table of the DRAM 120.

[0065] This application does not limit the specific method for evicting data blocks stored in the target storage unit. As an example, the data can be deleted. As another example, the data can be stored in HDD 140 and then deleted. In this example, the first memory is used as a secondary cache, so the data evicted from SSD 130 (first memory) needs to be persistently stored in HDD 140 (second memory). It should be noted that the second memory is not mandatory; for example, the SSD (first memory) can be used as persistent storage instead of as a cache.

[0066] In this embodiment, since the indexes corresponding to data blocks are stored contiguously in a single storage unit, the index entries corresponding to each data block in all data blocks can be read out with a single I / O operation. This allows multiple data blocks to be evicted with a single I / O, reducing the number of SSD reads and improving system performance.

[0067] Optionally, in some embodiments, the index table cached in DRAM 120 is lost after a node experiences an abnormal power outage or normal startup. The SSD cache management system can trigger a recovery process, which requires restoring the cached index entries in SSD 130 to the index table in DRAM 120. The following describes... Figure 5This section provides a detailed description of the specific implementation process of the cache management system's recovery procedure.

[0068] Figure 5 This is a schematic flowchart illustrating a cache recovery method provided in an embodiment of this application. Figure 5 The method shown may include steps 510-560, which are described in detail below.

[0069] Step 510: Read superblock 310 to obtain the slab identifier in the SSD130 that needs to be recovered.

[0070] The SSD cache management system can trigger a cache recovery thread after a node experiences an abnormal power outage or a normal startup. The recovery thread can read... Figure 3 In the superblock 310, obtain the identifiers of the slabs in all SSDs 130 that need to be recovered.

[0071] It should be understood that the indexes that need to be recovered in DRAM 120 are the indexes corresponding to the KV pairs stored in SSD 130. Therefore, the identifier of a full slab in SSD 130 can be determined using the information recorded in superblock 310.

[0072] Step 520: Read the index entries in the slab of SSD 130.

[0073] The recovery thread can read index entries within the slab of the SSD 130, for example, it can read... Figure 3 In the middle, index entries 326-327 are stored in slab 322.

[0074] Step 530: Traverse all the index entries read and insert them into the index table in DRAM 120.

[0075] After the recovery thread reads the index entries from the slab in SSD 130 into DRAM 120, it can insert the read index entries into the index table of DRAM 120. For example, index entries 326-327 stored in slab 322 can be inserted into the index table of DRAM 120. This will be discussed in conjunction with... Figure 8-9 The specific implementation of inserting index entries in DRAM 120 will be described in detail here, and will not be repeated here.

[0076] Step 540: Determine if the index entry for the last slab has been read.

[0077] The recovery thread can repeat steps 520-530 above until all slabs in all SSD 130s have completed the recovery of index entries.

[0078] If the index entry in the last slab that needs to be recovered has not yet been read, you can continue to step 520 to read the index entry in the slab of SSD 130.

[0079] If the index entry of the last slab that needs to be recovered has been read, step 550 can be executed.

[0080] Step 550: End.

[0081] In this embodiment, during cache recovery, only one IO operation is needed to read multiple consecutively stored index entries in the slab, and these multiple index entries can be restored to the memory DRAM 120. Since it is not necessary to read the entire slab in the SSD 130, fast recovery can be achieved.

[0082] Optionally, in some embodiments, during the process of writing cached data, the aggregated data can be written to the SSD 130 after being aggregated in the DRAM 120, which can reduce the garbage collection overhead inside the SSD 130.

[0083] For details, please see Figure 6 In this embodiment, a cache segment can be reserved in DRAM 120, and this cache can be managed with a fixed granularity. As an example, data stored in DRAM 120 can be managed using slabs as storage units, such as slab 610, slab 620, slab 630, and slab 640. After a storage unit in DRAM 120 is full, the data in that storage unit can be written to a storage unit in SSD 130 all at once. For example, after slab 620 in DRAM 120 is full, the data stored in slab 620 can be written to slab 322 in SSD 130 all at once.

[0084] It should be understood that the slab status in SSD 130 can be divided into: free slab (no KV pairs are stored in any KV space in this slab of SSD 130), full slab (all KV pairs are stored in any KV space in this slab of SSD 130), and partially full slab (new KV pairs can still be written to this slab in SSD 130).

[0085] It should be noted that the slab data structure in DRAM 120 is the same as that in SSD 130. Each slab can store management information (e.g., Figure 3 In the middle, head 325 is stored in slab 322), and multiple consecutive index entries are stored in a centralized manner (e.g. Figure 3 In slab 322, index entries 326-327 are stored, along with multiple key-value pairs (e.g., ...). Figure 3 In this embodiment, after the slab cache in DRAM 120 is full, all the data stored in the slab in DRAM 120 can be written to the slab in SSD 130.

[0086] The following combination Figure 7 This application provides a more detailed description of the specific implementation process of cached data writing in the embodiments of this application. It should be noted that... Figure 7 The examples provided are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific numerical values ​​or specific scenarios illustrated. Figure 7 The examples are obviously subject to various equivalent modifications or changes, and such modifications or changes also fall within the scope of the embodiments of this application.

[0087] Figure 7 This is a schematic flowchart illustrating a method for caching write data, as provided in an embodiment of this application. Figure 7 The method shown may include steps 710-790, which are described in detail below.

[0088] Step 710: The cache management system allocates free key-value space.

[0089] The cache management system can allocate storage space for data to be written when writing data is in progress. The system first attempts to allocate storage space for data from a partially full slab in DRAM 120. If there is no partially full slab in DRAM 120, storage space can be allocated from an idle slab in DRAM 120, and that idle slab can be set to a partially full state.

[0090] Step 720: The cache management system determines whether the number of free slabs in DRAM 120 is below the water level.

[0091] The cache management system can check the number of free slabs in DRAM 120 after the data to be written is stored in memory DRAM 120.

[0092] If the cache management system determines that the number of free slabs in DRAM 120 is less than the water level (which can be a preset number of free slabs), it means that the cache space in DRAM 120 is insufficient, and the data stored in the full slabs of DRAM 120 needs to be written to the slabs of SSD 130, then step 730 can be executed.

[0093] If the cache management system determines that the number of free slabs in DRAM 120 is not less than the water level, then step 740 can be executed.

[0094] Step 730: Trigger a full slab writeback in DRAM 120.

[0095] If the cache management system determines that the cache space in DRAM 120 is insufficient, it can write the data stored in the full slab of DRAM 120 to the slab of SSD 130. It can then set the full slab in DRAM 120 as an idle slab, and the newly written data can continue to be cached in the idle slab.

[0096] Step 740: Write KV data.

[0097] If the cache management system determines that the number of free slabs in DRAM 120 is not less than the water level, then the written KV data can be cached in the KV space allocated in DRAM 120.

[0098] Step 750: Has an index item with the same hashkey been found?

[0099] The cache management system can look up the index table in DRAM 120 after the written data is cached in DRAM 120.

[0100] If an index entry matching the key to be written can be found in the index table of DRAM 120, it means that data has already been stored on that key in HDD 140, indicating that the write operation is an update operation, and step 760 can be executed.

[0101] If no index entry matching the key to be written is found in the index table of DRAM 120, it means that there is no data stored on that key in HDD 140. This indicates that the write operation is a new data write operation, and step 770 can be executed.

[0102] Step 760: Update the index entries.

[0103] After determining that the write operation is an update operation, the cache management system can update the original index entry in DRAM 120 with the new index entry corresponding to the newly written data.

[0104] Step 770: Allocate free index entries.

[0105] After determining that the write operation is a new data write operation, the cache management system can allocate a free index entry space from the DRAM 120's index table for the newly written data. For details on the specific implementation of inserting a new index entry into the DRAM 120's index table, please refer to the following text. Figure 8-9 The description in the text will not be repeated here.

[0106] Step 780: Write the index entry.

[0107] After allocating a free index entry space for the newly written data, the index entry corresponding to the newly written data can be stored in the free index entry space.

[0108] Step 790: End.

[0109] In this embodiment, aggregated write requests can be used to store the data to be written in the DRAM 120 at the granularity of a storage unit. The aggregated data can then be written to the SSD 130, thereby reducing the garbage collection overhead inside the SSD 130.

[0110] Optionally, in some embodiments, during the process of inserting a new index entry into the index table of DRAM 120, a free index entry needs to be allocated, and the new index entry can be written to this free index entry. Taking a hash index table as an example, in this embodiment, a free index entry can first be allocated for the new index entry from the Bugu hash table. If there is no free index entry to allocate in the Bugu hash table, a free index entry can then be allocated for the new index entry from the chained hash table.

[0111] Figure 8 This is a schematic structural diagram of a hash table in a DRAM 120 provided in an embodiment of this application. Figure 8 The hash table may include a Bubble Hash Table 810 and a Chained Hash Table 820.

[0112] The Bugu hash table 810 can include multiple hash buckets, and each hash bucket can include an array of multiple index entries. Each index entry can record the hash values ​​calculated by two hash functions for the key (e.g., hashkey1, hashkey2), and the offset of the key-value pair in the SSD 130.

[0113] The chained hash table 820 can include multiple hash buckets, each containing multiple members. Each member has at least one pointer to the next member (this pointer can even be a bidirectional pointer). Each member contains an array of one or more hash index entries, and each hash index entry records the hashkey and offset.

[0114] In traditional chained hash tables, each member includes a hash index entry, and each hash index entry has at least one pointer to the next hash index entry, resulting in significant memory overhead. The chained hash table in this embodiment stores multiple index entries in each hash bucket, reducing pointer overhead and thus reducing the hash table's memory space consumption.

[0115] The following combination Figure 9 This application provides a more detailed description of the specific implementation method for inserting a new index entry in the index table of DRAM 120 in the embodiments of this application. It should be noted that... Figure 9 The examples provided are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific numerical values ​​or specific scenarios illustrated. Figure 9 The examples are obviously subject to various equivalent modifications or changes, and such modifications or changes also fall within the scope of the embodiments of this application.

[0116] Figure 9 This is a schematic flowchart illustrating a method for caching write data, as provided in an embodiment of this application. Figure 9 The method shown may include steps 910-960, which are described in detail below.

[0117] Step 910: Calculate hashkey1, and find the corresponding Bugu hash bucket 1 based on hashkey1.

[0118] The cache management system can calculate a hashkey1 based on the first hash function. It can then find the corresponding Cuckoo Hash Bucket 1 based on hashkey1. For example, it can take the modulo of hashkey1 and find the corresponding Cuckoo Hash Bucket 1 based on the result.

[0119] Step 915: Traverse the Bugu hash bucket 1 and determine if a free index item can be found.

[0120] The cache management system can find the corresponding Bugu hash bucket 1 based on hashkey1, and can traverse Bugu hash bucket 1 to determine whether there is a free index item in hash bucket 1.

[0121] If a free index entry can be found in hash bucket 1, proceed to step 960.

[0122] If no free index entry is found in hash bucket 1, you can proceed to step 920.

[0123] Step 920: Calculate hashkey2, and find the corresponding Bugu hash bucket 2 based on hashkey2.

[0124] The cache management system can calculate a hashkey2 based on the second hash function. Furthermore, it can find the corresponding Cuckoo Hash Bucket 2 based on hashkey1. For example, it can take the modulo of hashkey2 and find the corresponding Cuckoo Hash Bucket 2 based on the result.

[0125] Step 925: Traverse the Bugu hash bucket 2 and determine if a free index item can be found.

[0126] The cache management system can find the corresponding Bugu hash bucket 2 based on hashkey2. It can then iterate through Bugu hash bucket 2 to determine if there is a free index entry within it.

[0127] If a free index entry can be found in hash bucket 2, proceed to step 960.

[0128] If no free index entry is found in hash bucket 2, you can proceed to step 930.

[0129] Step 930: Based on the calculated hashkey1 or hashkey2, find the corresponding chained hash bucket 3.

[0130] If the cache management system cannot find a free index item in the Bugu hash bucket, it can find the corresponding chain hash bucket 3 in the chain hash table based on the calculated hashkey1 or hashkey2, and allocate a free index item in chain hash bucket 3.

[0131] Step 935: Traverse each member in the chained hash bucket 3 and determine if a free index item can be found.

[0132] The cache management system can find the corresponding chained hash bucket 3 in the chained hash table based on hashkey1 or hashkey2. It can then iterate through each member of chained hash bucket 3 to determine if a free index item can be found.

[0133] If a free index entry can be found in chained hash bucket 3, proceed to step 960.

[0134] If no free index entry is found in chained hash bucket 3, you can proceed to step 940.

[0135] Step 940: Assign a new chain member.

[0136] If the cache management system does not find a free index entry in chained hash bucket 3, it can reallocate a member in chained hash bucket 3.

[0137] Step 945: Was the space allocation successful?

[0138] The cache management system can determine whether the allocation of a member was successful after reallocating a member in the chained hash bucket 3.

[0139] If the allocation is successful, proceed to step 950.

[0140] If the allocation fails, proceed to step 955.

[0141] Step 950: Allocate the first index entry for the member.

[0142] If the cache management system successfully allocates a member in chained hash bucket 3, it can store the new index entry in the first free index space of that member.

[0143] Step 955: Select the first index entry of the existing bucket.

[0144] If the cache management system fails to allocate a member in chained hash bucket 3, it can store the new index entry in the first index entry of the existing bucket. It can also store entries in other index spaces based on the index entries stored in the first index entry of the existing bucket.

[0145] Step 960: End.

[0146] The above text combined Figures 1 to 9 This application provides a method for deleting index entries in memory, as described in detail in its embodiments. The following describes embodiments of the apparatus provided by this application. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the preceding method embodiments.

[0147] Figure 10 This application provides an embodiment of an apparatus 1000 for deleting index entries in memory. The apparatus 1000 may include: a selection module 1010, a reading module 1020, a deletion module 1030, and a processing module 1040.

[0148] The selection module 1010 is used to select the target storage unit to be eliminated from the plurality of storage units.

[0149] The read module 1020 is used to read all index entries in the target storage unit, wherein all index entries in the target storage unit are stored consecutively in the target storage unit.

[0150] The deletion module 1030 is used to delete all index entries read from the memory index table.

[0151] The processing module 1040 is used to mark the target storage unit as free.

[0152] Optionally, in some embodiments, the apparatus further includes a storage module for storing multiple data blocks from the target storage unit into the HDD.

[0153] Optionally, in some embodiments, the reading module is specifically used to: read all index entries in the target storage unit at once using a starting address and a length, wherein the starting address is the starting address of all index entries in the target storage unit, and the length is the total length of all index entries in the target storage unit.

[0154] Optionally, in some embodiments, the index table in memory includes multiple members, each member containing an index entry corresponding to each of the multiple data blocks.

[0155] Optionally, in some embodiments, the first memory records information about the plurality of storage cells, including the number of storage cells and / or the number of storage cells in an idle state.

[0156] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described above.

[0157] This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described above.

[0158] Various aspects or features of this application can be implemented as methods, apparatus, or articles of manufacture using standard programming and / or engineering techniques. As used herein, the term "article of manufacture" encompasses a computer program accessible from any computer-readable device, carrier, or medium. For example, computer-readable media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). Additionally, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0159] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0160] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0164] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0165] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for deleting index entries in memory, characterized in that, include: A target storage unit to be evicted is selected from multiple storage units stored in a first memory. Each of the multiple storage units stores multiple data blocks and an index entry corresponding to each data block. All index entries corresponding to the multiple data blocks stored in each storage unit are stored consecutively in each storage unit. The target storage unit stores multiple first data blocks and an index entry corresponding to each first data block. All index entries corresponding to the multiple first data blocks are stored consecutively in the target storage unit. All index entries corresponding to the plurality of first data blocks are read from the target storage unit through a single input / output (IO) operation; Delete all the index entries read from the memory index table, which records the index entries corresponding to the data blocks of the plurality of storage units; The target storage unit is marked as free.

2. The method according to claim 1, characterized in that, Before deleting all the index entries read from the memory index table, the method further includes: Multiple data blocks in the target storage unit are stored in the second memory.

3. The method according to claim 1, characterized in that, The step of reading all index entries corresponding to the plurality of first data blocks from the target storage unit through a single input / output (IO) includes: Using a starting address and a length, all index entries corresponding to the plurality of first data blocks are read from the target storage unit through the single input and output. The starting address is the starting address of all index entries in the target storage unit, and the length is the total length of all index entries in the target storage unit.

4. The method according to any one of claims 1 to 3, characterized in that, The memory index table includes multiple members, and each member contains an index entry corresponding to each of the multiple data blocks.

5. The method according to any one of claims 1 to 3, characterized in that, The plurality of storage units are stored in a first memory, which records information about the plurality of storage units, including the number of storage units and / or the number of storage units that are in an idle state.

6. An apparatus for deleting index entries in memory, characterized in that, include: A selection module is used to select a target storage unit to be evicted from multiple storage units. The multiple storage units are stored in a first memory. Each of the multiple storage units stores multiple data blocks and an index entry corresponding to each of the multiple data blocks. All index entries corresponding to the multiple data blocks stored in each storage unit are stored consecutively in each storage unit. The target storage unit stores multiple first data blocks and an index entry corresponding to each of the multiple first data blocks. All index entries corresponding to the multiple first data blocks are stored consecutively in the target storage unit. The read module is used to read all index entries corresponding to the plurality of first data blocks from the target storage unit through a single input / output (I / O); The deletion module is used to delete all the index entries read from the memory index table, which records the index entries corresponding to the data blocks of the multiple storage units; A processing module is used to mark the target storage unit as free.

7. The apparatus according to claim 6, characterized in that, The device further includes: A storage module is used to store multiple data blocks from the target storage unit into a second memory.

8. The apparatus according to claim 6, characterized in that, The reading module is specifically used for: Using a starting address and a length, all index entries corresponding to the plurality of first data blocks are read from the target storage unit through the single input and output. The starting address is the starting address of all index entries in the target storage unit, and the length is the total length of all index entries in the target storage unit.

9. The apparatus according to any one of claims 6 to 8, characterized in that, The memory index table includes multiple members, and each member contains an index entry corresponding to each of the multiple data blocks.

10. The apparatus according to any one of claims 6 to 8, characterized in that, The plurality of storage units are stored in a first memory, which records information about the plurality of storage units, including the number of storage units and / or the number of storage units that are in an idle state.

11. A storage manager, characterized in that, The storage manager is used to perform the following procedures: A target storage unit to be evicted is selected from multiple storage units stored in a first memory. Each of the multiple storage units stores multiple data blocks and an index entry corresponding to each data block. All index entries corresponding to the multiple data blocks stored in each storage unit are stored consecutively in each storage unit. The target storage unit stores multiple first data blocks and an index entry corresponding to each first data block. All index entries corresponding to the multiple first data blocks are stored consecutively in the target storage unit. All index entries corresponding to the plurality of first data blocks are read from the target storage unit through a single input / output (IO) operation; Delete all the index entries read from the memory index table, which records the index entries corresponding to the data blocks of the plurality of storage units; The target storage unit is marked as free.

12. A computer storage medium, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 5.

13. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 5.

Images