Separated memory-oriented double-layer extensible cache system and method

Abstract

The invention belongs to the technical field of distributed storage, and discloses a double-layer extensible cache system and method oriented to a separated memory, the system comprises at least one computing server CS and a memory resource pool formed by at least one memory server MS, and each CS and the memory resource pool are interconnected through RDMA; a cache CiCS used for sharing cache objects for all clients running in the CS is constructed in each CS, a cache CiMS used for storing the cache objects is constructed in the memory resource pool, and all the clients running in all the CSs can access all the cache objects stored in the CiMS; the CiCS can accelerate the access of the CiMS, and the CiMS can accelerate the access of the underlying storage system. In addition, the dynamic expansion and elasticity of the cache system are realized according to actual requirements, and the cache service does not need to be paused or the performance is not obviously degraded. According to the invention, the performance and the expansibility of the double-layer extensible cache of the separated memory are improved.

Description

Technical Field

[0001] The present invention belongs to the field of distributed storage technology, and more specifically, relates to a double-layer scalable cache system and method for separate memory. Background Art

[0002] To improve CPU and DRAM utilization and reduce hardware costs, a split memory architecture has been proposed. It decouples the CPU and DRAM resources of a single server into a compute server (CS) and a memory server (MS). The compute server is equipped with a large number of CPU cores and a small amount of DRAM as a runtime cache. The memory server is equipped with a controller with ample memory and low-power processing units to perform control tasks (for example, network connectivity and memory management). The compute server and the memory server are interconnected via high-speed communication protocols such as RDMA and CXL. Most split memory-based systems use RDMA for interconnection. CSs use unilateral primitives such as RDMA_WRITE, RDMA_READ, or RDMA_CAS to byte-address the memory in the MS and access cold or hot data based on the load distribution without using the CPU in the MS. Currently, commercial RDMA has a single-transaction network communication latency of 2-4 μs, and CXL devices built using FPGAs also have a latency in the microsecond range. This makes the latency for CS accessing memory in the MS 20 times higher than the latency for accessing local memory. Therefore, it is necessary to build a more fine-grained cache in the CS, namely, the cache built in the compute server (CiCS). In the future, after the emergence of commercial CXL devices with nanosecond latency, tiered cache systems will continue to be valuable because clients can simultaneously access the bandwidth of DIMMs (Dual In-line Modules) and PCIe (Peripheral Component Interconnect Express).

[0003] The performance of a cache system based on split memory is closely related to its architecture design. This is because split memory is a typical two-layer architecture, similar to a multi-level CPU cache architecture. It is necessary to fully utilize the characteristics and performance of each level of cache to fully utilize the advantages of split memory. However, this brings a serious problem: cache consistency. Figure 1 As shown in the figure, there are four common cache architectures in existing DM-based cache systems, databases, or indexes, except Figure 1 (a) does not use CiCS, which can directly avoid the problem of cache inconsistency. The other three (b)-(d) all use different methods to solve the problem of cache inconsistency. Figure 1In the architecture mode of (a) without CiCS, although not using CiCS can directly avoid the problem of cache inconsistency, CSs always need to remotely access the cache built in the memory server (Caching in Memory Server, CiMS), so it will cause additional network overhead and large latency. Figure 1 In the architecture mode of (b) SW / MR which is a write-many-read-one architecture, only one read-write node (RW node) can actively modify CiCS, avoiding multiple nodes from modifying. However, this architecture usually requires a coordination node to manage the reference situation of cache objects in different nodes and propagate the latest cache objects in the CiCS of the RW node to other read-only nodes (RO nodes), which consumes additional server resources and network resources. Figure 1 In the architecture mode of (c) Sharding which is a sharding architecture, each CS has different partitions, and the key spaces contained in different partitions do not cross. Therefore, there is no cache consistency problem between different CiCS shards. However, the disadvantage is that each partition can only be accessed by the corresponding client, so the scalability of the cache is severely limited, and problems such as load imbalance may also occur. Figure 1 In the architecture mode of (d) Coherence portocol which is a software-based cache consistency protocol architecture, it directly constructs a cache consistency protocol at the software level, broadcasts the changes of cache objects in CiCSs, and designs corresponding strategies to maintain the consistency of all relevant cache objects in different CiCSs. However, due to the need for broadcasting, the corresponding cluster scale is severely limited. Therefore, the existing cache architectures limit the performance and elasticity advantages of disaggregated memory.

[0004] In addition, when conditions such as load and memory resources change, the cache system should be scaled up and down to ensure its hit rate and space utilization. In the process of cache scaling up / down, the most core is how to adjust the cache index. For example, the common expansion strategies of hash-based cache systems include: progressive rehashing, hierarchical hash rehashing, and scalable hash rehashing. Compared with progressive rehashing and hierarchical hash rehashing, scalable hash rehashing incrementally creates hash subtables (subtables), expands the suffix of the key of the cache object, and migrates the part of the cache objects that match the new hash subtable in sequence, only needing to move part of the cache objects. However, for each cache object that needs to be migrated, it needs to use RDMA_WRITE / CAS to move the cache object. Since concurrent insertions and migrations may occur in the hash subtable, in order to ensure the concurrent correctness of the hash subtable during the expansion process, locks / complex lock-free operations are often required, which severely limits the performance and scalability of the cache system. Summary of the Invention

[0005] In view of the above defects or improvement requirements of the prior art, the present invention provides a two-layer scalable cache system and method for a disaggregated memory, aiming to improve the performance and scalability of the two-layer scalable cache of the disaggregated memory.

[0006] To achieve the above object, the present invention provides a two-layer scalable cache system for a disaggregated memory, including:

[0007] At least one computing server CS and a memory resource pool composed of at least one memory server MS, where each CS and the memory resource pool are interconnected through RDMA; among them, a cache CiCS for sharing cache objects for all clients running in the CS is built in each CS, and a cache CiMS for storing cache objects is built in the memory resource pool, and the clients running in all CSs are all used to access all the cache objects stored in CiMS;

[0008] When the client initiates a Get request for reading a target cache object, the client accesses CiCS in its own CS. If the client obtains the target cache object in CiCS or only obtains the remote address pointer of the target cache object, the cache hits, otherwise the cache misses.

[0009] When the cache hits, if the target cache object is obtained in CiCS, the client uses the one-sided RDMA_READ operation to read the version number of the target cache object stored in CiMS based on the remote address pointer of the target cache object, and determines whether the version number is consistent with the version number of the target cache object obtained from CiCS. If so, the Get request is completed. If not, the client reads the target cache object in CiMS based on the version number read from CiMS to complete the Get request; if the remote address pointer of the target cache object is obtained in CiCS, the client uses the one-sided RDMA_READ operation to read the target cache object from CiMS based on the remote address pointer to complete the Get request.

[0010] When the cache misses, if the client can directly read the target cache object from CiMS, the Get request is completed and inserted into CiCS where the client is located; otherwise the client reads the target cache object from the lower storage layer, completes the Get request, and inserts the read target cache object into CiMS.

[0011] Further, CiMS is a data structure based on extensible hashing;

[0012] When the cache misses, the client first accesses the hash table in the CiMS. If the target cache object is found in the hash table, the client can directly read the target cache object from the CiMS; otherwise, the client reads the target cache object from the lower storage layer.

[0013] Further, when the client initiates a Set request for updating the target cache object, the client accesses the CiCS in its CS. When the cache hits, based on the remote address pointer of the target cache object, the client uses the RDMA_CAS operation to lock the target cache object stored in the CiMS, reads the target cache object from the CiMS, and then uses the RDMA_WRITE operation to complete the update and unlocking of the target cache object in the CiMS.

[0014] When the cache misses, if the client reads the target cache object from the CiMS, the target cache object read will be directly updated in the CiMS; if the client reads the target cache object from the lower storage layer, the target cache object read will be inserted into the CiMS.

[0015] Further, each CiCS includes three LRU caches: LRU1, LRU2, and LRU3. Among them, LRU1 and LRU2 contain the hash value of the cache object key, the remote address pointer of the corresponding cache object in the CiMS, the length, and the local access frequency; LRU3 contains the hash value of the cache object key, the remote address pointer of the corresponding cache object in the CiMS, the length, the local access frequency, the value of the corresponding cache object in the CiMS, and the version number.

[0016] When the client initiates the Get request and the cache misses, if the client can directly read the target cache object from the CiMS, inserting it into the CiCS where the client is located includes:

[0017] If LRU1 is not full, the target cache object is inserted into LRU1; if LRU1 is full, the client inserts the coldest cache object in LRU1 into LRU2, and then inserts the target cache object into LRU1. The coldest cache object refers to the cache object at the tail of the LRU linked list. If LRU2 is also full at this time, the client compares the local access frequencies of the coldest cache object in LRU1 and the coldest cache object in LRU2, and discards the one with the smaller local access frequency.

[0018] When the client initiates the Get request and the cache hit occurs, if the remote address pointer of the target cache object is obtained in LRU1 and the target cache object is read in CiMS, the client updates the target cache object to the head of the LRU1 linked list; if the remote address pointer of the target cache object is obtained in LRU2 and the target cache object is read in CiMS, when LRU3 is not full, the client inserts the target cache object into LRU3, when LRU3 is full, the client first evicts the coldest cache object in LRU3, then inserts the target cache object into LRU3, and inserts the coldest cache object evicted from LRU3 into LRU2. If LRU2 is also full at this time, the client compares the local access frequencies of the coldest cache object evicted from LRU3 and the coldest cache object in LRU2, and discards the one with the smaller local access frequency; if the target cache object is obtained in LRU3, when the client reads the version number of the target cache object from CiMS according to the remote address pointer of the target cache object, it also includes reading the key of the target cache object from CiMS. When the key read from CiMS is the same as the key of the target cache object to be read in the Get request and the version numbers are the same, the Get request is completed, and the target cache object is updated to the head of the LRU3 queue; if the keys are different, the target cache object is evicted from LRU3; if the keys are the same but the version numbers are different, the value of the target cache object is deleted from LRU3 and it is evicted to LRU2;

[0019] For the burst cache access of the client, after the client obtains the target cache object from CiMS or from the lower storage layer, it inserts it into LRU1.

[0020] Furthermore, CiMS includes an extensible hash table and discrete cache objects; each hash sub-table of the extensible hash table includes several hash buckets, each hash bucket includes multiple hash slots, each hash slot corresponds to a memory block, and each memory block is used to store a cache object; each hash slot contains: a fingerprint fp for determining whether the hash slot may point to the target cache object, the size len of the cache object, the address pointer of the cache object, a flag sig, and the timestamp when the client last accessed the cache object;

[0021] Inserting the read target cache object into CiMS includes:

[0022] Determining the hash bucket where the target cache object is located in CiMS according to the hash value of the target cache object;

[0023] When the hash bucket where it is located is not full and there are empty memory blocks in CiMS, the client applies for an empty memory block, writes the target cache object into this memory block using the RDMA_WRITE operation, and then modifies the {fp, len, pointer} information in the hash slot corresponding to this memory block to the information corresponding to the target cache object using the RDMA_CAS operation;

[0024] When the hash bucket where it is located is not full but the number of memory blocks in CiMS is insufficient, the client first executes the cache replacement logic and then the insertion operation, specifically including:

[0025] Maintain the latest access timestamp timestamp of each hash bucket in CiMS in CS, and divide the hash buckets into n groups according to timestamp. Each group contains a batch of hash buckets with similar timestamps. The group with a larger timestamp indicates that the hash buckets in it are hotter. When performing cache replacement, the probability of evicting cache objects in hotter hash buckets is smaller; when performing cache replacement, the client selects the group of hash buckets with the smallest timestamp, randomly selects a hash bucket from it, evicts the coldest cache object in it, and reclaims the corresponding memory block; inserts the target cache object into the reclaimed memory block;

[0026] When the hash bucket where it is located is full, the client evicts the coldest cache object in the hash bucket, reclaims the corresponding memory block; inserts the target cache object into the reclaimed memory block.

[0027] Further, the client executes the LRU algorithm based on the timestamps of each cache object in the hash bucket to obtain the coldest cache object in the hash bucket.

[0028] Further, it also includes the process of expanding or / and shrinking CiMS: re-hashing the hash table in CiMS. During the re-hashing period, the client does not perform the insertion operation of cache objects on CiMS;

[0029] Re-hashing the hash table in CiMS includes:

[0030] Dividing the hash sub-table to be migrated into different background threads for operation according to regions; each background thread sequentially reads multiple consecutive hash buckets in the hash sub-table to be migrated and counts the hash slots that need to be migrated in each hash bucket;

[0031] During the expansion process, the background thread uses the batch RDMA_WRITE operation to write the hash slots that need to be migrated into the corresponding hash buckets in the new hash sub-table;

[0032] During the capacity reduction process, the background thread uses batch RDMA_WRITE operations to write the hash slots to be migrated into the corresponding hash buckets in the target hash sub-table; if there are not enough empty hash slots in the target hash sub-table, the colder cache objects in the target hash sub-table are first removed to recycle the corresponding hash slots; then the background thread uses batch RDMA_WRITE operations to write the hash slots to be migrated into the corresponding hash buckets in the target hash sub-table.

[0033] During the process of writing to the new hash sub-table or the corresponding hash bucket of the target hash sub-table, the flag bit sig in the migrated hash slots of the old hash bucket is set to 0xFF using the RDMA_WRITE operation, so that the client can still find the target cache object through this hash slot during rehashing; after rehashing is completed, the values of the hash slots with the flag bit sig being 0xFF have been migrated to the new hash sub-table, and the corresponding hash slots in the old hash sub-table are recycled.

[0034] The present invention also provides a two-layer scalable cache method for disaggregated memory, which is applied to the client in the two-layer scalable cache system for disaggregated memory as described in any one of the above, and the method includes:

[0035] When initiating a Get request for reading a target cache object, access the CiCS in its corresponding CS. If the target cache object is obtained in the CiCS or only the remote address pointer of the target cache object is obtained, the cache hits; otherwise, the cache misses.

[0036] When the cache hits, if the target cache object is obtained in the CiCS, based on the remote address pointer of the target cache object, use a unilateral RDMA_READ operation to read the version number of the target cache object stored in the CiMS, and determine whether the version number is consistent with the version number of the target cache object obtained from the CiCS. If so, the Get request is completed; if not, the target cache object is read from the CiMS based on the version number read from the CiMS, and the Get request is completed; if the remote address pointer of the target cache object is obtained in the CiCS, based on the remote address pointer, use a unilateral RDMA_READ operation to read the target cache object from the CiMS, and complete the Get request.

[0037] When the cache misses, if the target cache object can be directly read from the CiMS, the Get request is completed and inserted into the CiCS where the client is located; otherwise, the target cache object is read from the lower storage layer, the Get request is completed, and the read target cache object is inserted into the CiMS.

[0038] Further, when initiating a Set request for updating a target cache object, access the CiCS in its CS. When the cache is hit, based on the remote address pointer of the target cache object, use the RDMA_CAS operation to lock the target cache object stored in the CiMS, read the target cache object in the CiMS, and then use the RDMA_WRITE operation to complete the update and unlocking of the target cache object in the CiMS;

[0039] When the cache is not hit, if the target cache object is read from the CiMS, the target cache object read will be directly updated in the CiMS; if the target cache object is read from the lower storage layer, the target cache object read will be inserted into the CiMS.

[0040] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the two-layer scalable cache method for separated memory as described above.

[0041] Generally speaking, through the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:

[0042] (1) The two-layer scalable cache architecture for separated memory of the present invention includes at least one computing server CS and a memory resource pool composed of at least one memory server MS. Each computing server includes a running client thread and a cache CiCS built in the computing server, and all client threads running in the computing server CS share the CiCS; the memory resource pool includes a cache CiMS built in the memory server MS, and client threads running in all computing servers CS can access all cache objects in the CiMS. Therefore, optimal scalability can be obtained, and there will be no problems such as load imbalance. To ensure the consistency between caches on the computing side, the cache system uses the write around mode when updating cache objects, does not actively update the caches on the computing side, but directly updates the cache objects in the CiMS (insert the target cache object read into the CiMS), maintaining the consistency between caches on the computing side with relatively low overhead. To verify the consistency between the caches on the computing side and the caches on the memory side, when the cache on the computing side is hit, the client reads the version information in the cache object on the memory side and verifies whether it is consistent with the version information in the cache object on the computing side, thus achieving efficient consistency detection, and only one RDMA_READ access is required to complete the consistency verification without broadcasting, which can be applied to large-scale cluster scenarios. Therefore, the performance of the two-layer scalable cache for separated memory is improved.

[0043] (2) Further, in order to implement an efficient and low-memory-consumption cache in the computing server, the present invention designs an efficient CiCS. The basic component unit of CiCS is three LRU caches. The LRU1 cache can be used to receive burst access from the client, thereby caching requests that are frequently accessed in the short term; the LRU2 cache stores the remote address information of the relatively hot cache objects, thereby accelerating the remote access of the cache objects in the MS with lower memory overhead. The LRU3 cache stores the remote address, hash value of the key, value, and version information of the hottest cache objects. Based on the maintenance process of the three LRUs in CiCS, finally, the hottest part of the data is in LRU3, and the second-hottest part of the data is in LRU2. Therefore, caching their values in LRU3 can reduce the network bandwidth consumed when remotely reading from CiMS.

[0044] (3) Preferably, before verifying the version number, the key is verified first. This is because in order to reduce the memory overhead of CiCS, LRU1-LRU3 store the hash values of the cache object keys. Therefore, there may be multiple keys accessing the same CiCS cache object. To verify whether the cache object in CiCS corresponds to the current request, when the client accesses the cache object in CiMS according to the address pointer in CiCS, in addition to reading the version number, it will also read the key field at the same time. If the key fields do not match, then it will access the hash table in CiMS according to the key of the cache object in CiMS, then find the accurate target cache object, and finally update CiCS.

[0045] (4) Further, in order to achieve the dynamic expansion and elasticity of CiMS, the present invention separates the index (extensible hash) of the cached object from the cached object, uses the extensible hash to index the cached object, and the address and size of the cached object are stored in the hash slots of the hash table. During the migration process, only an efficient index structure adjustment is required. During the expansion or contraction of CiMS, multiple background threads migrate the hash slots in the hash sub-table to the new sub-table (or target sub-table) in regions and batches. At the same time, the sig flag in the migrated hash slots in the old hash bucket is set to 0xFF using the RDMA write operation. Therefore, the client can still find the target data item through this hash slot during rehashing. Thus, the migration of hash slots does not affect the normal access and in-place update of the cached object. Different background threads are responsible for migrating different regions of the hash table, so they can execute concurrently. In addition, the background threads batch-read the hash buckets, count the hash slots to be migrated, and batch-write the target hash slots into the new hash sub-table (or target sub-table) using the RDMA Write primitive, thereby saving a large amount of RDMA. The caching system of the present invention designs a low-latency CiMS access operation using the CiCS and RDMA batching principles, reducing the number of round trips in the RDMA network. Description of the Drawings

[0046] Figure 1 are four common cache architectures based on disaggregated memory;

[0047] Figure 2 is the diagram of the two-layer extensible cache architecture for disaggregated memory in the embodiment of the present invention;

[0048] Figure 3 is the basic structure and logic of CiCS in the caching system of the embodiment of the present invention;

[0049] Figure 4 is the basic structure and logic of CiMS in the caching system of the embodiment of the present invention;

[0050] Figure 5 is the cache access logic for the two-layer cache collaboration in the embodiment of the present invention;

[0051] Figure 6 is the lightweight cache expansion mechanism in the embodiment of the present invention. Detailed Embodiment

[0052] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0053] Example 1

[0054] like Figure 2 As shown, an embodiment of the present invention provides a two-layer scalable cache system for separated memory, including: at least one computing server CS and a memory resource pool (MemoryPool) composed of at least one memory server MS; each computing server CS and the memory resource pool are interconnected via RDMA; each computing server includes a running client thread and a cache (CiCS) built in the computing server, and all clients (client threads) running in the computing server CS share the CiCS; the memory resource pool includes a cache (CiMS) built in the memory server MS, and client threads running in all computing servers CS can access all cache objects in the CiMS, thereby achieving optimal scalability. In an embodiment of the present invention, CiCS is composed of a data structure based on an LRU linked list, and CiMS is composed of a data structure based on an extensible hash.

[0055] For a Get request (read operation request, used to read the target cache object) initiated by the client, the client first accesses the CiCS in the CS where it is located. If the client hits the requested target cache object in the CiCS (cache hit), including: obtaining the target cache object or obtaining the remote address pointer (remote pointer) of the target cache object. When the target cache object is obtained, the client uses a one-sided RDMA_READ operation to directly read the version number (version) of the target cache object stored in the CiMS based on the remote address pointer (remote pointer) of the obtained target cache object, and determines whether the version number obtained from the CiMS is consistent with the version number of the target cache object obtained by the client from the CiCS. If so, it means that the value of the target cache object obtained in the CiCS is the latest, and the Get request ends, that is, the client has read the target cache object. If not, the client reads the target cache object in the CiMS based on the version number obtained from the CiMS, and the Get request ends.

[0056] If the client obtains the remote address pointer (remotepointer) of the target cache object from the CiCS, the client uses a one-sided RDMA_READ operation to directly read the target cache object from the CiMS based on the obtained remote address pointer, and the Get request ends.

[0057] If the client fails to hit the requested target cache object in CiCS, then the client directly reads the target cache object from CiMS and determines whether the target cache object can be read in CiMS: The client first accesses the hash table in CiMS. If the target cache object is found in the hash table, then the client reads the target cache object in CiMS. Otherwise, the client needs to read the target cache object from the lower storage layer. If the client directly reads the target cache object from CiMS, then the Get request ends. If the client does not find the target cache object in CiMS, then the client reads the target cache object from the lower storage layer of the cache system, the Get request ends, and then inserts it into CiMS through a Set request. Specifically, when inserting into CiMS, (1) if it is determined according to the hash value of the target cache object that the hash bucket where it is located in CiMS is not full and there is an empty memory block in CiMS, directly insert the target cache object into an empty memory block; (2) if the hash bucket where the target cache object is located in CiMS is not full, but there is no empty memory block in CiMS, it is necessary to perform a replacement operation first and then an insertion operation. The client reads the colder hash bucket in CiMS and executes the LRU algorithm in the computing server according to the access timestamp of the valid slot in the hash bucket, so as to calculate the cache object to be evicted, allocate the memory block occupied by the cache object to be evicted from this hash bucket to the target cache object, and then insert the target cache object. (3) If the hash bucket where the target cache object is located in CiMS is full, then the client will read this hash bucket and execute the LRU algorithm, select the coldest cache object among them, and replace this cache object with the target cache object. When the cache capacity is limited and the hit rate is low, it is possible to choose to increase the cache capacity, apply for more memory from MS, and then use multiple background threads to batch migrate slots to resize the hash table.

[0058] For the Set request of the client (a write operation request used to update the target cache object), the client first accesses the CiCS in its CS. If the cache is hit, the client uses the RDMA_CAS operation to lock the target cache object stored in the CiMS according to the remote pointer of the target cache object obtained from the CiCS, uses the RDMA_READ operation request to implement the read task of the target cache object, and then uses the RDMA_WRITE operation to complete the update and unlocking of the target cache object in the CiMS. If the CiCS cache is not hit, the client needs to remotely read the hash table in the CiMS to determine whether the target cache object exists in the CiMS. If the target cache object does not exist in the CiMS, the target cache object obtained from the lower storage layer needs to be inserted into the CiMS. If the target cache object exists in the CiMS but is not loaded into the CiCS, the client directly updates the target cache object in the CiMS.

[0059] Preferably, the basic structure and logic of the CiCS in the cache system of the embodiment of the present invention are as Figure 3 shown. Each CiCS consists of three LRU caches. LRU1 and LRU2 contain the hash value of the cache object key, the remote pointer of the corresponding cache object in the CiMS, the length of the cache object (len), and the local access frequency (freq). In addition to the hash value of the cache object key, the remote pointer of the corresponding cache object in the CiMS, the length of the cache object (len), and the local access frequency (freq), LRU3 also includes the value (value) and version number (version) of the corresponding cache object in the CiMS. In the embodiment of the present invention, the total number of cache objects in LRU1, LRU2, and LRU3 is 1% of the load. Among them, the number of cache objects in LRU3 is 0.1% of the load. Therefore, the CiCS only requires very little memory. Preferably, the capacity of LRU1 is less than the capacity of LRU2

[0060] In the embodiment of the present invention, for the Get request, if the client does not hit the requested target cache object in the CiCS, after remotely obtaining the target cache object from the CiMS, it further includes inserting it into the CiCS where the client is located, specifically including:

[0061] (1) If LRU1 is not full, construct the hash value (hash) and remote pointer (remotepointer) of the target cache object into a new CiCS item and add it to LRU1.

[0062] (2) If LRU1 is full, the client will try to insert the coldest cache object in LRU1 into LRU2, and then insert the target cache object into LRU1. If LRU2 is also full at this time, the client will compare the local access frequency (freq) of the cache object to be inserted at this time (the coldest cache object in LRU1) with the local access frequency of the coldest cache object in LRU2. The client will discard the one with the lower local access frequency (the cache object with the least frequent access). For example, if the local access frequency of the coldest cache object in LRU2 is lower, the client will evict and discard it from LRU2. If the local access frequency of the cache object to be inserted at this time is lower, it will be directly discarded. This can, to a certain extent, obtain the advantages of LFU (Least Frequently Used). Among them, the coldest cache object refers to the cache object at the tail of the LRU linked list.

[0063] In the embodiment of the present invention, for a Get request, when the cache is hit, the local access frequency of the corresponding target cache object in CiCS is increased. In the embodiment of the present invention, each time the cache is hit, the local access frequency of the corresponding target cache object in CiCS is increased by 1. If the client hits the requested target cache object in CiCS, if the remote address pointer of the target cache object is found in LRU2 and the target cache object in CiMS is successfully read using its remote address pointer, when LRU3 is not full, the client will add the target cache object stored in CiMS to LRU3. If LRU3 is full at this time, then the client will evict the coldest cache object in LRU3, add the target cache object to LRU3, and add the evicted coldest cache object in LRU3 to LRU2. Similarly, if LRU2 is also full at this time, the client will compare the frequency (freq) in the cache object to be inserted (the coldest cache object evicted from LRU3 at this time) with the frequency of the coldest cache object in LRU2, and evict the one with the lower frequency (the cache object with the least frequent access) from LRU2.

[0064] If the remote address pointer of the target cache object is found in LRU1 and the target cache object in CiMS is successfully read using its remote address pointer, then there is no need to insert at this time, and the client only updates the position of the target cache object in LRU1 (places the target cache object at the head of the LRU1 linked list).

[0065] If the target cache object is found in LRU3, consistency verification is required. If the consistency verification passes, the client will place the target cache object at the head of the LRU3 queue; if the consistency verification fails and the key of the target cache object to be read in the Get request is inconsistent with the key of the target cache object read in CiMS, it means that the target cache object in LRU3 is invalid and needs to be evicted from LRU3; when the keys are the same but the version numbers in LRU3 and CiMS are inconsistent, it indicates that the cache object has been updated. The cache object in LRU3 will be evicted to LRU2 and the value information of the cache object will be deleted in LRU3.

[0066] According to the migration of cache objects in these several stages, finally, the hottest part of the data is in LRU3 and the second-hottest part of the data is in LRU2. Therefore, their values are cached in LRU3 to reduce the network bandwidth consumed when remotely reading from CiMS.

[0067] For the cache access bursts from the client, they may not have temporal locality. At this time, when the client accesses CiCS and there is no corresponding cache object in any of the three LRUs, the client may need to obtain these cache objects from CiMS or from the underlying storage layer. At this time, the client will insert the cache objects corresponding to these burst cache accesses into LRU1. For the specific insertion process, please refer to the above process of inserting cache objects into LRU1. Since it is a burst cache access, after the LRU1 insertion process for several rounds, this part of the requests will be removed from CiCS to prevent them from entering LRU2 and LRU3, thus avoiding the contamination of CiCS by accesses without temporal locality. In addition, the cache objects in CiCS will become invalid due to the update and eviction of cache objects in CiMS. In the embodiments of the present invention, how the cache system responds to these two situations will be explained separately in the processes of Get and Set requests later.

[0068] Preferably, the basic structure and logic of CiMS in the cache system of the embodiments of the present invention are as Figure 4 shown. CiMS consists of an extensible hash table and discrete cache objects. Among them, the directory contains the starting address of the hash subtable, and CS caches the directory. The extensible hash table is composed of multiple hash subtables, each hash subtable is composed of several hash buckets, each hash bucket is composed of multiple hash slots, each slot corresponds to a memory block, and each memory block is used to store a cache object. Among them, the memory block corresponding to the valid slot already stores a cache object, and the memory block corresponding to the empty slot is also empty, as Figure 4As shown. Each slot contains: (1) fp, used to preliminarily determine whether this slot may point to the target cache object; (2) len, representing the size of the cache object; (3) pointer, representing the address of the cache object; (4) sig, which is a flag bit. When the hash table is resized, after this slot is migrated, this flag bit will be set to 0xFF (intermediate state); (5) timestamp, representing the timestamp when the client last accessed this cache object, used to execute the LRU algorithm in the client. Among them, fp, len, and pointer form an 8-byte atomic field and are atomically modified using the RDMA_CAS (Compare and Swap) operation when inserting or deleting a cache object.

[0069] The cache object in CiMS consists of keylength (the length of the key), value length (the length of the value), version, key, value, and lock (the lock, with states including locked and unlocked). When the client inserts a target cache object into CiMS, the client first determines the hash bucket in CiMS where the target cache object is located according to the hash value of the target cache object, and then there are the following situations:

[0070] (1) When the hash bucket it is in is not full and there is an empty memory block in CiMS, the client first applies for an empty memory block, uses the RDMA_WRITE operation to write the target cache object into this memory block, and then uses the RDMA_CAS operation to modify the {fp, len, pointer} information in the corresponding empty slot of this memory block to the information corresponding to this target cache object. When the client evicts the target cache object, it first sets {fp, len, pointer} to empty, and then reclaims the memory block pointed to by pointer.

[0071] (2) When the hash bucket where it is located is not full, but the memory block capacity in CiMS for storing cache objects is insufficient, the insertion operation of the client will trigger the cache replacement logic of the cache system. Since the access information (timestamp) of all cache objects is stored in CiMS, the client cannot understand the detailed access behavior of the cache objects. The client also cannot directly maintain the access information of all cache objects in CS because this would require CS to consume a large amount of memory. Therefore, in the embodiments of the present invention, access information at the hash bucket level is maintained in CS, and the hash buckets are divided into n groups according to the latest access timestamp. Each group contains a batch of buckets with similar latest access timestamps. The group with a larger latest access timestamp indicates that the hash buckets in it are hotter. Therefore, when performing cache replacement, the probability of evicting cache objects in hotter hash buckets is smaller. When cache replacement occurs, the client selects a group with the smallest probability (corresponding to the group of hash buckets with the smallest latest access timestamp), randomly selects a hash bucket from it, and then evicts the coldest cache object in it. The recycled memory block is used to store the new target cache object. Among them, the coldest cache object is determined based on the LRU algorithm. The access timestamps of the cache objects in the hash bucket are executed with the LRU algorithm to obtain the coldest cache object in the hash bucket.

[0072] (3) When the hash bucket where it is located is full, the insertion operation on this hash bucket will also trigger the cache replacement logic of the cache system: the client will read this hash bucket, execute the LRU algorithm, select the coldest cache object in it, and replace this cache object with the target cache object.

[0073] As Figure 5 shown, it is the cache access logic for the double-layer cache collaboration in the embodiments of the present invention. The collaborative work of CiCS and CiMS in the cache system is crucial. For example, when CiCS hits, the client can obtain the remote address of the target cache object or the value value and version version information of the cache object in CiCS. With the help of this information, it can help the cache system reduce the consumption of network I / O and bandwidth. If there is no CiCS, then the client needs to remotely access the hash table set in CiMS to obtain the remote address of the target cache object, and then read the entire cache object. On the other hand, even for the most frequently accessed part of the data, the client still needs to repeatedly read from CiMS.

[0074] CiCS can help the client in CS access the target cache object more quickly. When the cache system performs the Set operation, it utilizes the idea of write-around, and does not actively update CiCS. Instead, it directly updates CiMS to ensure that the cache in CiMS is always up-to-date and consistent. When the CiCS cache hits, the client can obtain the remote address of the target cache object or the value and version information of the cache object in CiCS, and it is necessary to verify with the cache object in CiMS to confirm cache consistency. Therefore, the tricky cache consistency problem on the computing side is solved. In addition, the cache access logic designed in the present invention for the cooperation of the double-layer cache completes the Get and Set operations with very low network overhead.

[0075] Specifically, for the Get request, the client queries CiCS where the target cache object to be read is located according to the key in the Get request. Figure 5In (a) Get, it shows the case of cache hit. If the target cache object (hash, version, remote pointer, value) is found in the LRU3 of CiCS, that is, the hash value of the key of the target cache object to be read in the Get request is equal to the hash value hash of the cache object in the LRU3 of CiCS, which means the target cache object is found in the LRU3 of CiCS (cache hit). Then, it is necessary to read the key and version number of the target cache object from CiMS according to its remote address pointer (remote pointer). After that, the key and version number of the target cache object read from CiMS are respectively verified for consistency with the key of the target cache object to be read in the Get request and the version number in LRU3. If both the key and version number are the same, then the value of the target cache object stored in CiCS is the latest and is consistent with the target cache object in CiMS. To maintain the cache object access information (timestamp) in CiMS, the client will initiate an asynchronous RDMA_WRITE operation to write the latest timestamp into the slot corresponding to the target cache object. This process is not in the critical path. If the key and version number are not both the same, first judge whether the keys of the two are the same. If the key of the target cache object to be read in the Get request is inconsistent with the key in CiMS, it means the target cache object in LRU3 is invalid, and this target cache object needs to be evicted from LRU3, and directly read the accurate target cache object in CiMS. If the keys of the two are the same, but the version numbers in LRU3 and CiMS are inconsistent, it means the cache object has been updated. The cache object in LRU3 will be evicted to LRU2, and the value information of this cache object will be deleted in LRU3. Preferably, before verifying the version number, the key is verified first. This is because to reduce the memory overhead of CiCS, LRU1 - LRU3 store the hash value (hash) of the cache object key. Therefore, there may be multiple keys accessing the same CiCS cache object. To verify whether the cache object in CiCS corresponds to the current request, when the client accesses the cache object in CiMS according to the address pointer in CiCS, in addition to reading the version number, the key field will also be read at the same time. If the key field does not match, then the hash table in CiMS will be accessed according to the key of the cache object in CiMS, and then the accurate target cache object will be found, and finally CiCS will be updated.Another case is that the client fails to hit the target cache object in LRU3 but hits the target cache object in LRU1 or LRU2, that is, the hash value of the key of the target cache object to be read in the Get request is not equal to the hash value hash of the cache object in LRU3 of CiCS, while the hash value of the key of the target cache object to be read in the Get request is equal to the hash value hash of the cache object in LRU1 or LRU2 of CiCS. Then the client will find the remote pointer of the target cache object in LRU1 or LRU2 of CiCS. Then the client directly initiates an RDMA_READ operation to read the complete target cache object (key length, value length, version, key, value, lock) in CiMS, and then the client asynchronously updates the timestamp in CiMS. Generally speaking, when CiCS hits, the client only needs one RDMA_READ operation to complete the Get operation, thus significantly reducing the latency and network bandwidth consumption of the cache system.

[0076] If CiCS misses, that is, the client queries CiCS according to the key of the target cache object to be read in the Get request and fails to find a matching hash value hash in LRU1 - LRU3. Then the client will first access the hash table in CiMS based on the key of the target cache object to be read in the Get request to find the address (pointer) of the target cache object. Based on this address (pointer), the client further queries the target cache object in CiMS. If the client finally finds the target cache object in CiMS, then the client will further insert the cache object into CiCS to ensure the consistency and timeliness of CiCS. If the client finally fails to find the target cache object, then it will read the data item corresponding to the target cache object from the underlying storage (such as a database), and then execute the Set request to cache it in the cache system, that is, insert it into CiMS.

[0077] Specifically, for the Set request, see Figure 5In (b) Put, if the CiCS cache hits, based on the remote pointer of the target cache object, the client initiates batch RDMA_CAS operation and RDMA_READ operation requests, and completes the locking and reading tasks of the target cache object in CiMS within one RTT (round-trip time). After successfully locking and obtaining the target cache object, it is judged whether the key of the target cache object locked in CiMS is the same as the key of the target cache object in the Set request. If they match, an RDMA_WRITE operation request is initiated to update the value and version in the target cache object locked in CiMS, and set the lock to the idle state. Finally, similar to Get, the latest timestamp is asynchronously written into the target slot in CiMS. After the client updates the cache object in CiMS, it does not actively update the cache object in CiCS. If the value and version of the corresponding cache object exist in the LRU3 of CiCS, they will become stale and will be verified as stale during the next Get operation, and the corresponding cache object in CiCS will be maintained by the next Get operation.

[0078] If the CiCS cache misses, it may be that the target cache object does not exist in CiMS or the target cache object exists in CiMS but has not been loaded into CiCS. For the former, a new cache object needs to be inserted into CiMS. For the latter, the hash table in CiMS needs to be remotely read first, and then the target cache object can be found. The insertion process of the cache object and the possible CiMS cache replacement lines are as described above.

[0079] As a further design of the present invention, a lightweight cache extension mechanism is designed, as Figure 6 shown. In the cache system of the embodiment of the present invention, the index structure (data organization structure) of CiMS is an extensible hash, that is, the essence of the expansion / contraction of CiMS is the increase / decrease of the hash sub-tables. For example, when the hash sub-table increases, a part of the data in the old sub-table will be migrated to the new sub-table. To solve the possible read / write / write concurrency conflicts during the migration process, atomic operations are usually used to modify the corresponding slots. However, this will introduce a large number of RDMA atomic operations, which may cause a serious decline in network performance.

[0080] For a cache system, it is mainly used to serve read-intensive loads and has no strong demand for write performance. Therefore, in the embodiments of the present invention, when the cache capacity is limited and the hit rate is low, the insertion operation in CiMS will repeatedly cause cache replacement operations due to insufficient hash slots. At this time, it is necessary to perform rehashing (resize) on the hash table in CiMS. During the rehashing (resize) of the cache system, the client does not perform cache object insertion operations on CiMS, thus avoiding the most troublesome concurrent conflicts. In addition, in the embodiments of the present invention, multiple background threads are used to batch migrate the slots in the hash sub-table, thereby greatly reducing the time required for resize without affecting the cache system to provide normal cache services.

[0081] Specifically, during the resize process, the cache system divides the hash sub-table to be migrated into different regions for different background threads to operate. Each background thread sequentially reads multiple consecutive hash buckets in the hash sub-table to be migrated and counts the slots that need to be migrated in each hash bucket. Then the background thread uses batch RDMA write operations to write the slots that need to be migrated into the corresponding hash buckets in the new hash sub-table. At the same time, the sig flag in the slots that have been migrated in the old hash bucket is set to 0xFF using RDMA write operations. Therefore, the client can still find the target data item through this slot during resize. After resize is completed, the values of the slots with sig being 0xFF have been migrated to the new sub-table, and the corresponding slots in the old sub-table will be regarded as invalid and thus recycled. Each background thread is responsible for a different area, so there is no impact on each other. The migration tasks of each background thread also have no conflict with the normal access of the original hash sub-table. Therefore, the entire hash resize process is a batch lock-free migration, so the efficiency is very high.

[0082] The downsizing process of CiMS is similar to the upsizing process, except that the slots to be migrated need to be inserted into the empty slots in the target hash bucket. If there are not enough empty slots, replacement will be performed according to the LRU algorithm. Specifically, each background thread sequentially reads multiple consecutive hash buckets in the hash sub-table to be migrated, and counts the slots that need to be migrated in each hash bucket. Then the background thread uses batch RDMA write operations to write the slots that need to be migrated into the target hash sub-table. If there are not enough empty slots in the target hash sub-table, first perform a replacement operation based on all the slots in the target hash sub-table using the LRU algorithm, removing the colder slots in the target hash sub-table. Then the background thread uses batch RDMA write operations to write the slots that need to be migrated into the corresponding hash buckets in the target hash sub-table. At the same time, use the RDMA write operation to set the sig flag in the migrated slots in the old hash bucket to 0xFF. So the client can still find the target data item through this slot during resize. After resize is completed, the values of the slots with sig being 0xFF have been migrated to the new sub-table, and the corresponding slots in the old sub-table will be regarded as invalid and thus recycled. Similar to upsizing, each background thread is responsible for a different area, so there is no impact on each other. The migration task of each background thread also has no conflict with the normal access to the original hash sub-table.

[0083] Generally speaking, the cache system makes full use of the ideas of RDMA batching and coalescing to achieve lightweight and efficient upsizing / downsizing.

[0084] The DM-based two-level scalable cache system of the present invention includes a small and efficient CiCS and a hash-based CiMS. The CiCS of the cache system accelerates the access to CiMS, and CiMS can accelerate the access to the underlying storage system (such as a database). In addition, the cache system can achieve dynamic expansion and elasticity according to actual needs, without suspending the cache service or obvious performance degradation.

[0085] Embodiment 2

[0086] The embodiment of the present invention provides a two-level scalable caching method for disaggregated memory, which is applied to the client in the two-level scalable caching system for disaggregated memory as described in Embodiment 2 above. The method includes:

[0087] When initiating a Get request for reading a target cache object, access the CiCS in its CS. If the target cache object is obtained in the CiCS or only the remote address pointer of the target cache object is obtained, the cache hits; otherwise, the cache misses.

[0088] When the cache hits, if the target cache object is obtained in CiCS, based on the remote address pointer of the target cache object, use the one-sided RDMA_READ operation to read the version number of the target cache object stored in CiMS, and determine whether the version number is consistent with the version number of the target cache object obtained from CiCS. If so, complete the Get request. If not, read the target cache object in CiMS based on the version number read from CiMS and complete the Get request; if the remote address pointer of the target cache object is obtained in CiCS, based on the remote address pointer, use the one-sided RDMA_READ operation to read the target cache object from CiMS and complete the Get request;

[0089] When the cache misses, if the target cache object can be directly read from CiMS, complete the Get request and insert it into the CiCS where the client is located; otherwise, read the target cache object from the lower storage layer, complete the Get request, and insert the read target cache object into CiMS.

[0090] When initiating a Set request for updating the target cache object, access CiCS in its corresponding CS. When the cache hits, based on the remote address pointer of the target cache object, use the RDMA_CAS operation to lock the target cache object stored in CiMS, read the target cache object in CiMS, and then use the RDMA_WRITE operation to complete the update and unlocking of the target cache object in CiMS;

[0091] When the cache misses, if the target cache object is read from CiMS, directly update the read target cache object in CiMS; if the target cache object is read from the lower storage layer, insert the read target cache object into CiMS.

[0092] The related technical solutions are the same as above and will not be elaborated here.

[0093] Embodiment 3

[0094] The embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the two-layer scalable cache method for separated memory in the above Embodiment 2.

[0095] Specifically, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0096] The related technical solutions are the same as above and will not be elaborated here.

[0097] Those skilled in the art can easily understand that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A two-layer scalable cache system for a separated memory, characterized in that Including: At least one computing server CS and a memory resource pool composed of at least one memory server MS, where each CS and the memory resource pool are interconnected through RDMA; among them, in each CS, a cache CiCS for sharing cache objects by all clients running in this CS is built, and in the memory resource pool, a cache CiMS for storing cache objects is built, and the clients running in all CSs are all used to access all cache objects stored in CiMS; When the client initiates a Get request for reading a target cache object, the client accesses CiCS in its corresponding CS. If the client obtains the target cache object or only obtains the remote address pointer of the target cache object in CiCS, the cache hits, otherwise the cache misses; When the cache hits, if the target cache object is obtained in CiCS, the client, based on the remote address pointer of the target cache object, uses the unilateral RDMA_READ operation to read the version number of the target cache object stored in CiMS, and determines whether the version number is consistent with the version number of the target cache object obtained from CiCS. If so, the Get request is completed. If not, the client reads the target cache object in CiMS based on the version number read from CiMS, and completes the Get request; if the remote address pointer of the target cache object is obtained in CiCS, the client, based on the remote address pointer, uses the unilateral RDMA_READ operation to read the target cache object from CiMS, and completes the Get request; When the cache misses, if the client can directly read the target cache object from CiMS, the Get request is completed and inserted into CiCS where the client is located; Otherwise, the client reads the target cache object from the lower storage layer, completes the Get request, and inserts the read target cache object into CiMS.

2. The two-layer scalable cache system for a disaggregated memory according to claim 1, wherein CiMS is a data structure based on extensible hashing; When the cache misses, the client first accesses the hash table in CiMS. If the target cache object is found in the hash table, the client can directly read the target cache object from CiMS. Otherwise, the client reads the target cache object from the lower storage layer.

3. The two-layer scalable cache system for a discrete memory according to claim 1 or 2, characterized in that, When the client initiates a Set request for updating a target cache object, the client accesses CiCS in its corresponding CS. When the cache hits, the client, based on the remote address pointer of the target cache object, uses the RDMA_CAS operation to lock the target cache object stored in CiMS, reads the target cache object in CiMS, and then uses the RDMA_WRITE operation to complete the update and unlocking of the target cache object in CiMS; When the cache misses, if the client reads the target cache object from CiMS, the target cache object read will be directly updated in CiMS; if the client reads the target cache object from the lower storage layer, the read target cache object will be inserted into CiMS.

4. The double-layer scalable cache system for a disaggregated memory according to claim 3, wherein Each CiCS includes three LRU caches, namely LRU1, LRU2, and LRU3. Among them, LRU1 and LRU2 contain the hash value of the cache object key, the remote address pointer of the corresponding cache object in CiMS, the length, and the local access frequency. LRU3 contains the hash value of the cache object key, the remote address pointer of the corresponding cache object in CiMS, the length, the local access frequency, the value of the corresponding cache object in CiMS, and the version number. When the client initiates the Get request and the cache misses, if the client can directly read the target cache object from CiMS, it inserts it into the CiCS where the client is located, including: If LRU1 is not full, insert the target cache object into LRU1. If LRU1 is full, the client inserts the coldest cache object in LRU1 into LRU2, and then inserts the target cache object into LRU1. The coldest cache object refers to the cache object at the tail of the LRU linked list. If LRU2 is also full at this time, the client compares the local access frequencies of the coldest cache object in LRU1 and the coldest cache object in LRU2, and discards the one with the smaller local access frequency. When the client initiates the Get request and the cache hits, if the remote address pointer of the target cache object is obtained in LRU1 and the target cache object is read from CiMS, the client updates the target cache object to the head of the LRU1 linked list. If the remote address pointer of the target cache object is obtained in LRU2 and the target cache object is read from CiMS, when LRU3 is not full, the client inserts the target cache object into LRU3. When LRU3 is full, the client first evicts the coldest cache object in LRU3, then inserts the target cache object into LRU3, and inserts the evicted coldest cache object in LRU3 into LRU2. If LRU2 is also full at this time, the client compares the local access frequencies of the evicted coldest cache object in LRU3 and the coldest cache object in LRU2, and discards the one with the smaller local access frequency. If the target cache object is obtained in LRU3, while the client reads the version number of the target cache object from CiMS according to the remote address pointer of the target cache object, it also includes reading the key of the target cache object from CiMS. When the key read from CiMS is the same as the key of the target cache object to be read in the Get request and the version numbers are the same, the Get request is completed, and the target cache object is updated to the head of the LRU3 queue. If the keys are different, the target cache object is evicted from LRU3. If the keys are the same but the version numbers are different, the value of the target cache object is deleted from LRU3, and it is evicted to LRU2. For the sudden cache access of the client, after the client obtains the target cache object from CiMS or from the lower storage layer, it inserts it into LRU1.

5. The two-layer scalable cache system for separated memory according to claim 4, wherein CiMS includes an extensible hash table and discrete cache objects; each hash sub-table of the extensible hash table includes a number of hash buckets, each hash bucket includes multiple hash slots, each hash slot corresponds to a memory block, and each memory block is used to store a cache object; each hash slot contains: a fingerprint fp for determining whether the hash slot may point to the target cache object, the size len of the cache object, the address pointer of the cache object, a flag bit sig, and the timestamp when the client last accessed the cache object. Inserting the read target cache object into CiMS includes: Determining the hash bucket where the target cache object is located in CiMS according to the hash value of the target cache object; When the hash bucket it is in is not full and there are empty memory blocks in CiMS, the client applies for an empty memory block, writes the target cache object into the memory block using the RDMA_WRITE operation, and then modifies the {fp, len, pointer} information in the hash slot corresponding to the memory block to the information corresponding to the target cache object using the RDMA_CAS operation; When the hash bucket it is in is not full but the number of memory blocks in CiMS is insufficient, the client first executes the cache replacement logic and then the insertion operation, specifically including: Maintaining the latest access timestamp of each hash bucket in CiMS in CS, and dividing the hash buckets into n groups according to the timestamp. Each group contains a batch of hash buckets with similar timestamps. The group with a larger timestamp indicates that the hash buckets in it are hotter. When performing cache replacement, the probability of evicting cache objects in hotter hash buckets is smaller; when performing cache replacement, the client selects the group of hash buckets with the smallest timestamp, randomly selects a hash bucket from it, evicts the coldest cache object in it, and reclaims the corresponding memory block; inserts the target cache object into the reclaimed memory block; When the hash bucket it is in is full, the client evicts the coldest cache object in the hash bucket and reclaims the corresponding memory block; inserts the target cache object into the reclaimed memory block.

6. The double-layer scalable cache system for a disaggregated memory according to claim 5, wherein The client executes the LRU algorithm based on the timestamps of the cache objects in the hash bucket to obtain the coldest cache object in the hash bucket.

7. The double-layer scalable cache system for a disaggregated memory according to claim 5, wherein It also includes the process of expanding or / and shrinking CiMS: rehashing the hash table in CiMS. During rehashing, the client does not perform the insertion operation of cache objects on CiMS; Rehashing the hash table in CiMS includes: Dividing the hash sub-tables to be migrated into different background threads for operation according to regions; each background thread sequentially reads multiple consecutive hash buckets in the hash sub-tables to be migrated and counts the hash slots that need to be migrated in each hash bucket; During the expansion process, the background thread uses the batch RDMA_WRITE operation to write the hash slots that need to be migrated into the corresponding hash buckets in the new hash sub-table; During the capacity reduction process, the background thread uses batch RDMA_WRITE operations to write the hash slots to be migrated into the corresponding hash buckets in the target hash sub-table; if there are not enough empty hash slots in the target hash sub-table, the colder cache objects in the target hash sub-table are first removed to recycle the corresponding hash slots; then the background thread uses batch RDMA_WRITE operations to write the hash slots to be migrated into the corresponding hash buckets in the target hash sub-table. During the process of writing to the new hash sub-table or the corresponding hash bucket of the target hash sub-table, the RDMA_WRITE operation is used to set the flag bit sig in the migrated hash slots in the old hash bucket to 0xFF, so that the client can still find the target cache object through this hash slot during the rehashing period; after the rehashing is completed, the values of the hash slots with the flag bit sig being 0xFF have been migrated to the new hash sub-table, and the corresponding hash slots in the old hash sub-table are recycled.

8. A two-layer scalable cache method for a discrete memory, characterized in that Applied to the client in the two-layer scalable cache system for separated memory as described in any one of claims 1-7, the method includes: When initiating a Get request for reading a target cache object, access the CiCS in its CS. If the target cache object or only the remote address pointer of the target cache object is obtained in the CiCS, the cache hits; otherwise, the cache misses. When the cache hits, if the target cache object is obtained in the CiCS, based on the remote address pointer of the target cache object, use the one-sided RDMA_READ operation to read the version number of the target cache object stored in the CiMS, and determine whether the version number is consistent with the version number of the target cache object obtained from the CiCS. If so, the Get request is completed; if not, based on the version number read from the CiMS, read the target cache object in the CiMS to complete the Get request; if the remote address pointer of the target cache object is obtained in the CiCS, based on the remote address pointer, use the one-sided RDMA_READ operation to read the target cache object from the CiMS to complete the Get request. When the cache misses, if the target cache object can be directly read from the CiMS, the Get request is completed and it is inserted into the CiCS where the client is located; otherwise, the target cache object is read from the lower storage layer, the Get request is completed, and the read target cache object is inserted into the CiMS.

9. The method according to claim 8, wherein When initiating a Set request for updating a target cache object, access the CiCS in its CS. When the cache hits, based on the remote address pointer of the target cache object, use the RDMA_CAS operation to lock the target cache object stored in the CiMS, read the target cache object in the CiMS, and then use the RDMA_WRITE operation to complete the update and unlocking of the target cache object in the CiMS. When the cache misses, if the target cache object is read from the CiMS, the target cache object read will be directly updated in the CiMS. If the target cache object is read from the lower storage layer, the read target cache object is inserted into the CiMS.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the two-layer scalable cache method for separated memory as described in claim 8 or 9.

Images