Cache replacement method and device, and system

Abstract

The present application relates to the field of computers, and provides a cache replacement method and device, and a system. A compute node in a distributed system sends to a memory node a first read operation request used for instructing to read access information of a plurality of data objects from the memory node; then, the compute node uses a first cache replacement algorithm among a plurality of cache replacement algorithms to respectively calculate the popularities of the plurality of data objects on the basis of the access information of the plurality of data objects; finally, when needing to perform cache replacement, on the basis of the popularities of the plurality of data objects, the compute node sends to the memory node a cache replacement request used for instructing to replace cold data in the plurality of data objects with a data object to be inserted into the memory node. In this way, the compute node executes a cache replacement algorithm on the basis of global access information, thereby realizing accurate cache replacement in the distributed system of a disaggregated memory architecture.

Description

Cache replacement method, device and system

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 28, 2023, with application number 202311090908.1 and application name “A method, device and other equipment for cache replacement” and the Chinese patent application filed with the State Intellectual Property Office on September 15, 2023, with application number 202311196479.6 and invention name “Cache replacement method, device and system”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computers, and in particular to a cache replacement method, device, and system. Background Art

[0003] A distributed system typically consists of multiple compute nodes (CNs) and memory nodes (MNs). Compute nodes access and operate on cached data objects in the memory nodes. When cache replacement is needed, the memory nodes use a cache replacement algorithm to replace the locally stored cached data.

[0004] However, in a distributed system with a disaggregated memory architecture, compute nodes typically directly access cached data stored in memory nodes. The memory node's processor cannot obtain information about the compute node's access to cached data. Therefore, without global access information for the distributed system, the memory node cannot accurately replace cached data that needs to be replaced.

[0005] Summary of the Invention

[0006] The present application provides a cache replacement method, device and system, thereby improving the overall cache hit rate of a distributed system.

[0007] In a first aspect, the present application provides a cache replacement method, which is applied to a computing node in a distributed system with a separated memory architecture. The distributed system also includes a memory node, and the cache data stored in the memory node includes multiple data objects. The computing node bypasses the processor (central processing unit, CPU) of the memory node when accessing the cache data stored in the memory node. The computing node sends a first read operation request to the memory node, and the first read operation request is used to indicate that access information of multiple data objects is read from the memory node. Then, the computing node uses the first cache replacement algorithm among multiple cache replacement algorithms to calculate the heat of multiple data objects based on the access information of multiple data objects. Finally, when the cache replacement of cached data is needed, the computing node sends a cache replacement request to the memory node according to the heat of the multiple data objects, and the cache replacement request is used to indicate that cold data in the multiple data objects is replaced with the data object to be inserted into the memory node.

[0008] The cold data may be one or more data objects whose heat is less than a heat threshold among multiple data objects, or the first n data objects after sorting multiple data objects from low to high heat, where n is a positive integer.

[0009] Based on the above cache replacement method, the compute node obtains global access information of the distributed system based on the metadata of multiple candidate data objects in the cache data. This allows the compute node to select a first cache replacement algorithm based on the global access information to perform cache replacement on the cached data stored by the memory node. In this way, the compute node executes the cache replacement algorithm based on the global access information, achieving accurate cache replacement in a distributed system with a split memory architecture.

[0010] As one possible implementation, a memory node stores metadata for cached data. When the cached data includes multiple data objects, the metadata for the cached data includes metadata for each of the multiple data objects. For example, if the multiple data objects include a first data object, the metadata for the first data object includes access information regarding the first data object. After the compute node accesses the first data object, it sends a metadata update request to the memory node. The metadata update request indicates an update to the access information for the first data object.

[0011] Based on the above implementation method, compared to the memory node that cannot perceive the computing node's access information to the cached data when executing the cache replacement algorithm, the memory node maintains the global access information of the distributed system based on data access, so that the computing node can obtain effective global access information from the memory node to execute the cache replacement algorithm.

[0012] Optionally, the access information includes a last access timestamp and a number of accesses. For example, the access information for the metadata of a first data object includes a last access timestamp and a number of accesses to the first data object. In this way, after obtaining the access information for the data object, the computing node can use a cache replacement algorithm to accurately calculate the cold data that needs to be replaced from the memory node based on the last access timestamp and the number of accesses.

[0013] As a possible implementation, the metadata of cached data stored in the memory node is stored in a hash table format. The hash table includes multiple hash slots, each of which is used to store the metadata of a data object. Each hash slot includes a length field, a pointer field, a last access timestamp field, and an access count field.

[0014] The length field indicates the length of the data object to which the metadata belongs. The pointer field indicates the address of the data object to which the metadata belongs in the memory node. The last access timestamp field indicates the time when the data object to which the metadata belongs was last accessed. The access count field indicates the number of times the data object to which the metadata belongs has been accessed.

[0015] Optionally, the hash table includes multiple hash buckets, and each hash bucket includes multiple hash slots.

[0016] Optionally, each hash slot can be divided into an atomic field and a metadata field. The atomic field may include an object identification field, a length field, and a pointer field. The object identification field is used to indicate the identifier of the data object to which the metadata belongs. The metadata field may include an access timestamp field, an access count field, a data content field, and an access count field. The data content field is used to indicate the hash value of the data content of the data object to which the metadata belongs. The access count field is used to indicate the number of times the data object to which the metadata belongs has been accessed.

[0017] Based on the above implementation method, the memory node stores the metadata of the cached data in the format of a hash table. The computing node can quickly query the metadata of the data object based on the hash table, and it is convenient for the computing node to perform subsequent operations on the metadata, thereby improving the overall data access efficiency of the distributed system.

[0018] As one possible implementation, the metadata update request includes a first write operation request and a first atomic count operation request. The first write operation request is used to instruct the value of the last access timestamp field in the metadata of the second data object to be set to the timestamp when the compute node initiated the data access. The first atomic count operation request is used to instruct the value of the access count field to be incremented by one. In this way, the compute node operates on the metadata of the data object through atomic operations, ensuring overall data consistency in the distributed system.

[0019] As a possible implementation, after replacing cold data from a memory node, the computing node may also convert the metadata of the cold data into a historical record item, providing a data basis for subsequent detection of whether different cache replacement algorithms cause access misses. Taking the example of cold data including a second data object, after replacing the second data object from the memory node, the computing node sends a historical record item generation request to the memory node. The historical record item generation request is used to instruct the metadata of the second data object to be converted into a historical record item. The historical record item is used to indicate the second cache replacement algorithm used when the second data object is cache replaced.

[0020] Optionally, similar to metadata, a history entry is stored in a hash slot of a hash table. The hash slot includes a type field and an algorithm identifier field. The type field indicates that the data object to which the history entry belongs has been replaced by the cache. The algorithm identifier indicates a second cache replacement algorithm to be used when the data object to which the history entry belongs is replaced by the cache.

[0021] Optionally, the historical record item is converted from the metadata of the data object. The historical record item generation request includes an atomic exchange operation request and a second write operation request, the atomic exchange operation request is used to indicate that the value of the length field of the metadata of the second data object is modified to the value of the type field, and the second write operation request is used to indicate that the value of a field used to represent access information in the metadata of the second data object is replaced with the value of the second cache replacement algorithm used when the second data object is replaced by the cache. In this way, the computing node can convert the metadata of the object data into historical data items through the historical record item generation request, thereby realizing the reuse of the hash table and reducing the consumption of storage resources of the memory node.

[0022] As a possible implementation method, when the computing node misses accessing the second data object stored in the memory node, it determines that the memory node stores a historical record item of the second data object. Then, the computing node sends a second read operation request to the memory node. The second read operation request is used to read the historical record item of the second data object from the memory node, and the value of the algorithm identification field of the historical record item of the second data object indicates the second cache replacement algorithm. Finally, the computing node reduces the weight of the second cache replacement algorithm. In this way, the computing node determines the cache replacement algorithm to be used when replacing the data object based on the historical record item of the data object that was erroneously replaced out of the memory node, thereby dynamically adjusting the weights of multiple cache replacement algorithms according to the data access pattern, so that the computing node can select a cache replacement algorithm suitable for the current data access pattern based on the weights of multiple cache replacement algorithms, thereby reducing the probability of replacing cache data that the computing node needs to access out of the memory node, thereby improving the overall cache hit rate of the distributed system.

[0023] Optionally, after reducing the weight of the second cache replacement algorithm, the computing node sends a third write operation request to the memory node. The third write operation request is used to instruct the memory node to write the reduced weight of the second cache replacement algorithm to the memory node, so that the memory node merges the reduced weight of the second cache replacement algorithm into the global algorithm weight and returns the merged global algorithm weight to the computing node. In this way, when the memory node is unable to perceive global data access information, the global update of the algorithm weights in the distributed system is achieved, ensuring the accuracy of the weights of multiple cache replacement algorithms recorded by the computing node.

[0024] As a possible implementation method, when the remaining storage space of the memory node is less than the size of the data object to be inserted, or when the remaining storage space of the memory node is less than a preset threshold, the computing node determines that the cached data needs to be replaced and sends a cache replacement request to the memory node.

[0025] In a second aspect, the present application provides a cache replacement method, which is applied to computing nodes and memory nodes in a distributed system with a separated memory architecture. The cache data stored in the memory node includes multiple data objects, and the computing node bypasses the processor of the memory node when accessing the cache data stored in the memory node. The computing node sends a first read operation request to the memory node. The memory node returns access information of multiple data objects to the computing node based on the first read operation request. Then, the computing node uses the first cache replacement algorithm among multiple cache replacement algorithms to calculate the heat of multiple data objects based on the access information of multiple data objects. Finally, when the cache data needs to be replaced, the computing node sends a cache replacement request to the memory node based on the heat of the multiple data objects. The cache replacement request is used to indicate that cold data in multiple data objects is replaced with data objects to be inserted into the memory node.

[0026] In a third aspect, the present application provides a cache replacement device, comprising a transceiver module and a processing module. The transceiver module is used to send a first read operation request to a memory node, and the first read operation request is used to read access information of multiple data objects from the memory node, and the cache data stored in the memory node includes multiple data objects. The processing module is used to use the first cache replacement algorithm among multiple cache replacement algorithms to calculate the heat of multiple data objects based on the access information of multiple data objects. The transceiver module is also used to send a cache replacement request to the memory node according to the heat of multiple data objects when cache replacement of cache data is required, and the cache replacement request is used to indicate that cold data in multiple data objects is replaced with data objects to be inserted into the memory node.

[0027] As a possible implementation manner, the cache replacement device may further include other modules that execute the operation steps of the cache replacement method described in the first aspect.

[0028] In a fourth aspect, the present application provides a cache replacement system, comprising a client and a server. The client is used to send a first read operation request to a memory node, and the first read operation request is used to read access information of multiple data objects from the memory node, and the cache data stored in the memory node includes multiple data objects. The server is used to return access information to the client according to the first read operation request. The client is also used to calculate the heat of multiple data objects respectively based on the access information of multiple data objects using the first cache replacement algorithm among multiple cache replacement algorithms. The client is also used to send a cache replacement request to the memory node according to the heat of multiple data objects when cache replacement of cached data is required, and the cache replacement request is used to indicate that cold data in multiple data objects is replaced with data objects to be inserted into the memory node.

[0029] Regarding the technical principles and beneficial effects of the second, third and fourth aspects, please refer to the relevant description of the first aspect mentioned above and will not be repeated here.

[0030] In a fifth aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the cache replacement method described in any possible implementation of the first aspect.

[0031] In a sixth aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the cache replacement method described in any possible implementation of the second aspect.

[0032] In a seventh aspect, the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed on a computer, causes the computer to execute the cache replacement method described in any possible implementation of the first aspect.

[0033] In an eighth aspect, the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed on a computer, causes the computer to execute the cache replacement method described in any possible implementation of the second aspect.

[0034] In a ninth aspect, the present application provides a computer-readable storage medium comprising a computer program or instructions that, when executed on a computer, causes the computer to execute the cache replacement method described in any possible implementation of the first aspect.

[0035] In a tenth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes a computer program or instructions that, when executed on a computer, causes the computer to execute the cache replacement method described in any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG1 is a schematic diagram of the architecture of a distributed system provided by this application;

[0037] FIG2 is a schematic diagram of a flow chart of a cache replacement method provided by the present application;

[0038] FIG3 is a schematic diagram of the structure of a hash table provided in this application;

[0039] FIG4 is a flow chart of an access information update step provided by the present application;

[0040] FIG5 is a schematic diagram of the structure of a historical record entry provided by this application;

[0041] FIG6 is a flow chart of steps for converting a historical record item provided by the present application;

[0042] FIG7 is a flow chart of a priority adjustment step of a cache replacement algorithm provided by the present application;

[0043] FIG8 is a schematic structural diagram of a cache replacement device provided by the present application;

[0044] FIG9 is a schematic structural diagram of another cache replacement device provided by the present application;

[0045] FIG10 is a schematic structural diagram of a cache replacement system provided by the present application;

[0046] FIG11 is a schematic diagram of the structure of a computing device provided by the present application;

[0047] FIG12 is a schematic diagram of the structure of a computing device cluster provided in this application. DETAILED DESCRIPTION

[0048] The cache replacement method, device, and system provided in the embodiments of the present application can be applied to cache replacement scenarios in distributed systems with a separate memory architecture. A brief introduction to the technologies that may be involved in the present application is given.

[0049] (1) Distributed systems

[0050] A distributed system, also known as a distributed computer system, is a system composed of multiple discrete computing devices connected by communication lines. The various system functions (such as processing, control, and storage) are distributed across these computing devices. Distributed systems, by function, include distributed storage systems, distributed computing systems, distributed message queue systems, and distributed machine learning systems.

[0051] A distributed storage system is one that stores data across multiple independent storage nodes. Distributed network storage systems employ a scalable architecture, utilizing multiple storage nodes to share the storage load. This not only improves the reliability, availability, and access efficiency of the storage system, but also facilitates scalability.

[0052] (2) Separate memory architecture

[0053] A split memory architecture is a distributed system architecture that physically separates the processor and memory located on the same motherboard in traditional architectures, creating two independent resource pools: a compute pool and a memory pool. The compute pool contains multiple compute nodes, each of which contains a large amount of compute resources (such as processor (Central Processing Unit, CPU) cores) and a small amount of storage resources (such as dynamic random access memory (DRAM)). The storage resources in the compute nodes are used as runtime cache. The memory pool contains a large number of memory nodes, each of which contains a large amount of storage resources and a small amount of compute resources. The compute resources of the memory nodes are used for management tasks, such as network connection management and memory space management, while the storage resources are used for data storage.

[0054] The compute pool and memory pool are interconnected over a network, allowing compute nodes to directly access data on memory nodes without going through the memory pool's processors, effectively improving the resource utilization and elasticity of the cache system. The compute pool and memory pool can be connected via Remote Direct Memory Access (RDMA) or Remote Procedure Call (RPC).

[0055] (3) Cache replacement algorithm

[0056] A cache replacement algorithm, also known as a cache algorithm, is used in cache devices with limited storage space (such as memory nodes in a distributed system under a separated memory architecture). Its goal is to make the limited cache space store as much frequently accessed hot data as possible to maximize the cache hit rate. A cache replacement algorithm generally consists of three steps. First, the cache replacement algorithm records all access information to data objects in the cache space, such as access timestamps and access counts. Second, the cache replacement algorithm determines the popularity of data objects in the cache space based on the access information in the record. Finally, the cache replacement algorithm maintains the popularity of data objects in a specific cache data structure to facilitate rapid cache replacement of cold data when the cache space is full.

[0057] Cache replacement algorithms include the least recently used (LRU) algorithm, the least frequently used (LFU) algorithm, the random replacement algorithm, the first in first out (FIFO) algorithm, the size replacement algorithm, and the like.

[0058] The present application provides a cache replacement method, in particular, a cache replacement method in which a computing node selects a cache replacement algorithm that conforms to the current data access pattern. In a distributed system with a separated memory architecture, a computing node sends a first read operation request to a memory node, and the first read operation request is used to read access information of multiple data objects in cached data from the memory node. Based on the access information of multiple data objects, the computing node uses the first cache replacement algorithm among multiple cache replacement algorithms to respectively calculate the heat of multiple data objects. When the computing node needs to perform cache replacement on the cached data, it sends a cache replacement request to the memory node based on the heat of the multiple data objects, and the cache replacement request is used to instruct to replace the cold data in the multiple data objects out of the memory node. In this way, the computing node obtains the global access information of the distributed system based on the metadata of the multiple candidate data objects in the cached data, so that it can select the first cache replacement algorithm according to the global access information to perform cache replacement on the cached data stored in the memory node. The computing node executes the cache replacement algorithm according to the global access information, thereby realizing accurate cache replacement in the distributed system with a separated memory architecture.

[0059] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0060] As shown in FIG1 , the distributed system 100 provided in the present application includes a computing node cluster and a memory node cluster. The computing node cluster constitutes a computing pool, and the memory node cluster constitutes a memory pool.

[0061] A computing node cluster includes one or more computing nodes 110 (three computing nodes 110 are shown in FIG1 , but are not limited to three computing nodes 110). Each computing node 110 can communicate with each other. A computing node 110 is a computing device, such as a server, desktop computer, or storage array controller. In terms of hardware, as shown in FIG1 , a computing node 110 includes at least a processor 112, memory 113, and a network interface card 114. Processor 112 is a central processing unit (CPU) that processes data access requests from outside the computing node 110 or requests generated within the computing node 110. For example, when processor 112 receives write data requests from users, it temporarily stores the data in these write data requests in memory 113. When the total amount of data in memory 113 reaches a certain threshold, processor 112 sends the data stored in memory 113 to memory node 120 for persistent storage. Processor 112 is also used for data computation or processing, such as metadata management, data deduplication, data compression, virtualized storage space, and address translation. Figure 1 shows only one CPU 112. In actual applications, there are often multiple CPUs 112, wherein one CPU 112 has one or more CPU cores. This embodiment does not limit the number of CPUs or CPU cores.

[0062] Memory 113 refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a high speed, and serves as a temporary data storage for the operating system or other running programs. Memory can include multiple types of memory. For example, memory can be either random access memory (RAM) or read-only memory (ROM). For example, random access memory is dynamic random access memory or storage class memory (SCM). However, DRAM and SCM are only exemplary in this embodiment, and memory can also include other random access memories, such as static random access memory (SRAM). As for read-only memory, for example, it can be programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), etc. In addition, memory 113 can also be a dual in-line memory module or a dual in-line memory module (DIMM), that is, a module composed of dynamic random access memory. In practical applications, multiple memories 113 can be configured in a computing node 110, and different types of memories 113 can be configured. Furthermore, memories 113 can be configured to have a power-saving feature. This feature ensures that data stored in memories 113 will not be lost even if the system loses power and then powers back on. Memory with this power-saving feature is called non-volatile memory.

[0063] The network card 114 is used to communicate with the memory node 120. For example, when the total amount of data in the memory 113 reaches a certain threshold, the computing node 110 can send a request to the memory node 120 through the network card 114 to persistently store the data. In addition, the computing node 110 can also include a bus for communication between the components inside the computing node 110. Functionally, since the main function of the computing node 110 in Figure 1 is computing business, remote storage can be used to achieve persistent storage when storing data, so it has less local storage than a conventional server, thereby saving cost and space. However, this does not mean that the computing node 110 cannot have local storage. In actual implementation, the computing node 110 can also have a small number of hard disks built in, or a small number of external hard disks.

[0064] Any computing node 110 can access any memory node 120 in the memory node cluster through the network. The memory node cluster includes multiple memory nodes 120 (three memory nodes 120 are shown in Figure 1, but are not limited to three memory nodes 120). A memory node 120 includes one or more processors 121, a network card 124 and multiple memories 125. The network card 124 is used to communicate with the computing node 110. The memory 125 is used to store data, such as cache data of the computing node 110. The memory 125 is similar to the above-mentioned memory 113 and will not be repeated here. The processor 121 is used to perform management tasks, such as network connection management and memory space management.

[0065] The distributed system 100 of the present application adopts a split memory architecture, whereby the compute node 110 bypasses the processor of the memory node 120 when accessing cached data stored in the memory node 120. For example, the compute node 110 directly accesses data on the memory node 120 through remote direct data access, remote procedure call protocol, Compute Express Link (CXL), etc., without passing through the processor 121 of the memory node 120. Therefore, compared with the memory node 120, the compute node 110 of the distributed system 100 can include a large number of CPU cores and a small amount of DRAM, while the memory node 120 can include a large number of DRAM and a small number of CPU cores.

[0066] When the computing node 110 directly accesses the data on the memory node 120 through remote direct data access, remote procedure call protocol, etc., compared with the traditional access control method by the processor of the memory node 120, the function of the processor is offloaded to the network card 114. The network card 124 completes data reading and writing, address conversion and other computing functions. At this time, the network card 124 is an intelligent network card, and the corresponding network card 114 is also an intelligent network card, such as an intelligent network card that supports RDMA. The network card 124 can include a processor, such as a CPU. The CPU is used to perform operations such as address conversion and reading and writing data. The processor of the network card 124 can also be a programmable electronic component, such as a data processing unit (DPU).

[0067] The computing nodes, memory nodes, etc. in the distributed system 100 provided in this application can be hardware devices such as computing servers and storage servers, or nodes in a cloud platform. For example, the distributed system 100 implements the functions of computing nodes, memory nodes, etc. on infrastructure such as computing server clusters and storage server clusters based on infrastructure as a service, platform as a service, and software as a service, and provides services (such as computing services, storage services, and network services) through virtualized cloud nodes.

[0068] Next, the cache replacement method provided by this application is described in detail with reference to the accompanying drawings.

[0069] The steps of the cache replacement method provided in this application are executed by the computing node 110 and the memory node 120 in the distributed system 100. Next, steps 210 to 230 of the cache replacement method are described in conjunction with Figure 2.

[0070] Step 210 : The computing node 110 sends a first read operation request to the memory node 120 .

[0071] As a possible example, computing node 110 sends a first read operation request to memory node 120. The first read operation request is used to read access information of multiple data objects from memory node 120. For example, the first read operation request is an RDMA read operation request. The multiple data objects may be randomly sampled.

[0072] Optionally, the access information includes a last access timestamp and a number of accesses. The last access timestamp indicates the time when the data object to which the access information belongs was last accessed. The number of accesses indicates the number of times the data object to which the access information belongs has been accessed.

[0073] This application limits the type of data included in access information. Depending on the type of data required by the cache replacement algorithm, access information can also include any type of data. For example, access information can also include insertion timestamps, object sizes, access interval durations, access overhead, etc.

[0074] In an embodiment of the present application, the access information is stored in the form of metadata in the memory node 120. In order to ensure the operational efficiency of the access information and reduce the space occupied by the access information, the format of the metadata can be a hash table.

[0075] As shown in FIG3 , a hash table 300 includes multiple hash buckets 301, each of which includes multiple hash slots 302 (only four are shown in FIG3 , but the number is not limited to four). Each hash slot 302 includes an atomic field and a metadata field. The atomic field includes an object identification field 303, a length field 304, and a pointer field 305, and the metadata field includes a last access timestamp field 306 and an access count field 307.

[0076] A hash slot 302 is used to store metadata for an object. The value stored in the object identifier field 303 indicates the data object to which the metadata belongs, and is called an object identifier. For example, if the data object is a key-value pair, the object identifier field 303 is used to store the key.

[0077] The value stored in the length field 304 is used to indicate the length of the data object to which the metadata belongs. The length of the data object can also be a type of access information, that is, the object size mentioned above.

[0078] The value stored in the pointer field 305 is used to indicate the address of the data object to which the metadata belongs in the memory node 120 .

[0079] The value stored in the last access timestamp field 306 is used to indicate the timestamp when the data object to which the metadata belongs was last accessed.

[0080] The value stored in the access count field 307 is used to indicate the number of times the data object to which the metadata belongs has been accessed.

[0081] This application does not limit the field types included in the hash slot 302. Depending on the type of data included in the access information, the hash slot 302 can also add or delete fields. For example, the metadata field also includes a hash value field 308 and an insertion timestamp field 309.

[0082] The value stored in the hash value field 308 is the hash value of the data object to which the metadata belongs.

[0083] The value stored in the insertion timestamp field 309 is used to indicate the timestamp when the data object to which the metadata belongs is inserted into the memory node 120 .

[0084] Based on the above-mentioned hash table 300, the computing node 110 sends a first read operation request to the memory node 120, which may carry an object identifier, and is used to query the value of the object identifier field 303 in the hash table 300 based on the object identifier, and read the values ​​of the last access timestamp field 306 and the access count field 307 in the hash slot 302.

[0085] This application does not limit the length of each field in the hash table 300. For example, the length of the object identification field 303 is 1 bit, the length of the length field 304 is 1 bit, the length of the pointer field 305 is 6 bits, and the length of the last access timestamp field 306, the number of accesses field 307, the hash value field 308 and the insertion timestamp field 309 are all 8 bits.

[0086] In a possible embodiment of the present application, the access information is instantaneous, that is, the computing node 110 needs to update the access information of the data object according to the access to the data object. The steps for computing node 110 to update the access information are shown in steps 410 to 430 of FIG. 4 below, and will not be repeated here.

[0087] Step 220 : The computing node 110 calculates the heat of the multiple data objects respectively using a first cache replacement algorithm from among the multiple cache replacement algorithms based on the access information of the multiple data objects.

[0088] As a possible example, the computing node 110 respectively substitutes the access information of the multiple data objects into the first cache replacement algorithm to respectively calculate the heat of the multiple data objects.

[0089] As one possible implementation, computing node 110 locally maintains multiple cache replacement algorithms and their priorities. Computing node 110 selects the cache replacement algorithm with the highest priority among the multiple cache replacement algorithms as the first cache replacement algorithm. The higher the priority of a cache replacement algorithm, the more suitable it is for the current data access pattern. The probability that a data object replaced by the first cache replacement algorithm will be frequently accessed is lower, which means that the probability of subsequent data object access misses is lower.

[0090] The priority of the cache replacement algorithm is adjusted in real time based on data access during the operation of the distributed system 100. For example, the priority of the cache replacement algorithm is adjusted in real time based on a historical record entry of an access miss occurring when a data object is incorrectly replaced using the cache replacement algorithm. The historical record entry in the embodiment of the present application can be obtained by converting metadata and is used to indicate the cache replacement algorithm used when the corresponding data object is replaced by the cache. For the specific data structure of the historical record entry, please refer to FIG. 5 and the related description below. For the conversion steps of the historical record entry, please refer to steps 610-620 shown in FIG. 6 below, which will not be repeated here.

[0091] Optionally, the multiple cache replacement algorithms maintained by the computing node 110 may include a least recently used algorithm, a least frequently used algorithm, a random replacement algorithm, a first-in-first-out algorithm, a length replacement algorithm, and the like.

[0092] In this embodiment, the specific steps of the computing node 110 adjusting the priority of the cache replacement algorithm based on the above historical record items are shown in steps 710 to 750 shown in FIG. 7 , which will not be repeated here.

[0093] Compared to having the cache replacement algorithm executed by memory node 120, having the compute node 110 execute the cache replacement algorithm avoids occupying the computing resources of memory node 120, thereby preventing data access throughput and latency from being affected by insufficient computing power of memory node 120. Furthermore, the compute node is able to perceive global access information and select a cache replacement algorithm appropriate for the current data access pattern. Using this global access information for cache replacement reduces the probability of cached data that the compute node needs to access being replaced out of the memory node, thereby improving the overall cache hit rate of the distributed system.

[0094] Step 230 : When the computing node 110 needs to perform cache replacement on cached data, it sends a cache replacement request to the memory node according to the popularity of multiple data objects.

[0095] As one possible example, computing node 110 calculates the heat of multiple data objects using a first cache replacement algorithm, then determines cold data among the multiple data objects based on the heat of the multiple data objects, and sends a cache replacement request based on the cold data to memory node 120. The cache replacement request is used to instruct the cold data among the multiple data objects to be replaced with the data object to be inserted into memory node 120.

[0096] As a first possible implementation method, when the computing node 110 inserts the data object to be inserted into the memory node 120, if it is determined that the remaining storage space of the memory node 120 is less than the size of the data object to be inserted, it is determined that the cache data stored in the memory node 120 needs to be cached.

[0097] As a second possible implementation, when the time elapsed after the last cache replacement of the cache data stored in the memory node 120 reaches a threshold, the computing node 110 determines that the cache replacement of the cache data stored in the memory node 120 needs to be performed.

[0098] As a third possible implementation manner, when the remaining space of the memory node 120 is less than a threshold, the computing node 110 determines that cache replacement of the cache data stored in the memory node 120 is required.

[0099] Optionally, the cold data is the data object with the lowest heat among the multiple data objects.

[0100] Optionally, the cold data is the top n data objects among multiple data objects sorted from low to high in terms of popularity, where n is a positive integer.

[0101] In the above two cold data determination methods, the step of the computing device 110 determining the cold data according to the heat of multiple data objects may be integrated into the cache replacement algorithm.

[0102] Optionally, the cache replacement request is an RDMA write operation request.

[0103] In an embodiment of the present application, compute node 110 bypasses processor 121 of memory node 120 when accessing cached data stored in memory node 120. Processor 121 of memory node 120 does not perform operations related to data access and only provides data storage functions, such as returning access information of multiple data objects to memory node 120 in response to a first read operation request. Therefore, FIG2 does not separately illustrate the operations performed by memory node 120. In other possible embodiments, the operations performed by memory node 120 can be presented as interactive operations with compute node 110. For example, between step 210 and step 220, the following step is further included: memory node 120 returns access information of multiple data objects to compute node 110 in response to the first read operation request.

[0104] Based on steps 210-230, the compute node obtains global access information for the distributed system based on the metadata of multiple candidate data objects in the cached data. This allows the compute node to select a first cache replacement algorithm based on the global access information to perform cache replacement on the cached data stored in the memory node. In this manner, the compute node executes the cache replacement algorithm based on the global access information, achieving accurate cache replacement in a distributed system with a split memory architecture.

[0105] The cache replacement algorithm provided in this application is generally described above based on Figures 2 and 3. The following describes in detail the steps for computing node 110 to update access information in conjunction with Figure 4. As shown in Figure 4, the steps for computing node 110 to update access information may include the following steps 410-430.

[0106] Step 410 : The computing node 110 sends a third read operation request to the memory node 120 .

[0107] As a possible example, when accessing data, the computing node 110 sends a third read operation request to the memory node 120 , indicating that the address of the first data object in the memory node 120 is to be read from the memory node 120 .

[0108] As a possible implementation, the metadata of the first data object is stored in the memory node 120 in the form of a hash table, and the third read operation request is used to instruct to read the value of the pointer field in the hash slot indicated by the object identifier from the memory node 120.

[0109] Optionally, when accessing data, the computing node 110 calculates a hash value based on the object identifier of the data object to be accessed, locates a hash bucket based on the hash value, and the third read operation request is used to read the value of the located hash bucket. For example, the third read operation request is an RDMA read operation request.

[0110] After reading the value of the located hash bucket, the computing node 110 traverses and queries the value of the pointer field of the first data object in the hash bucket according to the object identifier of the first data object.

[0111] Step 420 : The computing node 110 sends a fourth read operation request to the memory node 120 .

[0112] As a possible example, computing node 110 sends a fourth read operation request to memory node 120 based on the address of the first data object in memory node 120. The fourth read operation request carries the address of the first data object in memory node 120 and is used to instruct to read the data object from memory node 120 based on the address of the first data object in memory node 120.

[0113] As a possible implementation method, the metadata of the first data object is stored in the memory node 120 in the form of a hash table, and the address of the first data object in the memory node 120 carried by the fourth read operation request is the value of the pointer field of the metadata of the first data object.

[0114] Optionally, the fourth read operation request is an RDMA read operation request.

[0115] Step 430 : After accessing the first data object, the computing node 110 sends a metadata update request to the memory node 120 .

[0116] As a possible example, after accessing data of the first data object, the computing node 110 sends a metadata update request to the memory node 120 to instruct to update the access information of the first data object.

[0117] As a possible implementation, the metadata update request includes a first write operation request and a first atomic count operation request. Optionally, the first write operation request is an RDMA write operation request, and the first atomic count operation request is an RDMA fetch_and_add operation request.

[0118] Taking the example of metadata stored in memory node 120 in the form of a hash table, the first write operation request is used to instruct the value of the last access timestamp field in the metadata of the first data object to be set to the timestamp when computing node 110 initiated the data access, that is, the timestamp when computing node 110 sent the third read operation request. The first atomic count operation request is used to instruct the value of the access count field in the metadata of the first data object to be increased by one.

[0119] The above steps 410 to 430 are based on an example in which the computing node 110 performs a read operation on the memory node 120. In other possible embodiments, the computing node 110 may access the data object through a set operation, etc., which will not be repeated here.

[0120] Based on the above steps 410 to 430 , the computing node 110 realizes global perception of access to the data object and global update of access information, which provides a data basis for subsequent calculation of the popularity of the data object based on the access information of the data object.

[0121] In an embodiment of the present application, in addition to updating the metadata of the data object based on access to the data object, the computing node 110 can also transform the metadata of the data object replaced out of the memory node 120 after the data object is replaced out of the memory node 120 to obtain a historical record item to record the cache replacement algorithm used when the data object is replaced out of the memory node 120.

[0122] In the embodiment of the present application, in order to reduce the space occupied by the historical record items in the memory node 120 and the index overhead, the historical record items can be stored in a multiplexed hash table. First, the structure of the historical record items will be described with reference to FIG5 .

[0123] As shown in Figure 5, when a hash slot for storing metadata is used to store a history item, the hash slot includes an atomic field and a metadata field. The atomic field includes an object identification field 501, a type field 502, and a history item identification field 503. The metadata field includes a hash value field 504, an algorithm identification field 505, a last access timestamp field 506, and a number of accesses field 507.

[0124] A hash slot is used to store a historical data item of object data. The value stored in the object identification field 501 is used to indicate the data object to which the historical data item belongs, which is called the object identification. For example, if the data object is a key-value pair, the object identification field 501 is used to store the key.

[0125] The value stored in the type field 502 is used to indicate that the hash slot stores a historical record item. For example, the value of the type field 502 is 0xFF.

[0126] The value stored in the history record item identification field 503 is used to indicate the position of the history record item in the logical first-in-first-out queue.

[0127] The value stored in algorithm identification field 505 indicates the cache replacement algorithm used when the data object to which the historical record item belongs is cache-replaced. For example, the value stored in algorithm identification field 505 is a bitmap, where different bits of the bitmap correspond to different cache replacement algorithms. When the bit in the bitmap corresponding to the second cache replacement algorithm is a preset value, the value of algorithm identification field 505 indicates the second cache replacement algorithm.

[0128] The hash value field 504 , the last access timestamp field 506 , and the access count field 507 are the same as the corresponding fields in the metadata shown in FIG. 3 , and are not described in detail here.

[0129] Next, the steps for converting metadata and historical records are described in conjunction with FIG6 . The computing node 110 sends a historical record item generation request to the memory node 120 to execute the steps for converting metadata and historical records. Based on the data format of the historical records, the historical record item generation request sent by the computing node 110 to the memory node 120 may include an atomic swap operation request and a second write operation request. The second write operation request may be an RDMA write operation request, and the atomic swap operation request may be an RDMA compare_and_swap operation request. As shown in FIG6 , the steps for converting metadata and historical records may include the following steps 610 to 620.

[0130] Step 610: The computing node 110 sends an atomic swap operation request to the memory node 120.

[0131] As a possible example, after completing cache replacement for the second data object, computing node 110 sends an atomic swap operation request to memory node 120. The atomic swap operation request is used to instruct to modify the value of the length field of the metadata of the second data object to the value of the type field, such as 0xFF.

[0132] Step 620 : The computing node 110 sends a second write operation request to the memory node 120 .

[0133] As a possible example, the computing node 110 sends a second write operation request to the memory node 120, where the second write operation request is used to indicate that the value of a field in the metadata of the second data object used to represent access information is replaced with a value, such as a bitmap, used to represent the cache replacement algorithm used when the second data object is replaced by the cache. The replaced field is called the algorithm identification field 505.

[0134] As a possible embodiment, the field in the metadata to be replaced by the second write operation request may be any one of the last access timestamp field 306 , the access count field 307 , and the insertion timestamp field 309 .

[0135] In a possible embodiment of the present application, when the historical record item includes the historical record item identification field 503 , the above-mentioned step of converting metadata to the historical record item also requires converting the metadata pointer field 305 into the historical record item identification field 503 .

[0136] As a possible implementation method, the computing node 110 sends a second atomic counting operation request to the memory node 120 before step 610. The second atomic counting operation request is used to indicate that the number of global history items is increased by one, and the value before the number of global history items is increased by one is returned as the value of the history item identification field 503.

[0137] In this way, the computing node 110 sends the atomic swap operation request to the memory node 120 in step 610 and is also used to modify the value of the pointer field of the metadata of the second data object to the value of the historical record item identification field 503.

[0138] The function of the history record item identification field 503 is described below.

[0139] The value stored in the history item identifier field 503 is called a history item identifier, and the history item identifier is used to indicate the position of the corresponding history item in the logical first-in-first-out queue.

[0140] Assuming that the value of the history item identifier is u, the number of global history items is v, and the length of the logical FIFO queue is s, the position of the history item in the logical FIFO queue can be expressed as follows: Formula (1).

[0141] Among them, if k>s, it means that the historical record item to which the historical record item identifier belongs exceeds the range of the logical first-in-first-out queue. The computing node 110 can convert the historical record item back into metadata. The conversion method is the inverse step of converting metadata to historical record items, which will not be repeated here.

[0142] Based on the cache replacement algorithm used when the data object was replaced from the memory node 120, indicated by the algorithm identification field 505 of the historical record item, the computing node 110 can dynamically adjust the priorities of multiple cache replacement algorithms based on the historical data item. Next, with reference to FIG7 , the cache replacement algorithm priority adjustment steps will be described, using the example of multiple cache replacement algorithm priorities being weighted, where the cache replacement algorithm with a higher weight has a higher priority. As shown in FIG7 , the cache replacement algorithm priority adjustment steps may include the following steps 710-750.

[0143] Step 710: When the computing node 110 misses accessing the second data object stored in the memory node 120, the computing node 110 determines that the memory node 120 stores a history record item of the second data object.

[0144] As one possible example, when computing node 110 misses accessing a second data object stored in memory node 120, computing node 110 sends a query operation request to memory node 120 to query memory node 120 for a history entry of the second data object. If a history entry of the second data object is found, computing node 110 determines that memory node 120 stores the history entry of the second data object. If a history entry of the second data object is not found, computing node 110 determines that memory node 120 does not store the history entry of the second data object.

[0145] Step 720 : The computing node 110 sends a second read operation request to the memory node 120 .

[0146] As a possible example, the computing node 110 sends a second read operation request to the memory node 120 , where the second read operation request is used to read a history record item of a second data object from the memory node 120 .

[0147] As a possible implementation, the history record item of the second data object includes an algorithm identification field. For example, the value of the algorithm identification field is used to indicate a second cache replacement algorithm among multiple cache replacement algorithms.

[0148] Step 730: The computing node 110 reduces the weight of the second cache replacement algorithm.

[0149] As a possible example, computing node 110 determines that the value of the algorithm identification field of the history record item of the second data object indicates the second cache replacement algorithm, that is, the second cache replacement algorithm performs an incorrect data replacement, and updates the weight of the second cache replacement algorithm based on the following formula (2).

[0150] Among them, w represents weight, e represents natural constant, λ represents learning rate, d t Represents the penalty value used to reduce the weight, and d is set to a constant value to reduce the impact of older errors in historical records.

[0151] Step 740 : The computing node 110 sends a third write operation request to the memory node 120 .

[0152] As a possible example, after reducing the weight of the locally stored second cache replacement algorithm, the computing node 110 sends a third write operation request to the memory node 120. The third write operation request is used to instruct the memory node 120 to write the reduced weight of the second cache replacement algorithm to the memory node 120, so that the memory node 120 merges the reduced weight of the second cache replacement algorithm into the global algorithm weight of multiple cache replacement algorithms maintained by the memory node 120, and returns the merged global algorithm weight to the computing node 110.

[0153] Step 750 : The computing node 110 and the memory node 120 synchronize the global algorithm weights.

[0154] In this way, based on the above steps 710 to 750, in the distributed system 100, the computing node 110 maintains the weight of the locally maintained cache replacement algorithm based on the historical record items, and communicates with the memory node 120 to synchronize the global algorithm weights of multiple cache replacement algorithms in the entire distributed system 100, thereby ensuring the accuracy of the algorithm weights and enabling the computing node 110 to accurately select the cache replacement algorithm suitable for the current data access mode based on the weights of multiple cache replacement algorithms, thereby achieving accurate cache replacement in the distributed system 100 with a memory-separated architecture.

[0155] To facilitate the cache replacement method shown in FIG. 2 , the present application further provides a cache replacement device 800 , which can be used to implement the functions of the computing node 110 in the cache replacement method shown in FIG. As shown in FIG. 8 , the cache replacement device 800 includes a transceiver module 810 and a processing module 820 .

[0156] The transceiver module 810 is used to send a first read operation request to the memory node, where the first read operation request is used to read access information of multiple data objects from the memory node. The cache data stored in the memory node includes multiple data objects.

[0157] The processing module 820 is configured to calculate the heat of the multiple data objects respectively by using a first cache replacement algorithm among multiple cache replacement algorithms based on access information of the multiple data objects.

[0158] The transceiver module 810 is also used to send a cache replacement request to the memory node according to the heat of multiple data objects when cache replacement of cached data is required. The cache replacement request is used to indicate that cold data in multiple data objects is replaced with data objects to be inserted into the memory node 120.

[0159] The transceiver module 810 and the processing module 820 can be implemented in software or hardware. For example, the implementation of the transceiver module 810 will be described below using the transceiver module 810 as an example. Similarly, the implementation of the processing module 820 can refer to the implementation of the transceiver module 810.

[0160] As an example of a software functional unit, the transceiver module 810 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the transceiver module 810 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0161] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0162] As an example of a hardware functional unit, the transceiver module 810 may include at least one computing device, such as a server. Alternatively, the transceiver module 810 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0163] The multiple computing devices included in the transceiver module 810 can be distributed in the same region or in different regions. The multiple computing devices included in the transceiver module 810 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the transceiver module 810 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0164] It should be noted that, in other embodiments, any module in the transceiver module 810 and the processing module 820 can be used to execute any step in the cache replacement method, and the steps that the transceiver module 810 and the processing module 820 are responsible for implementing can be specified as needed. The full functions of the cache replacement device 800 are realized by respectively implementing different steps in the cache replacement method through the transceiver module 810 and the processing module 820.

[0165] If the return of data from memory node 120 to compute node 110 is considered an operation performed by memory node 120, then the cache replacement method also includes steps performed by memory node 120. To facilitate the demonstration of a cache replacement method for interacting between compute node 110 and memory node 120, this application also provides a cache replacement device 900, which can be used to implement the cache replacement method for interacting between compute node 110 and memory node 120. As shown in FIG9 , cache replacement device 900 includes a transceiver module 910.

[0166] The transceiver module 910 is configured to return a response to the computing node 110 according to the request of the computing node 110. For example, the transceiver module 910 is configured to return access information of multiple data objects to the computing node 110 according to the first read operation request of the computing node 110.

[0167] The transceiver module 910 may be implemented by software or hardware. For the implementation of the transceiver module 910, please refer to the example of the transceiver module 810 in FIG8 , which will not be described in detail here.

[0168] Since the cache replacement method provided in this application requires the computing node 110 and the memory node 120 to interact with each other in the steps shown in FIG2 , the computing node 110 typically uses a client to communicate with the server of the memory node 120. Therefore, this application also provides a cache replacement system 1000, as shown in FIG10 , which includes a client 1010 and a server 1020.

[0169] The client 1010 is used to send a first read operation request to the memory node, where the first read operation request is used to read access information of multiple data objects from the memory node. The cache data stored in the memory node includes multiple data objects.

[0170] The server 1020 is configured to return access information to the client according to the first read operation request.

[0171] The client 1010 is further configured to calculate the heat of the multiple data objects respectively using a first cache replacement algorithm among the multiple cache replacement algorithms based on the access information of the multiple data objects.

[0172] The client 1010 is also used to send a cache replacement request to the memory node according to the popularity of multiple data objects when cache replacement of cached data is required. The cache replacement request is used to indicate that cold data in multiple data objects is replaced with the data object to be inserted into the memory node.

[0173] The client 1010 and the server 1020 can be implemented by software or hardware. For example, the implementation of the client 1010 is described below. Similarly, the implementation of the server 1020 can refer to the implementation of the client 1010.

[0174] As an example of a software functional unit, the client 1010 may include code running on a computing instance. The computing instance may be at least one of a physical host (computing device), a virtual machine, a container, and other computing devices. Furthermore, the computing device may be one or more. For example, the client 1010 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, and each AZ includes one data center or multiple data centers with close geographical locations. Typically, a region may include multiple AZs.

[0175] Similarly, the multiple hosts / virtual machines / containers running the code can be distributed within the same VPC or across multiple VPCs. Typically, a VPC is located within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0176] In an embodiment of the present application, client 1010 may be a software module for executing the steps performed by computing node 110 in the cache replacement method shown in FIG2 , or may be cache replacement device 800 as shown in FIG8 . Server 1200 may be a software module for executing the steps performed by memory node 120 in the cache replacement method shown in FIG2 , or may be cache replacement device 900 as shown in FIG9 .

[0177] A module is an example of a hardware functional unit. Client 1010 may include at least one computing device, such as a server. Alternatively, client 1010 may be implemented using an ASIC or a PLD. The PLD may be implemented using a CPLD, FPGA, GAL, or any combination thereof.

[0178] The multiple computing devices included in client 1010 can be distributed in the same region or in different regions. The multiple computing devices included in client 1010 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in client 1010 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0179] This application also provides a computing device 1100. As shown in Figure 11, computing device 1100 includes a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. Processor 1104, memory 1106, and communication interface 1108 communicate with each other via bus 1102. Computing device 1100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 1100.

[0180] Bus 1102 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG11 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 1102 may include a path for transmitting information between various components of computing device 1100 (e.g., memory 1106, processor 1104, and communication interface 1108).

[0181] The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0182] The memory 1106 may include volatile memory, such as random access memory (RAM). The processor 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0183] Memory 1106 stores executable program code, and processor 1104 executes the executable program code to implement the functions of the various modules included in cache replacement device 800 or cache replacement device 900, thereby implementing the cache replacement method. In other words, memory 1106 stores instructions for executing the cache replacement method.

[0184] Alternatively, the memory 1106 stores executable code, and the processor 1104 executes the executable code to implement the functions of the client 1010 or the server 1020, thereby implementing the cache replacement method. That is, the memory 1106 stores instructions for executing the cache replacement method.

[0185] The communication interface 1108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0186] Considering that the cache replacement method provided in this application is applied to a distributed system 100, the respective infrastructures of the computing nodes 110 and memory nodes 120 of the distributed system 100 typically include multiple computing devices. Therefore, this application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0187] As shown in Figure 12, the computing device cluster includes at least one computing device 1100. The memory 1106 in one or more computing devices 1100 in the computing device cluster may store the same instructions for executing the cache replacement method.

[0188] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the cache replacement method. In other words, the combination of one or more computing devices 1100 can jointly execute the instructions for executing the cache replacement method.

[0189] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster may store different instructions, each for executing part of the functions of the cache replacement apparatus 800 or the cache replacement apparatus 900. In other words, the instructions stored in the memory 1106 in different computing devices 1100 may implement the functions of one or more modules included in the cache replacement apparatus 800 or the cache replacement apparatus 900.

[0190] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network (WAN) or a local area network (LAN), among others. FIG8 illustrates a possible implementation. As shown in FIG12 , two computing devices 1100A and 1100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1106 in the computing device 1100A stores instructions for executing the functions of one or more of the transceiver module 810 and the processing module 820. FIG12 takes the example of the memory 1106 in the computing device 1100A storing instructions for executing the functions of the transceiver module 810. At the same time, the memory 1106 in the computing device 1100B stores instructions for executing the functions of one or more of the transceiver module 810 and the processing module 820. FIG8 takes the example of the memory 1106 in the computing device 1100B storing instructions for executing the functions of the processing module 820.

[0191] It should be understood that the functionality of the computing device 1100A shown in FIG12 may also be implemented by multiple computing devices 1100. Similarly, the functionality of the computing device 1100B may also be implemented by multiple computing devices 1100.

[0192] The present application also provides a computer program product comprising instructions. The computer program product may be software or a program product comprising instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the cache replacement method shown in FIG2 , or the steps executed by the computing node 110 or the memory node 120 in the cache replacement method shown in FIG2 .

[0193] An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the cache replacement method shown in Figure 2, or the steps performed by the computing node 110 or the memory node 120 in the cache replacement method shown in Figure 2.

[0194] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (such as infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0195] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0196] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0198] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0199] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0200] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0201] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b and c can be single or multiple.

[0202] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A cache replacement method, characterized in that: A computing node in a distributed system of a separated memory architecture, the distributed system further comprising a memory node, the computing node bypassing a processor of the memory node when accessing cache data stored in the memory node, the method comprising: Sending a first read operation request to the memory node, where the first read operation request is used to read access information of multiple data objects from the memory node, and the cache data stored in the memory node includes the multiple data objects; Based on the access information of the multiple data objects, using a first cache replacement algorithm among multiple cache replacement algorithms to respectively calculate the heat of the multiple data objects; When the cache data needs to be replaced, a cache replacement request is sent to the memory node according to the heat of the multiple data objects, and the cache replacement request is used to instruct to replace the cold data in the multiple data objects with the data objects to be inserted into the memory node.

2. The method according to claim 1, characterized in that The memory node stores metadata of the cache data, the multiple data objects include a first data object, the metadata includes the access information, and the method further includes: After the computing node performs data access to the first data object, a metadata update request is sent to the memory node, where the metadata update request is used to indicate updating access information of the first data object.

3. The method according to claim 1 or 2, characterized in that: The access information includes a last access timestamp and a number of accesses.

4. The method according to claim 3, characterized in that The metadata is stored in a hash table, the hash table includes a plurality of hash slots, each hash slot is used to store metadata of a data object, and each hash slot includes a length field, a pointer field, a last access timestamp field, and an access count field; The length field is used to indicate the length of the data object to which the metadata belongs, the pointer field is used to indicate the address of the data object to which the metadata belongs in the memory node, the last access timestamp field is used to indicate the time when the data object to which the metadata belongs was last accessed, and the access count field is used to indicate the number of times the data object to which the metadata belongs is accessed.

5. The method according to claim 4, characterized in that The metadata update request includes a first write operation request and a first atomic counting operation request; The first write operation request is used to instruct to set the value of the last access timestamp field of the metadata of the first data object to the timestamp when the computing node initiates the data access; The first atomic counting operation request is used to instruct to increase the value of the access count field of the metadata of the first data object by one.

6. The method according to any one of claims 1 to 5, characterized in that The cold data includes a second data object. After sending a cache replacement request to the memory node according to the heat of the multiple data objects, the method further includes: A history item generation request is sent to the memory node, where the history item generation request is used to instruct the metadata of the second data object to be converted into a history item, where the history item is used to indicate a second cache replacement algorithm used when the second data object is replaced by cache.

7. The method according to claim 6, characterized in that The historical record item is stored in a hash slot of a hash table, and the hash slot includes a type field and an algorithm identification field; The type field is used to indicate that the data object to which the historical record item belongs has been replaced by the cache, and the algorithm identification field is used to indicate a second cache replacement algorithm used when the data object to which the historical record item belongs is replaced by the cache.

8. The method according to claim 7, characterized in that The history entry generation request includes an atomic swap operation request and a second write operation request, the atomic swap operation request is used to indicate that the value of the length field of the metadata of the second data object is modified to the value of the type field, and the second write operation request is used to indicate that the value of a field in the metadata of the second data object used to represent access information is replaced with a value used to represent a second cache replacement algorithm used when the second data object is replaced by the cache.

9. The method according to claim 8, characterized in that The method further comprises: When a second data object stored in the memory node is accessed without a hit, determining that the memory node stores a history record item of the second data object; Sending a second read operation request to the memory node, where the second read operation request is used to read a history record item of the second data object from the memory node, and a value of an algorithm identification field of the history record item of the second data object indicates the second cache replacement algorithm; The weight of the second cache replacement algorithm is reduced.

10. The method according to claim 9, characterized in that The method further comprises: A third write operation request is sent to the memory node, wherein the third write operation request is used to instruct the reduced weight of the second cache replacement algorithm to be written to the memory node, so that the memory node merges the reduced weight of the second cache replacement algorithm into the global algorithm weight, and returns the merged global algorithm weight to the computing node.

11. The method according to any one of claims 1 to 10, characterized in that The sending a cache replacement request to the memory node according to the heat of the multiple data objects includes: When the remaining storage space of the memory node is smaller than the size of the data object to be inserted, or when the remaining storage space of the memory node is smaller than a preset threshold, a cache replacement request is sent to the memory node according to the heat of the multiple data objects.

12. A cache replacement method, characterized in that: The computing node and the memory node in the distributed system of the separated memory architecture are applied, and the computing node bypasses the processor of the memory node when accessing the cache data stored in the memory node. The method includes: The computing node sends a first read operation request to the memory node, where the first read operation request is used to read access information of multiple data objects from the memory node, and the cache data stored in the memory node includes the multiple data objects; The memory node returns the access information to the computing node according to the first read operation request; The computing node calculates the heat of the multiple data objects respectively using a first cache replacement algorithm among multiple cache replacement algorithms based on the access information of the multiple data objects; When the computing node needs to perform cache replacement on cached data, it sends a cache replacement request to the memory node according to the heat of the multiple data objects, wherein the cache replacement request is used to instruct to replace the cold data in the multiple data objects with the data objects to be inserted into the memory node.

13. A cache replacement device, characterized in that: include: A transceiver module, configured to send a first read operation request to a memory node, wherein the first read operation request is used to read access information of multiple data objects from the memory node, and the cache data stored in the memory node includes the multiple data objects; A processing module, configured to respectively calculate the heat of the plurality of data objects using a first cache replacement algorithm among a plurality of cache replacement algorithms based on the access information of the plurality of data objects; The transceiver module is also used to send a cache replacement request to the memory node when cache replacement is required for the cached data, according to the heat of the multiple data objects, and the cache replacement request is used to indicate that the cold data in the multiple data objects is replaced with the data object to be inserted into the memory node.

14. A cache replacement system, characterized in that: include: A client is used to send a first read operation request to a memory node, wherein the first read operation request is used to read access information of multiple data objects from the memory node, and the cache data stored in the memory node includes the multiple data objects; The server is configured to return the access information to the client according to the first read operation request; The client is further configured to respectively calculate the heat of the multiple data objects using a first cache replacement algorithm among multiple cache replacement algorithms based on the access information of the multiple data objects; The client is also used to send a cache replacement request to the memory node when cache replacement is required for cached data, according to the heat of the multiple data objects, and the cache replacement request is used to indicate that cold data in the multiple data objects is replaced with the data object to be inserted into the memory node.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 11.

16. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device. The computing device cluster is enabled to execute the method as claimed in claim 12.

17. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 11.

18. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method as claimed in claim 12 .

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 11.

20. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, which, when executed by a computing device cluster, perform the method of claim 12 .

Images