Distributed memory management method based on network and page granularity management

Abstract

The invention discloses a distributed memory management method based on network and page granularity management. The method is characterized in that a node, page and block memory management method isadopted for a global address; partial memories from different nodes are packaged in a global page-based memory (GPM), and data transmission under page granularity is supported, memory management is performed based on the updated memory consistency protocol, so that the data on the local cache page is consistent with the data on the GPM, and a high-level application program deployed in the PDMM transparently accesses the GPM, thereby realizing low-delay and high-throughput inter-node access. Compared with the prior art, the inter-node access delay with low delay and high throughput is achieved,the method is simple and convenient to use, the problem that cache is invalid due to write operation in data intensive work is effectively solved, and the performance of the PDMM is superior to thatof other products of the same type.

Description

technical field [0001] The invention relates to the technical field of distributed memory management, in particular to a distributed memory management method based on network and page granularity. Background technique [0002] As the performance expansion of a single computer server slowed down, people began to create a distributed memory management platform similar to NUMA architecture across servers through low-latency remote access primitives that support RDMA networks. Under a pure NUMA architecture, these platforms (such as FaRM, Rack-Out and GAM) provide object-level granularity for memory management and RDMA operations. Compared with local memory access, the latency of InfiniBand QDR and RoCE deployed on GAM and FaRM in turn is almost 25 times and 100 times that of local memory access. Therefore, the access delay between nodes on the NUMA architecture in the distributed memory management platform will significantly reduce the execution speed of the application, espec...

AI Technical Summary

Problems solved by technology

But caching the hottest data items can only adapt to unbalanced scenarios in KVSs, but cannot handle other data-intensive applications

Cache data blocks that are frequently accessed in data-intensive workloads are easily expired due to sporadic write operations. Most of these operations require data consistency (for example, invalidation and update). Difficult to support the performance of such applications

Images