Cache Tier cache optimization method based on Ceph cluster

Abstract

The invention discloses a Cache Tier cache optimization method based on a Ceph cluster. The method comprises the steps: optimizing a cache, obtaining an existing cache pool mode of the Cache Tier, taking the request with the request size smaller than 128k as a small request, and taking the request with the request size larger than or equal to 128k as a large request; executing a writing operation,writing the small request into a cache pool, and writing the large request into a base pool through cache execution proxy writing operation; executing a reading operation, if the cache hits, traversing the object fragment information, and reading the object fragment information is read in a cache pool; after cache reading is finished, reading the data in a base pool; finally, splicing the data read from the cache pool and the base pool, filling and returning to the client. According to the method, the read-write mode of the cache pool is modified based on the processing mode of the requests of different sizes, and the read-write efficiency of the Ceph cluster is improved by optimizing the Cache Tier.

Description

technical field [0001] The invention belongs to the technical field of distributed system storage, and in particular relates to a Ceph cluster-based Cache Tier cache optimization method. Background technique [0002] Ceph's unique decentralized distributed architecture enables it to manage large-scale storage nodes. At the same time, its flat addressing design based on computing allows Ceph clients to communicate directly with any node on the server side, so Ceph has performance and cluster linearity Excellent feature of the extension. [0003] In a traditional Ceph cluster, in order to implement the isolation strategy between different storage pools, user data is not written to the local storage device of the OSD in one step, but an intermediate structure PG (Placement Group) is introduced to perform two mappings. First, the user data is cut into objects according to a fixed size, which is input as a random hash function, and evenly mapped to each PG; then, the PG is mappe...

AI Technical Summary

Problems solved by technology

However, due to the inherent limitations of the time locality of the LRU algorithm, LRU lacks long-term granular analysis of data, resulting in frequent access to hot data may be eliminated by occasionally accessed cold data, resulting in cache pollution

At this time, the cache pool will generate additional overhead due to processing cold data, which will reduce the performance of the cache pool and even the system. At the same time, the hit rate of hot objects in the cache pool will not be high, and finally the I / O throughput of the storage cluster will decrease.

On the other hand, SSD is expensive, and the capacity of the cache pool layer of hierarchical cache is limited. When a large amount of data stored in the cache pool layer reaches the cache full threshold, refresh or clear operations will be performed, and frequent refresh or clear operations will cause Ceph The IO performance did not achieve the desired result

Images