A memory and disk hybrid caching method based on cloud object storage

Abstract

The application discloses a memory and disk hybrid cache method based on cloud object storage, which is characterized by the design of the corresponding cache storage granularity and structure of the memory cache and the disk cache, the improvement of the adaptive threshold limited LRU cache strategy, the caching of the data stored on the cloud object storage through the hybrid cache, and the efficient data access of the data on the cloud object storage by using the cache. Compared with the prior art, the application has the advantages of higher priority selection of the data block to be put into the cache, greatly improved cache hit rate, greatly reduced time for querying the data on the cloud object storage, reduced number of times for accessing the cloud object storage, saved more data access time, simple method, good use effect, and wide application prospect.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a hybrid memory and disk caching method based on cloud object storage for OLAP workloads. Background Technology

[0002] Cloud object storage (OOS) has been widely adopted as a storage system in various large-scale data warehouses and data lakes. OOS provides a simple interface similar to key-value (KV) systems, supporting operations such as PUT, GET, and DELETE. Its key feature is that it only supports appending to data objects, not modifying already written data. Most cloud service providers offer object storage services, such as Amazon Web Services (S3), Microsoft Azure Blob Storage, Alibaba Cloud's OSS, and Tencent Cloud's COS, all of which support the S3 protocol. OOS offers advantages such as low cost, high scalability, and pay-as-you-go storage. However, because data retrieval relies on network transmission, its latency is much higher than local disks, requiring the use of local memory and disk space to accelerate data retrieval from cloud object storage.

[0003] In multi-tiered storage architectures, systems typically utilize caching or tiering strategies to improve overall performance. Tiered strategies require storing hot and cold data in separate tiers and dynamically migrating data based on load. Research on caching generally focuses on the DRAM / SSD storage tiers, while tiering strategies are typically used between two persistent storage tiers. However, with rapid hardware advancements and the addition of persistent memory and cloud object storage, modern storage hierarchies have become more complex. Therefore, in such multi-tiered architectures, traditional caching or tiering strategies may not be the most effective, and storage management strategies should be reconsidered.

[0004] In terms of caching strategies, common cache replacement strategies such as LRU and LFU have a long history of research. Many works have improved upon the basic LRU algorithm, with LIRS being a classic example. LIRS records data block access history, making it more accurate in distinguishing between hot and cold data. ARC uses an LRU list and an LFU list, and can dynamically adjust the length of these two lists based on workload. DLIRS improves upon LIRS, making it adaptable to workload changes. There's also the more novel LHD algorithm, which introduces the concept of hit density. It models and predicts the hit density of objects using historical load data, rather than relying on implicit assumptions, such as the LRU algorithm's assumption that recently used objects are more valuable. Some caching strategies are also tailored to specific storage architectures, designing KV, KP caches, and block caches, creating multi-level caches that can dynamically adjust the size allocation of each cache level based on workload changes.

[0005] However, in the database field, the most commonly used caching algorithm is LRU because it is simpler and more standardized to implement in engineering. However, LRU is not suitable for OLAP scenarios that primarily involve scanning large amounts of data. In OLAP scenarios, data is not stored in fixed-size pages like row-based storage; instead, individual columns are often stored as separate files. Due to differences in partitions, data types, and other characteristics between columns, the sizes of these files can vary significantly. A single large file scan can flush the entire cache, rendering the cache ineffective. Therefore, existing caching algorithms are not entirely suitable for OLAP workloads, and further research is needed to achieve efficient data access through caching.

[0006] The purpose of this invention is to address the shortcomings of existing technologies by designing a hybrid memory and disk caching method based on cloud object storage. It employs a structural design that combines the granularity of cache storage in memory and on disk, and utilizes an LRU caching method based on adaptive threshold limitations. Using cloud object storage as the storage layer, it performs hybrid caching of data stored on cloud object storage. By limiting the LRU threshold, it prioritizes storing smaller data items, thereby achieving a higher cache hit rate and saving more query time. This better adapts to the efficient data access of the stored database to cloud object storage under OLAP loads. The method is simple, effective, and prioritizes higher-yield data blocks for caching, improving cache hit rate, saving more data access time, and increasing work efficiency.

[0007] The objective of this invention is achieved as follows: a hybrid memory and disk caching method based on cloud object storage, characterized by the use of a hybrid cache based on memory and disk, a structural design method combining cache storage granularity in memory and disk, and an improvement on the LRU caching strategy based on adaptive threshold limits. The hybrid cache manager caches data stored on cloud object storage to adapt to the efficient data access of the stored database on cloud object storage under OLAP load.

[0008] The structural design combining the granularity of cache storage in memory and disk includes:

[0009] A1: The cached content of the memory cache is stored in memory, and LRU is implemented by doubly linked lists and hash tables. The storage granularity of the memory cache is a range of data files.

[0010] A2: The disk cache is also composed of doubly linked lists and hash tables. The storage granularity of the disk cache is a range of data files. The cache files are stored in a specified cache directory. After a restart, the database will look up the cache directory and read some metadata from the disk cache into memory.

[0011] A3: The way cache items are divided in memory cache and disk cache is related to the data access load. In order to ensure that there is no overlap in the range between cache items, the data file needs to be split according to the order of data access, which serves as the basis for dividing cache items.

[0012] The hybrid caching architecture design, which combines memory and disk caching granularity, is a three-tier storage structure of DRAM / SSD / Cloud Object Storage. Cloud object storage serves as the storage layer, with the original data files stored on it, while memory and disk act as the performance layer for caching data files.

[0013] The improvements to the adaptive threshold-based LRU caching strategy include:

[0014] B1: When the range of accessed data is less than the memory cache threshold, the data will be stored in the memory cache; if it is greater than the memory cache threshold, it will not be retained in memory.

[0015] B2: When the range of accessed data is less than the disk cache threshold, the data will be stored as a separate disk file on the local disk. If the range is greater than the disk cache threshold, the data will not be stored on the local disk.

[0016] B3: Both memory cache thresholds and disk cache thresholds need to be dynamically adjusted in real time based on load changes. When data access during a load period generally exceeds the threshold, the threshold should be appropriately increased; conversely, when data access during a load period generally falls below the threshold, indicating frequent data swapping in and out, the threshold should be decreased. This threshold adjustment is automatically implemented through ghost lists. One ghost list, A, stores data items that are not cached due to exceeding the threshold, while another ghost list, B, stores data items evicted when the cache is full. When an access hits data in ghost list A, and a larger, frequently accessed data item fails to be cached, the threshold should be appropriately increased; when an access hits data in ghost list B, and a smaller, frequently accessed data item is evicted, the threshold should be appropriately decreased.

[0017] The process of caching data stored in cloud object storage using hybrid caching and efficiently accessing data in cloud object storage using the cache includes the following steps:

[0018] C1: OLAP workloads typically involve range access to column data files, with varying file range sizes. When a read request arrives, it is determined whether the requested file and range have been hit in the memory cache.

[0019] C2: If the requested range is a full hit in the memory cache, the corresponding data is returned immediately. If it is a partial hit in the memory cache (i.e., it partially overlaps with the existing memory cache), the non-overlapping part needs to be searched in the disk cache. If it is a miss, the required range is searched in the disk cache.

[0020] C3: For the same file, the scope of the request needs to be considered in relation to existing disk cache. The part that has been cached is loaded from the disk file, and only the part that has not been cached needs to be downloaded from cloud object storage.

[0021] C4: When a data segment needs to be downloaded from cloud object storage, if the size of the data segment is within the disk cache threshold, then the data segment will be placed in the disk cache as a cache item, generating the corresponding disk file; if this range is within the memory cache threshold, then the data segment will be placed in the memory cache.

[0022] C5: When the memory cache is full, evict occupants using the LRU algorithm; when the disk cache is full, evict occupants also use the LRU algorithm.

[0023] Compared to existing technologies, this invention prioritizes data blocks with higher returns for caching, improving cache hit rate and saving more data access time. Due to the high latency and high throughput characteristics of cloud object storage, two data items differing in size by several orders of magnitude may only have a two-fold difference in retrieval time. Therefore, with limited cache capacity, caching smaller data items is more cost-effective. This invention, by limiting the LRU threshold, prioritizes storing smaller data items, thereby achieving a higher cache hit rate and saving more query time. Furthermore, this invention, through the design of a ghost list, enables the threshold to be automatically adjusted according to the load, eliminating the need for pre-setting the cache admission threshold manually. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the hybrid caching architecture of the present invention;

[0025] Figure 2 A diagram illustrating the partitioning of cache items;

[0026] Figure 3 This diagram illustrates the changes in cached data items when the requested data range is less than a threshold.

[0027] Figure 4 This diagram illustrates the changes in cached data items when the requested data range exceeds a threshold. Detailed Implementation

[0028] The present invention will be further described in detail below with specific embodiments of caching.

[0029] See Figure 1 The storage layer is a three-tiered storage architecture of DRAM / SSD / Cloud Object Storage. Cloud Object Storage serves as the storage layer, where the original data files are stored. Memory (DRAM) and disk (SSD) act as the performance layer for caching data files. Both memory and disk caches have a threshold value to control cache access. The following section uses... Figure 1 The data access sequence in the example will be explained in detail.

[0030] For query1, it accesses data in the range [0, 2] on file A, which corresponds to data segment ① in the diagram. This data is first read from cloud object storage into memory. The size of data segment ① is less than the disk cache threshold, so it is placed in the disk cache's LRU list. Simultaneously, data segment ① is also less than the memory cache threshold, so it is also placed in the memory cache's LRU list. Query2 is similar to query1, also accessing data on file A. For query3, data segment ③ is less than the disk cache threshold and can be placed in the disk cache's LRU list, but it is greater than the memory cache threshold, so it is not placed in the memory cache. In subsequent accesses, the data segment [3, 5] corresponding to query4 is placed in both the memory cache and disk cache, while a portion of the [3, 5] data segment [3, 6] corresponding to query5 has already been cached, so only the [5, 6] data segment, corresponding to data segment ⑤, needs to be cached. In practical applications, the overlapping access ranges of the same file may be more complex, and the handling of overlapping situations will be discussed in more detail later.

[0031] See Figure 2 The diagram illustrates how the scope of cached items is divided. There are two main storage methods for columnar storage: one is to store all columns together in a single file, and the other is to store each column separately in a separate file. These two methods do not need to be discussed separately; both use a range of data files as the cache unit and are applicable to both methods. In actual access, the access ranges of the same data file may overlap. To prevent cached items from overlapping within the same data file, the data file is split according to the order of data access, and this split serves as the basis for dividing cached items.

[0032] For example, consider a data access to a file with an offset starting from left_0 and a size of right_0 - left_0, meaning the access range is [left_0, right_0]. The overlap between this access range and existing cached entries for the same data file is discussed below:

[0033] Case 1: The access scope does not overlap with the existing cached item. In this case, the cached item and the access scope are consistent. A new cached item [left_0, right_0] is generated and loaded from the storage layer.

[0034] Scenario 2: The access scope overlaps with existing cached items, and the overlap is as follows: Figure 2As shown, in this case, the offset of the access range is to the left of the existing cached item, and a new cached item [left_0, left_1] will be generated. This data will be loaded from the storage layer, and finally the data segment returned is composed of the two cached items [left_0, left_1] and [left_1, right_0].

[0035] Scenario 3: The access scope overlaps with existing cached items, and the overlap is as follows: Figure 2 As shown, in this case, the left boundary of the access range is contained within the existing cache entry. A new cache entry [right_1, right_0] is generated, and this data is loaded from the storage layer. Finally, the returned data segment consists of [left_0, right_1] and [right_1, right_0].

[0036] Scenario 4: The access scope overlaps with existing cached items, and the overlap is as follows: Figure 2 As shown, in this case, the first and second halves of the access range overlap with the two existing cached items respectively, creating a gap in the middle. The data segment to be returned is divided into three segments: [left_0, right_1], [right_1, left_2], and [left_2, right_0]. The returned data segment is the requested data segment [right_1, left_2].

[0037] Case 5: The access scope completely overlaps with existing cached items, as shown in the following example. Figure 2 As shown, no new cached items are generated in this case. It returns the data within the [left_0, right_0] range of the existing cached items.

[0038] See Figures 3-4This section explains how cached data items change when the requested data range is less than and greater than a threshold, and how the threshold automatically adjusts based on these changes. Since memory and disk caches use the same caching strategy, this section uses memory cache as an example to illustrate how its threshold adapts to changes in load. The LRU cache uses two ghost lists: one for storing items evicted by LRU (those with data sizes below the threshold), and the other for storing items not yet added to the LRU (those with data sizes above the threshold). Initially, the threshold is infinite, allowing all data items to be cached, so the cache initially behaves like a normal LRU cache. When the cache is full and data items need to be evicted, they are evicted using the LRU algorithm and placed in the corresponding ghost list. The ghost list is also an LRU list, storing keys and data ranges, but not the actual data. The automatic threshold adjustment using ghost lists is categorized as follows:

[0039] Case 1: If the accessed data range is less than the threshold and the cache is hit, the cached item is moved to the tail of the LRU queue.

[0040] Scenario 2: The accessed data range is less than the threshold, the cache is not hit, and the ghost list is not found, such as... Figure 3 The requested data segment is a file with the key value K4 and a range of [0, 2]. This data segment satisfies condition 2. This data segment is read from the storage layer into memory and stored as a cache item, then placed at the tail of the LRU queue. If the LRU queue is full, a data item is selected from the head of the LRU queue for eviction, and its key and data range are stored at the tail of the ghost list of the eviction item. If the ghost list is full, the data item at the head of the ghost list is selected for eviction.

[0041] Scenario 3: The accessed data range is less than the threshold, resulting in a cache miss and a hit on the ghost list. The corresponding item in the ghost list is removed. Data reading and subsequent operations on the LRU queue and ghost list are the same as in Scenario 2. Since the ghost list of evicted items was hit, it indicates that the threshold is too high. This threshold needs to be lowered to reduce the possibility of large data block accesses flushing the buffer. This allows more frequently accessed, smaller data blocks to be stored in memory, improving the cache hit rate.

[0042] Scenario 4: The accessed data range exceeds the threshold, and the ghost list is not matched, such as... Figure 4The requested data segment is a file with the key value K5 and a range of [0, 10]. This data segment satisfies condition 4. At this point, a data item containing the key and the data range will be generated and placed at the end of the ghost list that is not cached. If the ghost list is full, the data item at the head of the ghost list will be evicted.

[0043] Scenario 5: The accessed data range exceeds the threshold, hitting the ghost list. The corresponding item in the ghost list is deleted. The fact that a ghost list not yet in the cache was hit indicates that the threshold is too low, preventing frequently accessed items with large data sizes from being cached. Therefore, the threshold needs to be increased.

[0044] The above is merely a further explanation of the present invention and is not intended to limit the scope of this patent. All equivalent implementations of the present invention should be included within the scope of the claims of this patent.

Claims

1. A cloud object storage based memory and disk hybrid cache method, characterized in that, The hybrid cache architecture design of the memory and the cache storage granularity in the disk is adopted, and the LRU cache method based on the adaptive threshold limit is used to store the data on the cloud object storage, and by limiting the LRU threshold, the data items with smaller data volume are preferentially stored, so that a higher cache hit rate is achieved, and the hybrid cache is used to adapt to the columnar database under the OLAP load to achieve efficient data access on the cloud object storage. The storage layer of the hybrid cache architecture design of the memory and the cache storage granularity in the disk is a three-layer storage structure of DRAM / SSD / Cloud Object Storage, the cloud object storage is used as the storage layer, the original data file is stored on the cloud object storage, the memory and the disk are used as the performance layer of the cached data file, and the specific design of the hybrid cache architecture includes: A1: The cache content of the memory cache is stored in the memory, the LRU is realized by a bidirectional list and a hash table, and the storage granularity of the memory cache is an interval of a data file; A2: The cached file is stored in a specified cache directory, and after restarting, the database searches the cache directory, reads some meta information of the disk cache into the memory, the disk cache is composed of a bidirectional list and a hash table, and the storage granularity is an interval of a data file; A3: The cache items of the memory cache and the disk cache are divided according to the order of data access; The LRU cache method based on the adaptive threshold limit specifically includes: B1: When the access data range is smaller than the memory cache threshold, the data is stored in the memory cache; if the access data range is greater than the memory cache threshold, the data is not retained in the memory; B2: When the access data range is smaller than the disk cache threshold, the data is stored as a separate disk file in the local disk; if the access data range is greater than the disk cache threshold, the data is not stored in the local disk; B3: The memory cache threshold and the disk cache threshold are dynamically adjusted in real time according to the change of the load, when the data access in the load is generally greater than the threshold, the threshold should be appropriately adjusted to be larger; on the contrary, when the data access in the load is generally smaller than the threshold, there is frequent swapping in and out, and the threshold should be adjusted to be smaller; The efficient data access on the cloud object storage by using the hybrid cache to adapt to the columnar database under the OLAP load specifically includes the following steps: C1: The OLAP type load accesses the range of the columnar data file, when the read request arrives, it is judged whether the request file and range hit in the memory cache; C2: If the requested range completely hits in the memory cache, the corresponding data is immediately returned; if it is partially hit, that is, there is partial overlap with the existing memory cache, the non-overlapping part needs to be found in the disk cache; if it is not hit, the required range needs to be found in the disk cache; C3: For the same file, the part that has been cached is loaded from the disk file, and the part that has not been cached needs to be downloaded from the cloud object storage. C4: When there is a data segment to be downloaded from the cloud object storage, if the size of the data segment is within the threshold of the disk cache, the data segment will be put into the disk cache as a cache item, and a corresponding disk file will be generated; if the size of the data segment is within the threshold of the memory cache, the data segment will be put into the memory cache; C5: When the memory cache is full or the disk cache is full, the LRU algorithm is used to evict.

2. The cloud object storage based memory and disk hybrid caching method of claim 1, wherein, The adjustment of the memory cache threshold and the disk cache threshold will be automatically implemented through a ghost list, wherein a ghost list A is used to store data items that are not put into the cache due to being greater than the threshold, and a ghost list B is used to store data items that are evicted when the cache is full. When the data in the ghost list A is accessed, the larger frequent data items fail to be put into the cache, indicating that the threshold should be appropriately adjusted to be larger. When the data in the ghost list B is accessed, the smaller frequent data items are evicted, indicating that the threshold should be appropriately adjusted to be smaller.

Images