HIERARCHICAL STORAGE FOR LSM-BASED NoSQL STORES

Abstract

Logically arranged hierarchy or tiered storage may comprise a layer of storage being a faster access storage (e.g. solid state drive (SSD)) and another (e.g., next) layer being a traditional disk (e.g. HDD). In one embodiment, compaction occurs within the higher layer, e.g., until there is no more room and then during the compaction sequence the data may be moved down to the lower layer. In another embodiment, compaction and migration to a lower layer may occur within the higher layer, e.g., based on one or more policies, even if the higher layer is not full. In one embodiment, the data between layers are maintained as disjoint. In one embodiment, the more recent versions are always in the higher layer and the older versions are always in the lower layer.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application is a continuation of U.S. Ser. No. 13 / 975,512, filed on Aug. 26, 2013, the entire content and disclosure of which is incorporated herein by reference.FIELD[0002]The present application relates generally to computers, and computer applications, data storage management, and more particularly to hierarchical storage for log-structured merge-tree (LSM) data.BACKGROUND[0003]To address challenges in flexible data schema and elasticity, new distributed data stores dealing with columns, key / value pairs, documents and graphs, collectively identified as NoSQL data stores, have emerged. A category of NoSQL stores are based on LSM (Log Structured Merge) tree. It is optimized for write, but does not necessarily have an optimized technique for point queries. LSM stores traditionally put data first in dynamic random-access memory (DRAM) and then move data to hard disk drive (HDD) storage hierarchy when allocated DRAM reaches maximum cap...

AI Technical Summary

Benefits of technology

The patent describes a system for storing data on different memory and storage devices with different characteristics. The data is organized in a hierarchy, with the most accessible devices at the top and the slowest devices at the bottom. The system can flush data from the top layer to a lower layer, and can compact and migrate data within a layer if needed. The technical effect is that the system optimizes data storage and retrieval by utilizing the characteristics of different devices in a hierarchical structure, leading to faster processing times and reduced latency.

Problems solved by technology

It is optimized for write, but does not necessarily have an optimized technique for point queries.

Read access is comparatively slow when data is in HDD.

While existing systems may remedy the slow response rate by using read cache buffers in DRAM, caching mechanisms come with their own set of complexities that contribute to the data storage systems.

Images