Adaptive Database Tiering for Storage Cost Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face challenges in reducing the costs of providing durability and resiliency without compromising performance, as they often rely on costly storage technologies for both high-performance and redundant data copies.
Innovation Solution
Implementing adaptive tiering for database data within replica groups, where storage nodes dynamically select and cache frequently accessed data locally, using index structures like probabilistic and non-probabilistic structures, and leveraging external storage for less frequently accessed data, allowing for efficient data distribution and parallel read operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If costly storage technologies are used for both high-performance and redundant data copies, then durability and resiliency are improved, but storage costs increase
Solution Approach 1:
The patent segments storage resources into different tiers: expensive local storage for frequently accessed data and cheaper external storage for less frequently accessed data. This segmentation allows the system to maintain durability and resiliency through replication while reducing overall storage costs by placing less critical data in cheaper storage locations.
Solution Approach 2:
The patent applies local quality by differentiating storage quality based on data access patterns. Frequently accessed data receives high-quality local storage for performance, while less frequently accessed data uses lower-quality external storage. This resolves the contradiction by matching storage quality to actual needs rather than uniformly applying high-quality storage everywhere.
2Speed
If all data is stored locally for high performance, then access speed is improved, but storage costs and system complexity increase
Solution Approach 1:
The patent segments data based on access patterns into hot data (frequently accessed) and cold data (less frequently accessed). Hot data is stored locally for fast access, while cold data is stored externally. This segmentation simplifies the system by allowing external storage to handle bulk data management, reducing the burden on local storage infrastructure.
Solution Approach 2:
The patent introduces an intermediary mechanism (adaptive tiering system with index structures) that mediates between local and external storage. This intermediary automatically manages data placement and retrieval, simplifying the overall system architecture by providing a unified interface for data access regardless of whether data resides locally or externally.
3Quantity of substance
If adaptive tiering is implemented with multiple storage nodes, then storage costs are reduced, but system complexity increases
Solution Approach 1:
The patent implements self-service through automated adaptive tiering where the system automatically monitors data access patterns and relocates data between storage tiers without manual intervention. Index structures enable automatic identification of frequently accessed data, and the system autonomously manages data placement, reducing operational complexity despite having multiple storage nodes.
Solution Approach 2:
The patent employs feedback mechanisms where the system continuously monitors data access patterns and uses this information to dynamically adjust data placement. This feedback loop allows the system to learn from actual usage patterns and optimize storage allocation over time, managing complexity through intelligent automation rather than rigid manual configuration.
Data Source
AI summary
A storage node of a database replica group may distribute different portions of data in local storage and external storage, where local storage and external storage are organized using different types of index structures. Responsive to receiving an access request for a database, a storage node may determine that an item of the database to be accessed by the request does not reside within a first portion of the database stored locally at the storage node. Responsive to this determination, the storage node may obtain from an external storage service a second portion of the database, the second portion including a plurality of items including the item, and the second portion organized according to a structure different from the first portion. The storage node may then store the plurality of obtained items in the first portion and process the request using the first portion of the database.