Selective Spool Storage Across Object and Local Database Tiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of managing spool data in relational database management systems (DBMS) involves high costs and inefficiencies due to the need for pre-allocating large volumes of block-based storage for temporary data, and the use of remote object stores with slower I/O speeds and egress charges, which are unsuitable for frequently accessed and updated spool data.
Innovation Solution
A DBMS selectively stores spool data in both remote object stores and local block-based storage based on factors like size, frequency of updates and access, and the need for performance metrics, using B-trees to manage data distribution and access, and leveraging object store elasticity for cost-effective and efficient data management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If pre-allocation of large volumes of block-based storage is used for spool data, then storage capacity is ensured, but storage costs increase and resource efficiency decreases
Solution Approach 1:
The patent implements dynamic storage allocation where the DBMS automatically determines whether to store spool data in local block-based storage or remote object storage based on real-time factors such as data size, access frequency, and update frequency. This dynamic approach replaces static pre-allocation with adaptive resource management, optimizing both capacity utilization and cost efficiency.
2Loss of energy
If remote object stores are used for spool data, then storage cost decreases, but I/O performance deteriorates due to slower access speeds
Solution Approach 1:
The patent applies local quality by making different parts of the storage system serve different purposes: hot spool data (frequently accessed or updated) is stored in local block-based storage for high-speed access, while cold spool data is stored in remote object storage for cost efficiency. The DBMS automatically classifies and routes data based on access patterns, ensuring optimal performance for each data type.
3Adaptability or versatility
If remote object stores are used for spool data, then storage elasticity increases, but egress charges increase for frequently accessed data
Solution Approach 1:
The patent introduces local block-based storage as an intermediary layer between the DBMS and remote object storage. Frequently accessed spool data is kept in this intermediary local storage, avoiding repeated data egress from remote storage and the associated charges. The system automatically manages data movement between the intermediary local storage and remote storage based on access patterns.
4Loss of energy
If selective spool data storage is implemented, then resource utilization efficiency improves, but system complexity increases
Solution Approach 1:
The patent implements self-service by enabling the DBMS to autonomously make storage decisions without requiring complex external management systems. The DBMS itself evaluates spool data characteristics and automatically determines the optimal storage location, eliminating the need for separate orchestration layers and reducing overall system complexity while maintaining high resource efficiency.
Data Source
AI summary
In some examples, a database system includes processing modules with access to a remote object store and a local database storage associated with the database system. The processing modules perform a database operation that involves use of a plurality of instances of spool data. The processing modules store a first instance of spool data in the remote object store based on a first characteristic of the first instance of spool data, and the processing modules store a second instance of spool data in the local database storage based on a second characteristic of the second instance of spool data.