Database Table Unloading via Access Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud database systems, unused tables often remain loaded in memory, leading to resource bottlenecks and inefficient processing due to inadequate unloading mechanisms that fail to consider the type and frequency of table access, resulting in incorrect or insufficient data unloading.
Innovation Solution
A system that determines whether to unload tables based on access thresholds, timestamps, and table sizes, using a cost function algorithm and machine learning to adjust thresholds and improve decision-making, distinguishing between business and housekeeping data access to optimize memory usage and processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If tables are kept loaded in memory to enable quick access, then data access speed is improved, but memory consumption increases and resource bottlenecks occur
Solution Approach 1:
The patent implements dynamic table unloading by continuously monitoring access patterns and automatically unloading tables from memory when they become unused. The system transitions tables between loaded and unloaded states based on real-time access metrics, allowing memory resources to be dynamically reallocated. This resolves the contradiction by making memory usage adaptive rather than static, maintaining fast access for active tables while freeing memory for inactive ones.
Solution Approach 2:
The system changes the state parameter of tables (loaded/unloaded) based on access frequency and recency metrics. By monitoring parameters such as access count and time since last access, the system dynamically adjusts which tables remain in memory, transforming the fixed memory allocation into a variable state that optimizes both access speed and memory consumption.
2Device complexity
If simple unloading mechanisms are used to reduce complexity, then device complexity is reduced, but unloading accuracy deteriorates leading to incorrect or insufficient data unloading
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors table access patterns and uses this information to make informed unloading decisions. The access pattern data feeds back into the unloading algorithm, allowing the system to accurately identify which tables are candidates for unloading. This feedback loop resolves the contradiction by providing the intelligence needed for accurate unloading without requiring complex manual intervention.
Solution Approach 2:
The system performs self-service by automatically monitoring its own access patterns and making unloading decisions without external intervention. The database management system itself generates and processes the metrics needed to determine which tables to unload, eliminating the need for complex external monitoring tools while maintaining high unloading accuracy through its inherent access pattern data.
3Quantity of substance
If tables are unloaded to reduce memory consumption, then memory usage is optimized, but system startup time increases due to reloading
Solution Approach 1:
The patent implements preliminary action by pre-loading tables that are predicted to be needed based on access pattern analysis. The system uses historical and real-time access metrics to identify tables that are likely to be accessed soon, ensuring they remain loaded or are quickly reloaded. This preliminary preparation resolves the contradiction by anticipating future access needs and reducing the actual startup time when tables are needed.
Solution Approach 2:
The system employs periodic monitoring and evaluation of table access patterns to determine unloading and reloading timing. By periodically assessing which tables should be unloaded and which should be kept loaded, the system optimizes the balance between memory usage and startup performance, preventing both excessive memory consumption and unnecessary reloading delays.
Data Source
AI summary
A system for unloading tables of a database is provided. In some aspects, the system performs operations including determining that a number of accesses to a table occurring within a time period has satisfied an access threshold. The operations may further include identifying, in response to the determining, a first timestamp indicating a most recent access to the table. The operations may further include determining whether a difference between a current timestamp and the first timestamp satisfies a first time threshold. The operations may further include comparing, in response to the difference satisfying the first time threshold, a ratio of the difference and a size of the table to a ratio threshold. The operations may further include unloading, in response to satisfying the ratio threshold, the table. The operations may further include adjusting, based on the feedback, the first time threshold and/or the ratio threshold.