Main Memory Database Record Classification for Performance Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Main memory database systems face performance degradation due to skewed access patterns, where infrequently accessed records remain in memory, leading to increased costs and reduced efficiency, as they waste cycles inspecting these records during frequent access searches.
Innovation Solution
Implementing a classification system that uses a record access log to estimate access frequencies using weighted averages with decreasing weights over time, allowing for the classification of records as 'hot' or 'cold' and migrating 'cold' records to secondary storage, thereby optimizing memory usage and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If all records are kept in memory in main memory database systems, then data accessibility is improved, but memory cost and system performance deteriorate due to wasted inspection cycles on cold records
Solution Approach 1:
The patent segments records into three categories based on access frequency: hot records (frequently accessed), lukewarm records (moderately accessed), and cold records (infrequently accessed). This segmentation allows the system to apply different storage strategies to different record types, keeping hot records in memory for fast access while migrating cold records to secondary storage, thereby resolving the contradiction between data accessibility and system performance.
Solution Approach 2:
The patent changes the storage location parameter for records based on their access frequency characteristics. By dynamically adjusting where records are stored (memory vs. secondary storage) according to their hot/lukewarm/cold classification, the system optimizes both accessibility for frequently accessed records and overall performance by reducing memory waste on cold records.
2Reliability
If all records are kept in memory, then data availability is improved, but memory cost increases due to storing infrequently accessed records
Solution Approach 1:
The patent extracts cold records from memory and migrates them to secondary storage based on access frequency analysis. This extraction reduces the quantity of data occupying memory resources, thereby lowering memory cost while maintaining data availability since cold records can be retrieved from secondary storage when needed.
Solution Approach 2:
The patent changes the storage location parameter for cold records from memory to secondary storage, optimizing the balance between data availability and memory cost. Hot and lukewarm records remain in memory for quick access, while cold records are moved out to reduce memory consumption.
3Measurement precision
If record access frequency is tracked continuously, then access pattern accuracy is improved, but computational overhead increases
Solution Approach 1:
The patent applies partial action by tracking access frequencies only sufficiently to classify records into hot, lukewarm, and cold categories, rather than continuously monitoring every access with high precision. This approach achieves the necessary measurement precision for classification while avoiding excessive computational overhead from overly granular tracking.
Solution Approach 2:
The patent changes the measurement approach from continuous high-precision tracking to periodic sampling that suffices for categorization. By adjusting the measurement parameters to match the actual needs of record classification rather than maximizing precision, the system reduces computational overhead while maintaining adequate access pattern accuracy.
Data Source
AI summary
Various technologies described herein pertain to classifying data in a main memory database system. A record access log can include a sequence of record access observations logged over a time period from a beginning time to an end time. Each of the record access observations can include a respective record ID and read timestamp. The record access log can be scanned in reverse from the end time towards the beginning time. Further, access frequency estimate data for records corresponding to record IDs read from the record access log can be calculated. The access frequency estimate data can include respective upper bounds and respective lower bounds of access frequency estimates for each of the records. Moreover, the records can be classified based on the respective upper bounds and the respective lower bounds of the access frequency estimates, such that K records can be classified as being frequently accessed records.