Intelligent Memory Block Replacement for Column-Based Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional memory replacement algorithms are inadequate for column-based in-memory databases, as they fail to consider correlations between data columns, leading to inefficient data swapping and reduced system performance due to frequent memory replacements.
Innovation Solution
A framework that calculates the amount of loaded data to be discarded from memory by selecting candidate data columns based on correlations with requested columns, using an association rule algorithm to predict future usage and optimize memory replacement, thereby improving data swapping efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional page-based memory replacement algorithms are used, then memory management can be implemented, but data scanning speed deteriorates and replacement efficiency worsens because columns are much larger than page frames and correlations between columns are not considered
Solution Approach 1:
The patent changes the fundamental parameter of memory replacement granularity from page-based to column-based, matching the data organization structure of column-oriented databases. It introduces correlation-based scoring as a new parameter to evaluate which columns to replace, considering both access patterns and data correlations. This resolves the contradiction by aligning memory management with the actual data access patterns and structure, reducing unnecessary replacements and improving scanning speed.
Solution Approach 2:
The patent implements a feedback mechanism by monitoring column access patterns and using this information to predict future access needs. The system continuously learns from access patterns and adjusts replacement decisions based on predicted future usage, combined with correlation analysis. This feedback loop reduces the frequency of replacements by making more accurate predictions about which columns will be needed soon.
2Quantity of substance
If memory size is limited, then system resource constraints are managed, but data processing performance deteriorates due to frequent memory replacements
Solution Approach 1:
The patent applies preliminary action by pre-calculating correlation scores between columns and maintaining an access pattern history before replacement decisions are needed. When memory replacement is required, the system already has pre-computed information about column correlations and access patterns, enabling faster and more accurate replacement decisions without sacrificing performance. This reduces the overhead of replacement operations.
Solution Approach 2:
The patent changes the replacement strategy from simple LRU (least recently used) to a correlation-based scoring system that considers multiple factors including access patterns, column correlations, and data size. This parameter change enables more intelligent selection of which columns to evict, reducing the frequency of replacements and maintaining better performance under memory constraints.
3Speed
If column-based memory organization is used, then data scanning speed is improved, but memory replacement complexity increases because columns are much larger than traditional page frames
Solution Approach 1:
The patent segments the memory replacement problem into manageable components: (1) access pattern analysis, (2) correlation calculation between columns, (3) scoring mechanism, and (4) replacement execution. By dividing the complex replacement decision into these segments, the system can maintain column-based organization for speed while managing complexity through structured processing of each aspect separately.
Solution Approach 2:
The patent introduces an intermediary scoring mechanism that mediates between the column-based data organization and the replacement decision process. The score, calculated based on access patterns and correlations, acts as an intermediary metric that simplifies the replacement decision without requiring complex direct analysis of column relationships, thus maintaining scanning speed while managing algorithmic complexity.
Data Source
AI summary
A framework for intelligent memory replacement of loaded data blocks by requested data blocks is provided. For example, various factors are taken into account to optimize the selection of loaded data blocks to be discarded from the memory, in favor of the requested data blocks to be loaded into the memory. In some implementations, correlations between the requested data blocks and the loaded data blocks are used to determine which of the loaded data blocks may become candidates to be discarded from memory.