Data Block Access Prediction with Adjacent-Block Read Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing databases struggle to accurately distinguish between hot and cold data, leading to inefficiencies in storage performance and cost-effectiveness due to the inability to predict long-term access frequencies and manage data types effectively.
Innovation Solution
A data processing method that utilizes a pre-trained access frequency prediction model to determine the type of data blocks by analyzing multidimensional features, including read/write features of a data block and its adjacent blocks, to optimize storage location and improve system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in a single storage location without access frequency prediction, then storage system is simple, but storage performance and cost-effectiveness deteriorate due to inability to distinguish hot and cold data
Solution Approach 1:
The system performs preliminary action by predicting future access frequencies of data blocks before actual access occurs. The access frequency prediction model analyzes historical access patterns and multidimensional features to forecast which data blocks will be frequently accessed, enabling proactive data placement decisions that optimize storage performance before access demands arise.
Solution Approach 2:
The system implements self-service by automatically classifying data blocks as hot or cold based on predicted access frequencies without requiring manual intervention. The storage system autonomously makes decisions about data placement and migration, continuously adapting to changing access patterns through the prediction model and feedback mechanisms.
2Measurement precision
If multidimensional features including adjacent block read features are used for prediction, then prediction accuracy improves, but computational complexity increases
Solution Approach 1:
The system segments the feature analysis process into distinct components: read features of the target data block, read features of adjacent data blocks, and read/write features. This segmentation allows the prediction model to process different feature types independently and combine them systematically, improving prediction accuracy while maintaining manageable computational complexity through structured feature engineering.
Solution Approach 2:
The system introduces an intermediary mechanism by analyzing features of adjacent data blocks to infer access patterns of the target block. Instead of directly analyzing all possible features of the target block, the system uses adjacent block features as intermediate indicators that correlate with target block access behavior, thereby improving prediction accuracy without proportionally increasing computational burden.
3Productivity
If data blocks are frequently migrated between storage locations based on access frequency, then storage performance improves, but system overhead and resource occupation increase
Solution Approach 1:
The system applies local quality by making data placement decisions specific to each data block's predicted access frequency rather than applying uniform storage policies. Hot data blocks are placed in high-performance storage locations with faster access speeds, while cold data blocks are placed in lower-cost storage locations. This localized optimization improves overall access efficiency without unnecessarily migrating all data blocks, thereby reducing system overhead.
Data Source
AI summary
This application discloses a data processing method. A first apparatus obtains a first feature of a first data block in storage space. The first feature includes a read/write feature related to the first data block and a read feature of a second data block adjacent to the first data block. The first apparatus inputs the first feature into an access frequency prediction model, to obtain access frequency of the first data block. The access frequency of the first data block is used to determine a type of the first data block. To be specific, in embodiments of this application, future access frequency of the first data block is predicted by using a multidimensional feature of the first data block (the read/write feature of the first data block and a read feature of an adjacent data block).