Data Storage Segmentation by Refresh Rate Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage methods, such as load balancing using data modulus operations, face inefficiencies when nodes are added or removed, leading to data redistribution and potential overload, as they fail to accurately load cached data and manage varying data refresh rates effectively.
Innovation Solution
A data storage method that classifies data by refresh rates and uses Hash calculations to determine storage segments, allowing efficient storage and retrieval without requiring full data redistribution upon node changes, by defining high and low refresh rate data based on volume thresholds and using a preset storage segment determination relationship.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data are evenly distributed to each storage node using data modulus operation, then data storage balance is improved, but data storage efficiency deteriorates when nodes are added or removed due to required full data redistribution
Solution Approach 1:
The patent segments data into different types (hot data, warm data, cold data) based on access frequency, and stores each type in dedicated storage regions with appropriate storage policies. This segmentation allows different storage strategies for different data types, avoiding the need to redistribute all data when nodes change, thus improving storage efficiency while maintaining balance.
Solution Approach 2:
The patent pre-calculates and stores mapping relationships between data identifiers and storage node identifiers before actual storage operations. When nodes are added or removed, only the mapping relationships need to be updated rather than redistributing all data, which significantly improves storage efficiency while maintaining balanced data distribution.
2Measurement precision
If all data are stored again upon addition or deletion of nodes, then data distribution accuracy is improved, but storage time increases and original cached data cannot be loaded accurately
Solution Approach 1:
The patent extracts and stores only the essential mapping information (data identifier to storage node identifier relationships) separately from the actual data. When nodes change, only this extracted mapping information needs to be updated, not the entire dataset, thus reducing storage time while maintaining distribution accuracy.
Solution Approach 2:
The patent pre-establishes and maintains mapping relationships between data and storage nodes. When nodes are added or removed, the system only needs to update the affected mapping entries rather than redistributing all data, significantly reducing the time required while ensuring accurate data distribution.
3Reliability
If frequent data storage operations are performed to maintain data distribution, then data storage accuracy is improved, but data storage overload occurs
Solution Approach 1:
The patent extracts and manages only the critical mapping metadata separately from the bulk data storage. This allows the system to maintain accurate data distribution information with minimal storage operations on the actual data, reducing storage overload while maintaining accuracy.
Solution Approach 2:
The patent applies different storage policies and operations to different data types (hot, warm, cold data) based on their access characteristics. This local differentiation reduces the frequency and intensity of storage operations needed for different data types, preventing storage overload while maintaining necessary accuracy for each data category.
Data Source
AI summary
A data storage method includes: acquiring target data to be stored, and classifying refresh rates of the target data to be stored according to a front-end system; subjecting the target data to be stored with high refresh rates as classified and the target data to be stored with low refresh rates as classified to a Hash calculation to obtain a first type Hash value and a second type Hash value; determining storage data segments corresponding to the first type Hash value and the second type Hash value according to a preset storage data segment determination relationship, and storing the target data to be stored with high refresh rates and the target data to be stored with low refresh rates into the storage data segments corresponding to the first type Hash value and the second type Hash value, respectively.