Partition Table Data Management Node Storage Policy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In existing data management systems, expired data in partition tables is deleted after exceeding a storage duration threshold, preventing upper-layer applications from accessing it, and there is a need to manage data storage costs effectively while maintaining data value.
Innovation Solution
A method where a management node assesses the creation duration and access history of data in a partition table, altering its storage policy to match the performance of the storage resource, allowing expired data to remain accessible by storing it based on a storage policy corresponding to a lower performance stage, thus optimizing storage resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is deleted after exceeding storage duration threshold, then storage costs are reduced, but upper-layer applications cannot access expired data
Solution Approach 1:
The patent changes the storage parameter from binary (stored/deleted) to multi-level (different storage stages with different performance and cost characteristics). Data transitions through multiple storage stages based on creation duration, allowing expired data to be retained in lower-cost storage while still being accessible, thus resolving the contradiction between cost reduction and data accessibility.
2Loss of information
If data is retained in high-performance storage, then data accessibility is maintained, but storage costs increase
Solution Approach 1:
The patent implements dynamic storage policy adjustment where data automatically transitions between storage stages based on its creation duration and access patterns. Expired data is dynamically moved from high-performance storage to lower-cost storage, maintaining accessibility while optimizing costs. This dynamic adaptation resolves the static contradiction between maintaining accessibility and reducing costs.
3Loss of energy
If multiple storage stages are implemented, then storage cost optimization is achieved, but system complexity increases
Solution Approach 1:
The patent implements self-service mechanisms where the storage system automatically manages data transitions between stages based on predefined policies and data characteristics. The management node autonomously evaluates data creation duration and access patterns, then automatically migrates data between storage stages without requiring complex manual intervention or oversight, thus reducing operational complexity while achieving cost optimization.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This application provides a method and an apparatus for managing data in a partition table, a server, and a storage medium, and belongs to the field of big data technologies. The method includes: A management node obtains a creation duration range of data of a target partition in the partition table; the management node further obtains a geographical area to which the target partition belongs, historical access information of the target partition, or a service type of the target partition; and then, when the creation duration range of the data of the target partition meets a predetermined time range of a first storage stage and a preset condition is met, the management node may alter the data that is of the target partition and that has been stored based on a storage policy corresponding to a second storage stage to be stored based on a storage policy corresponding to the first storage stage. Storage performance of the first storage stage is lower than storage performance of the second storage stage. Therefore, an upper-layer application can access data even if the data expires.