A data processing method and device for a hierarchical storage system

CN114416646BActive Publication Date: 2025-09-23上海妃鱼数字科技有限公司
View PDF 3 Cites -1 Cited by

Patent Information

Application Number
CN202210066308.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-20
Publication Date
2025-09-23
Estimated Expiration
2042-01-20

AI Technical Summary

Technical Problem

When querying target data, the log merge tree has poor read performance, especially when the load changes. The delay in reading data at different levels is uneven, resulting in a large number of I/O requests, affecting system performance and load adaptability.

Method used

By using Count-Min Sketch counters and Cuckoo filters to record the access popularity and hierarchy of key values, we can identify hot key values ​​and migrate them to higher-level storage during data merging and migration operations, reducing the lookup latency of hot key values ​​and improving read performance and load adaptability.

Benefits of technology

Effectively identifying and migrating hot key values ​​reduces the search latency of hot key values, improves the read performance of the log merge tree and the load adaptability of the system, and improves the overall search performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114416646B_ABST
    Figure CN114416646B_ABST
Patent Text Reader

Abstract

The present invention relates to a data processing method and device for a hierarchical storage system, which uses a counter to record the access popularity of each queried key value, and uses a data filter to record the level of each queried key value. The lower the level of the key value data, the slower the access speed. When the storage system performs a data merge operation on two data layers, it identifies valid hot key values, and retains the valid hot key values ​​in the current layer through temporary storage files. Then, through a hot key value migration operation, the temporary storage files are reversely merged and moved up to a higher-level data layer for storage. The more frequently accessed key values ​​will be stored in the upper layer, thereby reducing the search delay of the hot key value data. Compared with the prior art, the present invention performs an upward merge of hot key data through valid hot key value identification, which not only improves the overall search performance of the log merge tree, but also enhances the adaptability of the log merge tree to the load.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Movement of frequently accessed data chunks between storage tiers

    CN107728933A

  • Method for quickly reading key value storage data based on log structure merging tree

    CN112486994A

  • Distributed log condition query method and system based on cuckoo filter

    CN112988666A