Method and system for reducing garbage collection and write amplification in a key-value separation storage system

A technology of separate storage and write amplification, which is applied in the direction of memory system, file system, file system type, etc., can solve the problems of performance not increasing but degrading, and achieve the effects of improving efficiency, reducing read amplification, and reducing frequency

Active Publication Date: 2022-05-31
HUAZHONG UNIV OF SCI & TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Wisckey also has another shortcoming. For key-value pairs with small values, the performance does not increase but decreases after the key-value separation, and the problem of space enlargement is obvious.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for reducing garbage collection and write amplification in a key-value separation storage system
  • Method and system for reducing garbage collection and write amplification in a key-value separation storage system
  • Method and system for reducing garbage collection and write amplification in a key-value separation storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0062] The method includes:

[0069] key-value pair insertion step:

[0071] The first threshold can be determined according to the NVM type actually selected. In this embodiment, due to the selected NVM type

[0081] After the key-value pairs in the LSM-tree area perform Compaction (merge operation), some key-value pairs will be deleted

[0082] In order to improve the efficiency of garbage collection on the SSD, as a preferred implementation, as shown in FIG. 1, this embodiment

[0087] SSD garbage collection step: when the SSD is garbage collected, traverse the flag bit segment in the third storage area, and

[0089] Since when the SSD is garbage collected, it is only necessary to find the flag bit segment to determine whether the key-value pair in the data segment has

[0091] In order to improve the storage space utilization of the NVM, as a preferred implementation, the second storage area creates

[0093] Take the storage unit closest to the size of the key-value pair , ...

Embodiment 2

[0101] The key-value separation storage system also includes a data management module, and the data management module includes:

[0107] In this embodiment, the specific implementation steps of each module can refer to the descriptions in the above method embodiments, which will not be described here.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and system for reducing garbage collection and write amplification of a key-value separation storage system, belonging to the field of computer storage. The system includes memory, NVM and SSD. The first storage area and the second storage area in NVM are respectively according to LSM-tree and memory pool management data; the method includes: judging whether the size of the key-value pair to be inserted is greater than or equal to the first threshold, if so, inserting the key-value pair into the second storage area, and then inserting the key and the key-value address into the LSM ‑tree; otherwise, insert the key-value pair directly into the LSM-tree; when the key-value pair is deleted in the memory pool, mark the corresponding storage unit as a free memory unit; the amount of data in the second storage area is higher than the second threshold , flash the most infrequently accessed key-value pair to the SSD, and update the storage address of the key-value pair; set a flag segment on the NVM to identify the validity of the data segment key-value pair on the SSD. The invention can reduce the garbage collection and write amplification of the key-value storage system, and improve the overall performance of the key-value separation storage system.

Description

Method and system for reducing garbage collection and write amplification in key-value separation storage systems technical field The invention belongs to the field of computer storage, more specifically, relate to a kind of garbage that reduces the key-value separation storage system Method and system for recycling and write amplification. Background technique In recent years, with the rapid development of information technologies such as cloud computing, big data, the Internet of Things, artificial intelligence, and the With the extensive use of software such as online shopping, online ordering, Weibo, etc., the amount of data has exploded. How to store and manage it effectively Massive data poses new challenges to the existing computer system storage technology. As we all know, the sequential write of the disk is about 1000 times of random write, LSM-tree (Log-Structured The structure of Merge Tree, log structure tree) is to convert random writes into sequential...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F12/02G06F16/13G06F16/18G06F16/901
CPCG06F12/0253G06F12/023G06F16/13G06F16/1847G06F16/9027G06F2212/1016G06F2212/1032G06F2212/1044
Inventor 陈俭喜冯丹郑梦丽马莉珍陈彧陈鑫宇张兴锐
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap