Method and system for reducing garbage collection and write amplification of key-value separation storage system
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A separate storage and write amplification technology, applied in the direction of memory system, file system, file system type, etc., can solve the problems of performance not increasing but degrading, and achieve the effects of improving efficiency, reducing garbage collection operations, and reducing frequency
Active Publication Date: 2021-02-23
HUAZHONG UNIV OF SCI & TECH
View PDF14 Cites 11 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
Wisckey also has another shortcoming. For key-value pairs with small values, the performance does not increase but decreases after the key-value separation, and the problem of space enlargement is obvious.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0061] A method for reducing write amplification and garbage collection in a key-value separation storage system such as figure 1 As shown, it includes memory, NVM and SSD. A first storage area and a second storage area are created in NVM. The first storage area manages data in the form of LSM-tree, and the second storage area manages data in the form of memory pool;
[0062] Methods include:
[0063] LSM-tree insertion steps:
[0064] Insert the key-value pair into the variable table in the memory; after the variable table becomes an immutable table, the stored key-value pair will be flushed to the first storage area;
[0065] In a key-value separation storage system such as levelDB, before the key-value pair is inserted into the LSM-tree, it will be inserted into the variable table in the memory, that is, figure 1 The memtable in the variable table is specifically a kind of jump table; when the variable table is full, it will become an immutable table, that is, figure 1 T...
Embodiment 2
[0100] A key-value separation storage system, including: memory, NVM, and SSD. A first storage area and a second storage area are created in the NVM. The first storage area uses LSM-tree to manage data, and the second storage area uses a memory pool. manage data in the form of
[0101] The key-value separation storage system also includes a data management module, which includes:
[0102] The LSM-tree insertion unit is used to insert the key-value pair into the variable table in the memory; after the variable table becomes an immutable table, the stored key-value pair will be flushed to the first storage area;
[0103] The memory pool insertion unit is used to insert the key-value pair into the second storage area, and form a new key with the key key of the key-value pair and its storage address addr in the second storage area The value pair , and use the LSM-tree insertion unit to insert the key-value pair ;
[0104] The key-value pair insertion unit is used to ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention discloses a method and a system for reducing garbage collection and write amplification of a key-value separation storage system, and belongs to the field of computer storage, the systemcomprises a memory, an NVM and an SSD; and a first storage area and a second storage area in the NVM manage data according to an LSM-tree and a memory pool respectively; the method comprises the steps of judging whether the size of a to-be-inserted key value pair is greater than or equal to a first threshold value or not, if so, inserting the key value pair into the second storage area, and theninserting a key and a key value address into LSM-tree; otherwise, directly inserting the key value pair into LSM-tree; when the key value pair in the memory pool is deleted, marking the correspondingstorage unit as an idle memory unit; when the data volume in the second storage area is higher than a second threshold value, brushing down the key value pair with the least frequent access to the SSD, and updating the storage address of the key value pair; and setting a flag bit segment on the NVM, and identifying the validity of the key value pair of the data segment on the SSD. Garbage collection and write amplification of the key value storage system can be reduced, and the overall performance of the key value separation storage system is improved.
Description
technical field [0001] The invention belongs to the field of computer storage, and more specifically relates to a method and system for reducing garbage collection and write amplification of a key-value separation storage system. Background technique [0002] In recent years, with the rapid development of cloud computing, big data, Internet of Things, artificial intelligence and other information technologies and the extensive use of online shopping, online ordering, Weibo, and other software, the amount of data has exploded. How to effectively store and manage Massive data poses new challenges to existing computer system storage technologies. [0003] As we all know, the sequential writing of the disk is about 1000 times that of the random writing. The structure of LSM-tree (Log-StructuredMerge Tree, log structure tree) converts random writing into sequential writing, which greatly improves the performance of data writing to disk. However, the merging process of LSM-tree b...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more