Unlock instant, AI-driven research and patent intelligence for your innovation.

File merging method and device

A file and data technology, applied in the field of file merging methods and devices, can solve problems such as write amplification of solid-state hard disks, and achieve the effect of eliminating write amplification

Active Publication Date: 2018-11-02
HUAWEI TECH CO LTD
View PDF4 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, when the storage engine writes these data to the SSD again, it will cause the write amplification of the SSD.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • File merging method and device
  • File merging method and device
  • File merging method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0023] The data storage of the level database adopts the structure of Log-Structured Merge Tree (LSM-Tree), which stores billion-level Key-Value data persistently, and the Key-Value data is the key (English: key) and a value (English: value), the key can uniquely identify the data, and the value refers to the data to be written into the storage system (hereinafter referred to as data). LSM-Tree converts the random writing of the hard disk into sequential writing, thereby improving the speed of writing data. The embodiment of the present invention applies the LSM-Tree to a storage system including a solid state disk (English: Solid State Disk, SSD).

[0024] Combine below figure 1 and figure 2 , the application of the LSM-Tree described in this embodiment to a storage system including a solid-state disk is...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention provides a file merging method and device and a storage system. The storage system comprises a controller and a solid state disk. The controller reads a first file anda second file of the solid state disk. The first file comprises first data and a keyword of the first data. The second file comprises second data and the keyword of the second data. A corresponding relationship between a physical address of the first data and an identity of the first file, and the corresponding relationship between the physical address of the second data and the identity of the second file are stored in the controller. The controller judges whether the keyword of the first data is the same as that of the second data or not. When the keyword of the first data is different fromthat of the second data, the controller establishes a third file and redirects the first data and the second data to the third file. The write amplification of the solid state disk can be eliminated.

Description

technical field [0001] Embodiments of the present invention relate to the field of storage technology, and in particular, to a method and device for merging files. Background technique [0002] The log structured merge tree (LSMtree) is a data structure corresponding to the B+ tree. It is mainly to overcome the problem of low performance caused by a large number of random IOs of the B+ tree. The essence is to achieve balance before reading and writing, and sacrifice Read performance to greatly improve write performance. [0003] There are many KV storage engines implemented based on the LSMtree principle in the industry, and these storage engines are widely used in various data storage systems. The core idea of ​​these applications to implement the KV storage engine is to convert random IO into sequential IO writes, and then organize the data into partially ordered files through the memory and store them in the file system, and finally merge and sort the files in multiple f...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F3/06
CPCG06F3/0608G06F3/0643
Inventor 朱云生
Owner HUAWEI TECH CO LTD