Unlock instant, AI-driven research and patent intelligence for your innovation.

Method for writing data in a distributed storage system

A distributed storage and data technology, applied in the input/output process of data processing, file system, file system type, etc., can solve problems such as random writing, and achieve the effect of reducing randomness and reducing the number of data writes

Active Publication Date: 2021-07-16
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Also, writing data to log files and data files creates random write issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for writing data in a distributed storage system
  • Method for writing data in a distributed storage system
  • Method for writing data in a distributed storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0083] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0084] figure 1 It is a data flow of a distributed storage system based on the raft protocol in the prior art. Whenever new data is to be written, figure 1 Both the master process and the slave process in the log file write data in their respective log files and data files. The core idea of ​​the present invention is that, depending on the size of the data to be written, data is written to a log file in a non-transitory storage medium or data is written to a data file in a non-transitory storage medium, thereby avoiding The secondary write (that is, writing to both the log file and the data file) eliminates the write amplification problem. The present invention is particularly suitable for a distributed storage system that uses a disk as a non-...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a method for writing data in a distributed storage system. The distributed storage system includes memory and non-transitory storage media. A replication group including at least the main process is created in the distributed storage system, and log files and data files of the main process are stored on the non-transitory storage medium. The method includes: a main process receiving a data writing request; depending on the size of the data to be written, the main process writes the data to be written into a log file of the main process, or submits the data to a data file of the main process. The method of the invention enables to reduce the number of writes and eliminate the problem of write amplification.

Description

【Technical field】 [0001] The present invention relates to a distributed storage system. In particular, the invention relates to a method for writing data in a distributed storage system. 【Background technique】 [0002] In a distributed storage system, multiple copies of data are usually saved to improve the reliability of the storage system. Synchronization of data across multiple copies is usually achieved through log files. For example, the raft protocol is a replication group communication protocol, which communicates among the replicas in the replication group based on logs to achieve data consistency. [0003] However, in the prior art, copies belonging to different replication groups are stored on the same disk, and both log files and data files need to be operated on the disk. This creates problems with write amplification and random writes. Specifically, data is written to the log file and the data file at the same time, that is to say, user data is written to th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/18G06F16/182G06F16/13
CPCG06F3/061G06F3/064G06F3/0659G06F3/067G06F16/27G06F3/0604G06F3/065G06F16/1734
Inventor 马井玮
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD