Raft group data adaptive distribution method for a heterogeneous distributed key value storage system
A distributed key-value and storage system technology, applied in heterogeneous and Raft protocol, distributed storage, key-value storage fields, can solve problems such as slow system performance, slow nodes, problems, and failure to utilize fast node performance, etc. To achieve the effect of improving read and write performance
Active Publication Date: 2019-05-31
RENMIN UNIVERSITY OF CHINA
View PDF7 Cites 2 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
This balanced data and leader distribution method is no problem in the homogeneous distributed KV storage, but there are problems in the heterogeneous distributed KV storage
[0007] To sum up, the existing data distribution technology, such as balanced distribution technology, does not take heterogeneous scenarios into consideration. In heterogeneous distributed KV storage, the performance of fast nodes is not well utilized. On the contrary, the performance of the system is often given by slow nodes. slow down
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
[0025] Such as figure 2 As shown, the heterogeneous distributed key-value storage system Raft group data adaptive distribution system architecture provided by the present invention consists of three parts:
[0026] Client: used to write or read data, provide stress testing, and test system performance;
[0027] Heterogeneous distributed KV...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to a Raft group data adaptive distribution method for a heterogeneous distributed key value storage system, which comprises the following steps of: estimating the read-write performance of a node according to a fitting function between the read-write performance of a disk and the read-write performance of a node KV interface; distributing data according to the writing performance; acquiring data reading and writing conditions of the system at set intervals, and the data reading and writing conditions are judged; if yes, setting the Leader as the fastest node in all copies; and if the reading is the main reading, allocating the Leader according to the reading performance. According to the method, the characteristics of the Raft protocol are combined, the performance ofthe nodes is effectively utilized, and the read-write performance of the system is greatly improved.
Description
technical field [0001] The invention relates to a method for self-adaptive distribution of Raft group data in a heterogeneous distributed key-value storage system, and relates to the fields of distributed storage, key-value storage, heterogeneity and Raft protocol. Background technique [0002] Heterogeneous distributed key-value storage system: With the large-scale deployment of distributed storage systems, a considerable part of big data is stored in key-value (KV) storage systems. Compared with traditional relational databases, KV storage has higher scalability and efficiency to meet the requirements of big data. As the scale of the distributed KV storage system increases, the current heterogeneous distributed KV storage system will become a trend. There are two main reasons: 1. Disks such as SSDs have limited service life, and disks have a high damage rate in distributed storage systems, so disks are often replaced, and new disks generally have better performance and hi...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
Inventor 柴云鹏王洋洋
Owner RENMIN UNIVERSITY OF CHINA