Elastic Key-Value pair data storage method based on RDMA and HTM

Abstract

The invention provides an elastic Key-Value pair data storage method based on RDMA (Remote Direct Memory Access) and HTM (High Temperature Modulation). A server-client architecture mode is designed and followed, and the method comprises the steps that at a server side, an improved G-Cuckoo Hash data management mode is given in combination with a Hashmap; the kick-out infinite loop between Hash tables caused by searching for idle positions in the data insertion process is avoided. the bottleneck problem of key value pair storage performance caused by round response required by the traditional network message transmission between the client and the server is analyzed; an Infiniband remote direct memory access RDMA technology is used, an RDMA network communication engine is designed, a data access request is received, and a data request result is sent back; a hardware transaction memory HTM technology is utilized to realize a two-section lock protocol lock operation and ensure the atomiccharacteristics of data operation; and a key value is used for self-verifying the check code of the data to guarantee data consistency. The basic operation speed of the key value on the data can be greatly improved.

Description

technical field [0001] The invention belongs to the field of computer systems and data storage systems, and in particular relates to an elastic Key-Value pair data storage method based on RDMA and HTM, which can provide fast and large-scale data services. Background technique [0002] At this stage, the demand for data-intensive computing is increasing day by day, and the data provided for data-intensive computing is generally stored in storage systems. In addition to storage systems such as relational databases, more and more key-value storage systems are emerging, which can also meet the needs of fast access to data. For the key-value pair storage system, how to obtain low latency and high throughput of data access has become a topic worthy of in-depth research and discussion. [0003] In a distributed cluster environment, it is often necessary to carry out remote data access to key-value storage systems through traditional networks. In this case, the joint participation...

AI Technical Summary

Problems solved by technology

(3) Implementation based on Log-Structured-Merge tree (LSM tree), such as levelDB, which has the advantage of high write performance, but the disadvantage of poor read performance and easy to cause serious write amplification

[0010] Generally speaking, the elastic Key-Value key-value pair data storage method based on RDMA and HTM lacks strong software and program support such as high-performance data transmission network protocols, hardware transaction memory management that guarantees the atomic nature of data operations, etc.; On the other hand, there are still problems such as low efficiency of data management mode, no design of data cache to store recently accessed data, poor data consistency mechanism, etc., and the degree of parallelism of data operations needs to be further improved

Images