Elastic Key-Value pair data storage method based on RDMA and HTM

A data storage and key-value pair technology, applied in electrical digital data processing, transmission systems, electrical components, etc., can solve the problems of undesigned data cache storage, poor data consistency mechanism, write amplification, etc., to accelerate data access and Results transfer, high throughput, latency reduction effects

Active Publication Date: 2019-07-30
SHANGHAI JIAO TONG UNIV
View PDF2 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

(3) Implementation based on Log-Structured-Merge tree (LSM tree), such as levelDB, which has the advantage of high write performance, but the disadvantage of poor read performance and easy to cause serious write amplification
[0010] Generally speaking, the elastic Key-Value key-value pair data storage method based on RDMA and HTM lacks strong software and program support such as high-performance d

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Elastic Key-Value pair data storage method based on RDMA and HTM
  • Elastic Key-Value pair data storage method based on RDMA and HTM
  • Elastic Key-Value pair data storage method based on RDMA and HTM

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment

[0115] A specific embodiment of the present invention is as follows:

[0116] The key implementations and software interfaces of the elastic Key-Value storage system based on RDMA and HTM include:

[0117] (1) Software interface abstraction, the system provides basic operation interfaces for users to use, these interfaces include value setting function rhkv_set, value obtaining function rhkv_get and value updating function rhkv_update, etc.

[0118] The underlying details provided by the software interface are described as follows: For the function rhkv_get, the key parameter m is passed to the hash function Hash through the Infiniband network. The abstract function calculates two candidate positions through mask operation, and finds the actual value of the hash table, and returns the final result. For the function rhkv_set, the actual process may be more complicated. Specifically, the function first finds the key value entry in the cache stash. If the entry exists, update ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an elastic Key-Value pair data storage method based on RDMA (Remote Direct Memory Access) and HTM (High Temperature Modulation). A server-client architecture mode is designed and followed, and the method comprises the steps that at a server side, an improved G-Cuckoo Hash data management mode is given in combination with a Hashmap; the kick-out infinite loop between Hash tables caused by searching for idle positions in the data insertion process is avoided. the bottleneck problem of key value pair storage performance caused by round response required by the traditional network message transmission between the client and the server is analyzed; an Infiniband remote direct memory access RDMA technology is used, an RDMA network communication engine is designed, a data access request is received, and a data request result is sent back; a hardware transaction memory HTM technology is utilized to realize a two-section lock protocol lock operation and ensure the atomiccharacteristics of data operation; and a key value is used for self-verifying the check code of the data to guarantee data consistency. The basic operation speed of the key value on the data can be greatly improved.

Description

technical field [0001] The invention belongs to the field of computer systems and data storage systems, and in particular relates to an elastic Key-Value pair data storage method based on RDMA and HTM, which can provide fast and large-scale data services. Background technique [0002] At this stage, the demand for data-intensive computing is increasing day by day, and the data provided for data-intensive computing is generally stored in storage systems. In addition to storage systems such as relational databases, more and more key-value storage systems are emerging, which can also meet the needs of fast access to data. For the key-value pair storage system, how to obtain low latency and high throughput of data access has become a topic worthy of in-depth research and discussion. [0003] In a distributed cluster environment, it is often necessary to carry out remote data access to key-value storage systems through traditional networks. In this case, the joint participation...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F13/28H04L29/06
CPCG06F13/28H04L67/01G06F16/2255
Inventor 黄林鹏吴仁克李素敏
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap