Hybrid index-based hybrid memory performance optimization method and system

Abstract

The invention discloses a hybrid index-based hybrid memory performance optimization method and system, which belongs to the field of computer data storage, and comprises the following steps of: establishing a hash table in an NVM (Non-Volatile Memory) of a hybrid memory in advance, and establishing a skip table and a waiting queue in a DRAM (Dynamic Random Access Memory); for each operation request, if the operation request is an insertion operation, inserting the operation request into a hash table according to a key value of a key value pair to be inserted, and adding the key value pair andan insertion operation mark into a waiting queue; if the operation request is deletion operation, deleting the key value pair from the hash table according to the key value of the key value pair to bedeleted, and adding the key value pair and a deletion operation mark into a waiting queue; if the operation request is the single-point query operation, obtaining the key value pair from the hash table according to the key value of the key value pair to be queried; and if the operation request is a range query operation, synchronizing all the key value pairs in the waiting queue into the skip list, and obtaining all the key value pairs in the query range from the skip list. According to the invention, various operation requests including range query can be efficiently realized.

Description

technical field [0001] The invention belongs to the field of computer data storage, and more specifically relates to a hybrid index-based hybrid memory performance optimization method and system. Background technique [0002] Internet big data applications and cloud computing applications require fast access to large-scale data. As a typical representative of non-relational data storage, key-value store (KVS, Key Value Store) adopts unstructured data organization form to provide efficient Data insertion, point query, and scan query are supported. With its high access performance, high availability, and high scalability, key-value storage is widely used in data-intensive data-intensive applications such as network index engines and social networks. In application such as etc. Memory key-value pair storage mainly uses DRAM (Dynamic Random Access Memory) as a storage device, but DRAM is increasingly unable to meet the capacity requirements of big data applications due to facto...

AI Technical Summary

Problems solved by technology

[0003] The index structure is a key part of the high-performance memory key-value storage system. Due to the relatively poor write performance and limited durability of NVM in hybrid memory, the traditional DRAM-based index structure cannot be efficiently applied to new NVM devices.

The existing NVM-based index structure is mainly designed and optimized for a specific data structure. However, different index structures have their own shortcomings. For example, the hash table with O(1) time complexity does not support range Query; the index structure that supports range query, such as B / B+-tree, has a time complexity of only O(log(N)), and the operations of splitting and merging nodes included are more complicated to implement, etc.

[0004] In general, due to the characteristics of the medium itself and the limitations of the index structure, there is still a large room for optimization in the existing hybrid memory performance

Images