Unlock instant, AI-driven research and patent intelligence for your innovation.

Compact-structure key value pair storage structure and rapid key value pair finding method

A technology with a compact structure and a storage structure, which is applied in special data processing applications, instruments, and electrical digital data processing, etc. It can solve the problems of insufficient storage structure, slow running speed that cannot meet application requirements, and uncertainty in storage and search times, etc. problem, to achieve the effect of solving the low utilization rate of storage space, improving the search speed, and reducing the occupied space

Active Publication Date: 2018-05-11
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF5 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] The purpose of the present invention is to overcome the defect that the key-value pair storage structure of the existing method is not compact enough and in order to solve the current large-scale key-value pair storage and search time uncertainty, the software implementation speed is too slow to meet the application requirements and other problems , to meet the requirements of reasonable use of cache for less data replacement, and propose a compact key-value pair storage structure and fast key-value pair lookup method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Compact-structure key value pair storage structure and rapid key value pair finding method
  • Compact-structure key value pair storage structure and rapid key value pair finding method
  • Compact-structure key value pair storage structure and rapid key value pair finding method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0074] The specific steps of the first part of the key-value pair insertion process include:

[0075] Step 1, set the number of layers of the hash table and the number of storage blocks contained in each layer of hash table, and set the size of the Bloom filter corresponding to each layer of hash table;

[0076] In this embodiment. If image 3 As shown, the number of layers of the hash structure is set to 4. The first layer of hash table 2-left hashing includes 204,800 storage blocks, the second layer of hash table includes 500 storage blocks, and the third layer of hash table includes 50 storage blocks. Block. The fourth layer hash table includes 5 storage blocks;

[0077] Set the size of the first layer of Bloom filter to 5000 bytes, and the lower layer is successively reduced to 500 bytes and 50 bytes;

[0078] Step 2, setting the number N of storage units contained in each storage block of the hash table, setting the size of each storage unit to be M bits;

[0079] In t...

Embodiment 2

[0093] The first part of the key-value pair insertion process, combined with the attached image 3 and 5 ,Specific steps are as follows:

[0094] Step 1, set the number of layers of the hash table and the number of storage blocks contained in each layer of hash table, and set the size of the Bloom filter corresponding to each layer of hash table;

[0095] In this embodiment, the settings are the same as in Embodiment 1;

[0096] Step 2, setting the number N of storage units contained in each storage block of the hash table, setting the size of each storage unit to be M bits;

[0097] In this embodiment, the settings are the same as in Embodiment 1;

[0098] Step 3, inserting data into the hash table of the first layer, and performing operations according to judging whether there is a conflict;

[0099] In this embodiment, two hash operations Hash1 (key2) and Hash2 (key2) are performed on key2 in the key-value pair (key2, value2). The two positions corresponding to this val...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a compact-structure key value pair storage structure and a rapid key value pair finding method, and belongs to the technical field of real-time big data processing and key value inquiring. Based on the combined structure of hierarchical hash tables and a Bloom filter, by using a 2-left hashing hash structure for storage on the first layer, each layer of hash table is usedas an auxiliary table for storing data having storage confliction in the table on the upper layer, conflicting data is stored in combination with the Bloom filter, the result of judging whether a found key exists in a current set or not by the Bloom filter serves as an index for improving the finding speed, and therefore the storage space utilization rate is increased, the structure is compact, and the requirement of a software platform for rapid finding is met. The problems that current large-scale key value pair storage and finding time is not determined, finding speed is not consistent, andthe running speed of software is slow and does not meet the application requirement can be effectively solved at a high speed.

Description

technical field [0001] The present invention relates to a compact key-value pair storage structure and a fast key-value (key-value) pair search method, in particular to a multi-level hash table (hash table) and Bloom filter (bloom filter, BF) ) key-value pair storage and search method, belonging to the technical field of real-time big data processing and key-value query. Background technique [0002] With the increasing scale of the Internet and the increasing network traffic, the computer technology field has higher and higher requirements for big data storage and search speed. The key-value pair search problem is widely used in various directions in the computer field, such as big data processing and high-speed networks. [0003] However, the performance of the key-value pair lookup method is severely limited by hardware resources. Therefore, under the trend of network function virtualization, it is more important to adapt to the software platform and adapt to the develop...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/2255G06F16/2443G06F16/24552
Inventor 嵩天魏煜
Owner BEIJING INSTITUTE OF TECHNOLOGYGY