Unlock instant, AI-driven research and patent intelligence for your innovation.

A KV database storage and query method indexed by partial hash key

A query method and database technology, applied in the field of Kv database storage and query, can solve problems such as database stability decline, database database blocking, etc. Effect

Active Publication Date: 2021-07-27
北京顺实科技有限公司
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] For this reason, the present invention provides a kind of Kv database storage and query method with partial hash Key as the index, in order to overcome in the prior art when the amount of data stored in a single machine is too large, the stability of the database decreases, causing the database database to be blocked for a long time question

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A KV database storage and query method indexed by partial hash key
  • A KV database storage and query method indexed by partial hash key
  • A KV database storage and query method indexed by partial hash key

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0104] Embodiment 1. A single machine writes 750 million pieces of data. This is a limit test. The limit number of records in LevelDB is about 100 million. After the writing is successfully completed, the specific storage conditions are as follows:

[0105] project size content_db 698G One-level index + two-level index 2.3G fullKeys 18G fast_confluct 8.8G conflict_db 2.7G

[0106] In this embodiment, it takes 4.5 hours to load 750 million pieces of data, with an average of 46,000 pieces per second.

[0107] In this embodiment, after writing 750 million pieces, the working status of each module of the present invention is as follows:

[0108] project Times / ms slowest ms Total time spent in ms Total times conflict_db read 3 1843 12032678 40132197 conflict_db write 13 2635 6147489 80854022 fast_conflict read 33 2406 11231663 372366254 fast_conflict write 71 3492 7558215 540879018...

Embodiment 2

[0112] Example 2: Write 100 million records, insert 52,100 records per second, and take a total of 1920 seconds, which is 32 minutes; write 300 million records, insert 37,800 records per second, and take a total time of 7940 seconds, also That's 132 minutes.

[0113] In this embodiment, after writing 100 million pieces, the working status of each module of the present invention is as follows:

[0114] project Times / ms slowest ms Total time spent in ms Total times conflict_db read 0 599 29600 4086 conflict_db write 0 785 685271 67329 fast_conflict read 349 2 2858 997451 fast_conflict write 29 574 355041 10323821 full_Key read 1221 4 15501 18941274 full_Key write 52 1277 1460011 76955518 content_db write 15 246 4994556 77576465 content_db update 66 4 5139 343528

[0115] The statistics of the number of conflicts are as follows:

[0116] Number of index key conflicts The num...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a method for storing and querying a Kv database with a partial hash Key as an index, and belongs to the technical field of databases. The present invention obtains the original Key of the user data, converts the original Key into a hash Key and selects different parts from the hash Key as the index Key and the verification Key; using the index Key as the database index, when the When data is written into the database, the index key is used to locate the index item. When the location is completed, it is checked whether there is data in the address corresponding to the index item. If there is no data, the data is directly written into the index item. If there is data, the data is obtained. The conflict status of the index item: performing corresponding processing on the written data according to the obtained conflict status of the index item. The present invention reduces the data volume of the index Key to about 20%, and introduces a special conflict handling mechanism to reduce index conflicts, thereby further improving the stability of the database and increasing the storage and query speed.

Description

technical field [0001] The invention relates to the technical field of databases, in particular to a method for storing and querying a Kv database indexed by a partial hash Key. Background technique [0002] Kv database refers to the Key-value database, which is a kind of database that stores data in key-value pairs. It has fast query speed, large amount of stored data, and supports high concurrency. It is very suitable for querying through primary keys, but it cannot perform complex conditional queries. , supplemented by a real-time search engine for complex condition retrieval and full-text retrieval, it can replace relational databases such as MySQL with low concurrency performance, achieve high concurrency, high performance, and save dozens of times the number of servers. [0003] Currently mainstream Kv databases are represented by BerkeleyDB and LevelDB. [0004] BerkeleyDB addresses different storage needs by using a variety of storage structures. However, no matter...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22G06F16/2455
CPCG06F16/2255G06F16/2272G06F16/2455
Inventor 邹月明
Owner 北京顺实科技有限公司