Implementation method of high-performance data storage engine

A technology of data storage and implementation method, applied in the field of K-V database storage engine, can solve the problems of nonlinear attenuation of storage performance and unfavorable performance, and achieve the effect of ensuring stability and reducing implementation complexity

Active Publication Date: 2015-04-01
TIANJIN NANKAI UNIV GENERAL DATA TECH
View PDF4 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the known solutions, as the amount of data storage increases, it is often necessary to reorganize the data. When the stock data accumulates to a certain scale, the storage performance will experience a serious nonlinear attenuation; in addition, in orde

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Implementation method of high-performance data storage engine
  • Implementation method of high-performance data storage engine

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] The present invention will be further described below in conjunction with specific examples.

[0044] The present invention provides a hash-based K-V database storage engine, which is divided into two parts: metadata and data. For the storage format, refer to the attached figure 1 . as attached figure 1 As shown, the metadata is divided into two parts: a hash table and a deletion mark, and each hash value corresponds to a hash table element. Wherein, each hash table element includes the starting page number of the hash collision chain, the tail page number of the collision chain, and the offset of the free space of the tail page. The number of hash table elements (that is, the number of hash values) is fixed. In addition, the deletion mark is represented by a bitmap, each bit represents a row of data, 0 indicates that the data is valid, and 1 indicates that the data has been deleted. On the page of the data storage space, each piece of data contains the following in...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an implementation method of a high-performance data storage engine. The engine is mainly used for usage scenarios in which data are added in batches, data update operations are fewer, and only equality query based on key values is needed. Therefore, in the aspect of concurrent access, the engine supports write-once read many, namely one write operation and a plurality of read operations can be realized in parallel, and a plurality of write operations are executed serially only. Each piece of data stored in the engine needs one key value, the key values of a plurality of pieces of data can be repeated, and when the data are read, equality query can be only carried out based on the key values. In the aspect of performance, a linear relation is formed between data warehouse performance and the data size of incremental data, the data warehouse performance is only associated with the incremental data, and even if data on stocks reach the scale of hundred billon rows, the warehouse performance cannot be obviously reduced.

Description

technical field [0001] The invention belongs to the field of K-V database storage engines, and in particular relates to a method for organizing data storage structures by utilizing hash values ​​of key values. Background technique [0002] At present, the hash-based K-V database storage engine has been widely used. Due to its low time complexity in query, it can often obtain high query performance for precise query scenarios. However, in the known solutions, as the amount of data storage increases, it is often necessary to reorganize the data. When the stock data accumulates to a certain scale, the storage performance will experience a serious nonlinear attenuation; in addition, in order to support complex The concurrent transaction model and the improvement of transaction throughput often require the introduction of a log mechanism, which is detrimental to the performance of operations such as large-scale data appending and transaction rollback. Contents of the invention ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/2255
Inventor 孟祥滨孔德生崔维力武新
Owner TIANJIN NANKAI UNIV GENERAL DATA TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap