Method, device and system for automatically adjusting and optimizing parameters of key value pair storage engine

A storage engine, key-value pair technology, applied in the storage field

Pending Publication Date: 2019-08-16
NANJING UNIV +2
View PDF3 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Purpose of the invention: Aiming at the defects and deficiencies of the prior art, the present invention proposes an automatic parameter tuning method, device and system for a key-value pair storage engine, which can effectively deal with problems caused by changes in workload under the default parameter configuration. The problem of system performance degradation improves the environmental adaptability of the entire key-value pair storage engine

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method, device and system for automatically adjusting and optimizing parameters of key value pair storage engine
  • Method, device and system for automatically adjusting and optimizing parameters of key value pair storage engine
  • Method, device and system for automatically adjusting and optimizing parameters of key value pair storage engine

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0030] In one embodiment, a runtime parameter automatic tuning method applied to a key-value pair storage engine is provided, which can be applied to a storage engine based on a log structure merge tree. For example, it can be applied to storage systems based on RocksDB encapsulation, including but not limited to RocksDB, MyRocks, and the like.

[0031] Performance in a storage system is largely determined by the type of workload. The log-structured merge tree used by the RocksDB key-value pair storage engine is a write-optimized structure. This structure stores data hierarchically in memory and disk, and performs batch processing after accumulating a certain amount of data, at the expense of read performance. Sequential write I / O is realized under the condition. Therefore, improving the environmental adaptability of the entire key...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method, a device and a system for automatically adjusting and optimizing parameters of a key value pair storage engine, which can solve the problem that the system performance is reduced due to default parameter configuration caused by change of workload in a key value pair storage system. The method comprises the steps of preprocessing parameters according to a parameterconfiguration mode in a key value pair storage engine, and constructing a Restful service framework; constructing a Deep PILCO reinforcement learning model based on the model; operating the workloadof the upper database system, generating data and training a model; loading the model, outputting parameter configuration generated by the strategy for the input workloads, and applying the parameterconfiguration to a bottom key value pair storage engine. The invention is applied to a database system with a log structure merging tree as a storage engine and a scene where the workload changes frequently.

Description

technical field [0001] The invention belongs to the technical field of storage, and in particular relates to an automatic parameter tuning method for a key-value pair storage engine. Background technique [0002] With the continuous development and popularization of cloud computing and big data technology, the amount of data storage is showing an explosive growth trend. According to IDC's forecast in 2019, the global storage machine capacity will double during the 2019-2023 forecast period, growing to 11.7ZB, and the demand for large-scale storage is increasingly strong. With the development of large-scale distributed storage technology, research on the transformation of traditional relational databases has become a hot spot, and many new database systems with RocksDB as the storage engine have emerged. RocksDB is a key-value pair system based on log-structure merged trees. It has the advantage of turning random I / O into sequential I / O, greatly optimizes the performance of ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/21G06N3/04G06N3/08
CPCG06F16/22G06F16/217G06N3/084G06N3/045
Inventor 骆克云唐斌叶保留陆桑璐王剑钱琳
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap