Log pre-writing method and system based on RocksDB

A log and log file technology, applied in the field of database data security, to improve recovery speed, improve read and write performance, and avoid write speed problems

Pending Publication Date: 2022-03-04
上海沄熹科技有限公司
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, it poses a great challenge to both data storage devices and databases.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Log pre-writing method and system based on RocksDB
  • Log pre-writing method and system based on RocksDB

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0050]The RocksDB-based write-ahead log method includes the following steps:

[0051] The first step, the write-ahead log is written asynchronously

[0052] Including multi-threaded writing, reading and statistical task management and scheduling; by enabling a fixed number of threads bound to the core, accept, manage, execute and return the writing r...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention particularly relates to a log pre-writing method and system based on RocksDB. According to the log pre-writing method and system based on the RocksDB, a fixed number of threads bound with cores are started to accept, manage, execute and return a writing request of NVMe SSD equipment; partitioning, managing and optimizing the hybrid storage device by using a self-made file system; reading data of the high-speed NVMe SSD equipment by using the SPDK to realize pre-write log recovery; a synchronization mechanism of the pre-write file system is realized by using a sliding window algorithm and/or a bitmap algorithm and a busy waiting technology. According to the log pre-writing method and system based on the RocksDB, the high storage performance of the NVMe SSD can be fully played, the throughput speed of a database is greatly increased, it can be guaranteed that all hardware devices can be fully utilized, the problem of the writing speed of the database caused by the slow disk writing speed is avoided, the recovery speed of the database is increased, and the recovery efficiency of the database is improved. And the reading and writing performance of the database is greatly improved.

Description

technical field [0001] The invention relates to the technical field of database data security, in particular to a RocksDB-based pre-write log method and system. Background technique [0002] With the continuous development and improvement of flash memory technology, the capacity of solid-state disks built using NAND flash memory is getting larger and lower, and the price is getting lower and lower, and the storage cost per byte is getting lower and lower. Many enterprises can already afford the price of SSD. With the technological innovation of solid-state drives, the number of erasures that solid-state drives can withstand has been greatly increased. At the same time, compared with mechanical hard drives, solid-state drives have extremely high data transfer rates. Solid-state drives use SATA 3 interfaces, and the sequential read and sequential write speeds are both 400MB to 500MB per second. For NVMe (Non -Volatile Memory Express, non-volatile memory standard) SSD (hard d...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F16/13G06F16/172G06F16/178G06F16/18G06F16/23G06F16/27G06F11/14
CPCG06F3/0611G06F3/0643G06F3/0644G06F3/0659G06F3/068G06F16/13G06F16/172G06F16/178G06F16/1847G06F16/2308G06F16/27G06F11/1469G06F2201/80
Inventor 王泽坤
Owner 上海沄熹科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap