Merging device based on programmable device, data merging method and database system

A programming device and database technology, applied in the database field, can solve the problems of consuming CPU resources and reducing the reading and writing performance of the database system, and achieve the effects of reducing the occupation of CPU resources, improving performance jitter, and reducing the impact of reading and writing performance

Active Publication Date: 2020-09-04
ALIBABA GRP HLDG LTD
View PDF6 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the existing data merging process consumes a lot of CPU resources and reduces the read and write performance of the database system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Merging device based on programmable device, data merging method and database system
  • Merging device based on programmable device, data merging method and database system
  • Merging device based on programmable device, data merging method and database system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and corresponding drawings. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0021] In a database system using hierarchical storage, data records are first written into the memory table (memtable) in the memory, and after the memtable reaches a certain size (such as 64MB), it is then flushed to the SST file on the disk. Since this method is an all-append writing method without random writing, it takes advantage of the fact tha...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a merging device based on a programmable device, a data merging method and a database system. In the embodiment of the invention, based on a programmable device, the merging device comprising at least two mergers is realized. The merging device based on the programmable device is applied to a database system; wherein the merging unit is responsible for merging data records needing to be merged in a database system; the processing burden of the CPU can be unloaded, the occupation of CPU resources in the database system by data merging operation is reduced, the influence of the data merging operation on the read-write performance of the database system is reduced, the overall capacity of the database system is improved, and the problem of performancejitter is solved; in addition, the merging device of the embodiment comprises a plurality of mergers for executing different merging logics, so that diversified data merging requirements can be met; and different combiners are mutually independent, so that different data combination tasks can be prevented from being mutually blocked, and each data combination task has the characteristic of low time delay.

Description

technical field [0001] The present application relates to the technical field of databases, in particular to a programmable device-based merging device, a data merging method and a database system. Background technique [0002] Common key-value (Key-Value, KV) databases based on the LSM (Log-structured merge-tree) data structure include LevelDB and RocksDB based on the evolution of LevelDB. In LevelDB or RocksDB, most of the KV records are stored in the disk using a hierarchical storage method. The general process is: write the KV records to the memtable in the memory (memory table is a data structure that stores data in memory ), after the memtable reaches a certain size (such as 64MB), it is flashed to the SSTable (Sorted String Table, SST) file on the disk. [0003] Considering that there may be key overlaps between SSTable files in different time periods, the existing technology uses compaction to organize and compress existing KV records, and reorder files with overlap...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/21G06F16/2455G06F9/50
CPCG06F16/22G06F16/21G06F16/24552G06F9/5016Y02D10/00
Inventor 任麒斌
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap