Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Mergejoin asynchronous block operation method based on kv storage

An operation method and asynchronous technology, applied in the field of operator optimization, can solve problems such as low CPU utilization, large function call depth, and slow execution efficiency, and achieve the effects of avoiding multiple calls, reducing overhead, and improving utilization

Active Publication Date: 2022-05-31
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] Currently, in the KV storage mode, the mergejoin algorithm has the following problems: the execution process is serialized, and each module must wait for another module it calls to return the result before executing, the CPU utilization rate is not high, and the execution efficiency is slow
The matching results are returned by row, and each function will be called repeatedly. The function call depth is large and the overhead is large.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mergejoin asynchronous block operation method based on kv storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] NewSQL technology has developed very rapidly in recent years, and there are relatively mature systems at home and abroad. In the implementation of some open source systems in the industry, Rocksdb plays the core storage and query functions. Rocksdb is a high-performance embedded persistent key-value storage. The storage of many open source databases is embedded Rocksdb, which stores data in a key-value storage format.

[0027] Mergejoin is a query algorithm in relational query. Associative query is a query method in which multiple objects in the database use a certain connection relationship to query matching results. There are mainly inner joins, left joins, right joins, full joins, and reverse joins. etc., if the associated columns of the two tables are the primary key columns or index columns, and they are arranged in ascending / descending order (or the two columns have been sorted in the same order before the query), then use the two columns to be equal The mergejoin...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a mergejoin asynchronous block operation method based on KV storage, which belongs to the field of operator optimization technology. The method simultaneously starts multiple mergejoin operators for parallel calculation, and the mergejoin operators are executed asynchronously through pipeline communication; including a data reading module , the decoding module and the merge operation module, the reading data module sends the data block to the reading pipeline, the decoding module receives the data from the pipeline, and sends the decoded data in the form of data blocks to the decoding pipeline, the merge operation module establishes The result pipeline monitors the decoding pipeline, receives data blocks from the pipeline to perform merge matching operations, and sends the operation results to the result pipeline in the form of data blocks according to the set rules. The invention can effectively improve the utilization rate of the cpu, avoid multiple calling among modules, and reduce calling overhead.

Description

technical field [0001] The invention relates to the technical field of operator optimization, in particular to a mergejoin asynchronous block operation method based on KV storage. Background technique [0002] Currently, in the KV storage mode, the mergejoin algorithm has the following problems: the execution process is serialized, and each module has to wait for another module it calls to return the result before executing, the CPU utilization rate is not high, and the execution efficiency is slow. Returning the matching result by row will call each function repeatedly, and the function call depth is large and the overhead is large. Contents of the invention [0003] The technical task of the present invention is to provide a mergejoin asynchronous block operation method based on KV storage for the above deficiencies, which can improve the utilization rate of cpu, avoid multiple calls between modules, and reduce the overhead of calls. [0004] The technical solution adop...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/50G06F9/54
CPCG06F9/5038G06F9/544G06F9/546Y02D10/00
Inventor 柴毅张晖史大义牟冠学
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com