Implementation method for operator reuse in parallel database

An implementation method and database technology, applied in the field of database systems, can solve problems such as not considering operator reuse, resource waste, etc.

Active Publication Date: 2012-01-18
天津神舟通用数据技术有限公司
View PDF3 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In this case, each process or thread executes its own query on

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Implementation method for operator reuse in parallel database
  • Implementation method for operator reuse in parallel database
  • Implementation method for operator reuse in parallel database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

[0022] By scanning the multi-threaded parallel plan in the query optimization phase of the database to find reusable operators, modify the reusable operators into global shared operators, and change the plan structure, turning the plan tree into Plan the forest and further rewrite it as a directed graph. By executing the directed graph plan in parallel, the intermediate results of the materialized operators are reused during plan execution.

[0023] This method mainly comprises the following steps:

[0024] Plan generation phase:

[0025] step one:

[0026] Use the common query planning method to generate a serial query plan for the query, which is a binary tree structure. The query involves the partition table, so some leaf nodes are scans of the partition table. like figure 1 As shown, a certain query is select*from A, B, P where A.a=B.b...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an implementation method for operator reuse in a parallel database, comprising the following steps of: step 1, generating a serial query plan for query through a normal query planning method, wherein the query plan is a binary tree structure; step 2, executing the query plane by scanning from top to bottom, searching materialized reusable operators, changing the query plane structure, and changing thread level materialized operators into global reusable materialized operators; step 3, parallelizing the query plan changed in the step 2, and generating a plan forest for parallel execution of a plurality of threads; step 4, executing global reusable operator combination on the plan forest generated in the step 3, and generating a directed graph plan for the materialized reusable operators capable of being executed by the plurality of threads in parallel; step 5, executing own plan part in the directed graph by each thread in parallel, wherein the thread which executes the global reusable operator firstly is called a main thread, the main thread locks the global reusable operator and truly executes the operator and the plan of the operator, and other threads wait; step 6, unlocking the global reusable operator by the main thread after execution, wherein other threads start to read data from the global reusable operator and continue to execute own plan tree;and step 7, releasing the materialized data of the operator by the main thread after all the plans read the data of the global reusable operator.

Description

technical field [0001] The invention relates to a database system, in particular to a method for realizing operator multiplexing of a parallel database. Background technique [0002] With the development and popularization of information technology, data is rapidly expanding at an exponential rate, and processing massive data has become an important problem in the computer field. The research on OLAP, DSS, data mining, etc. emerging in the database field is essentially the research on massive data processing. [0003] At present, the popular technologies for solving massive data processing problems are parallel query technology and cluster technology. Parallel query technology has always been a research hotspot in the database field. The academic community has proposed a variety of parallel query architectures: Share-Everything (completely shared) architecture, Share-Memory (shared memory) architecture, Share-Disk (shared disk) Architecture and Share-Nothing (no sharing) a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 李阳何清法顾云苏冯柯蒋志勇徐岩饶路李晓鹏刘荣赵婧
Owner 天津神舟通用数据技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap