A Distributed In-Memory Columnar Database Compiler Executor Architecture

Abstract

The invention discloses a compiling executor architecture of a distributed memory column database, which solves the defect problem of execution speed in the execution mode of a DAG operator in a distributed memory column database. The compiling executor system on a stand-alone machine of the present invention includes a matching and scheduling device, an IR / Bitcode generator, a local IR code cache manager, a compiler, an executable code cache manager and an executor, and the local IR code cache manager includes a local The IR code cache unit, the executable code cache manager includes the executable code cache unit; the matching and scheduler is used to coordinate the local IR code cache manager and the executable code cache manager, manage the mapping information, and DAG for external requests The task performs matching code execution or code generation, and responds to external requests; the IR / Bitcode generator is used to generate IR codes for DAG tasks that do not match executable codes; the local IR code cache unit is used for IR / Bitcode The IR code generated by the Bitcode generator is used for cache management.

Description

technical field [0001] The invention relates to the technical field of executors for distributed memory columnar databases, in particular to a framework for compiling executors for distributed memory columnar databases. Background technique [0002] In the massive data analysis scenario of the big data era, distributed in-memory columnar databases have become irreplaceable and important products relying on their fast query and analysis functions. Unlike traditional databases such as MySQL and Oracle, which store data on a single machine, distributed in-memory columnar databases divide data into columns, and further divide into blocks and store them in multiple nodes. [0003] In a distributed in-memory columnar database, the SQL query statement entered by the user first undergoes lexical analysis to form a syntax tree, and is further processed by a logical optimizer and a physical optimizer to form an execution plan represented by DAG. During execution, each node of the DAG...

AI Technical Summary

Problems solved by technology

[0007] 2. Operators cannot be dynamically generated during operation because they are selected, written and compiled into the system by developers

One is that it is impossible to generate more efficient execution code based on the information in the SQL parsing and optimization phase; the other is that it is impossible to generate new functional operators according to the environment such as heterogeneous scenarios, or support user-defined functions (UserDefined Function, UDF)

Images