A cross-data center fast query method and system based on mpp engine
A technology across data centers and data, applied in the field of big data retrieval, it can solve problems such as narrow problem domain, inability to express through SQL, and difficult expansion of relational databases, so as to reduce analysis time, ensure global consistency, and avoid disk IO operations. Effect
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment Construction
[0041] The present invention will be further described below through specific embodiments and accompanying drawings.
[0042] The content of the present invention mainly includes the following aspects.
[0043] First, regarding the metadata identification, the present invention adopts a unified metadata identification, and utilizes the Hive metadata component to uniformly mark the data in different MPP engines. Including the storage structure and storage type to which the data table belongs. And use the corresponding MPP engine for fast query. The different MPP engines described in the present invention include Hive, Spark, HBase, etc., and the above-mentioned engines are integrated together in an MPP manner, centrally scheduled, and used in an MPP manner.
[0044] Second, in terms of data transmission, the present invention provides efficient and reliable data transmission through confirmation, retransmission and other mechanisms. In the data center, query through JDBC / ODB...
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More 



