Method and system of SQL-like queries based on HBase coprocessor

Abstract

The invention provides a method and a system of SQL-like queries based on an HBase coprocessor. The method and the system can standardize and unify query conditions and returned results, and can execute high-performance parallel computing of real-time data while intrusion into an upper-layer service is avoided. The method includes: preconfiguring the metadata of tables, which are stored by HBase, on an HBase client; establishing an upper-layer service access connection, parsing SQL query statements imported from the upper-layer service, and generating query requests; distributing the query requests to RegionServers by the HBase client; utilizing the HBase coprocessor by the RegionServers to carry out the parallel queries and processing on the data according to the query requests; and merging data, which is returned by the RegionServers, by the HBase client, and encapsulating a result set.

Description

technical field [0001] The invention relates to the field of computer technology and software, in particular to a method and system for SQL-like query based on an HBase coprocessor. Background technique [0002] In recent years, with the continuous development of the e-commerce industry, the storage, extraction, calculation, analysis, and display of big data have attracted more and more attention from major e-commerce companies. With the explosive growth of e-commerce data and the urgent need for big data processing and analysis, and due to the contradiction between data storage space and data access time, major e-commerce companies choose the distributed storage system HBase (HBase is a highly reliable , high-performance, column-oriented, and scalable distributed storage system) to store data to meet the high-efficiency insert and read operations on data. [0003] However, due to the diversification of business requirements, the native API query conditions for HBase and th...

AI Technical Summary

Problems solved by technology

[0005] However, in the above two methods, because Hive is based on the idea of ​​Map-Reduce, the data query response time is long, the real-time performance is poor, and the efficiency of data query and calculation on a smaller order of magnitude is low; while Phoenix has a strong HBase Intrusive, the table created by it can be recognized and used by HBase, but the table created by using HBase cannot be recognized by Phoenix, so it cannot meet the business needs of the existing online HBase data table

Therefore, the existing query methods for HBase cannot solve the problems that the existing HBase query process cannot meet the diversified needs of business needs, low query efficiency, and HBase resources cannot be fully utilized.

Images