Big-data-based query method and device, computer device and storage medium

Abstract

The present application relates to a big-data-based query method and device, a computer device and a storage medium. The method comprises the following steps of: receiving a query request uploaded bya terminal, wherein query request carries the query criteria, and the query data targeted by the query criteria are stored in the computer node cluster; determining a complexity of a query condition in response to a query request; when the complexity is lower than the preset threshold, calling the query engine Presto to query the corresponding target data in the computer node cluster according tothe query criteria. When the complexity is higher than the preset threshold, the query engine Hive is called to query the corresponding target data in the computer node cluster according to the querycriteria. The queried target data are obtained and sent to the terminal for display. Adopting the method, the corresponding query engine can be adaptively selected in the large data query process.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a query method, device, computer equipment and storage medium based on big data. Background technique [0002] With the development of big data technology, a variety of query engines based on big data have emerged. For example, query engine Hive and query engine Presto. In the traditional way, only a single query engine is used. A single query engine has its own disadvantages. The query engine Hive needs to perform multiple disk reads and writes during the query process, and each read and write to the disk will cause a delay. The query engine Presto is a memory-based distributed real-time query engine, which cannot perform queries that require connection operations on multiple data tables. As a result, there is a problem of long query delay or limited query data volume in the process of big data query. Contents of the invention [0003] Based on this, it is nec...

AI Technical Summary

Problems solved by technology

Traditionally, only a single query engine

A single query engine has its own disadvantages

The query engine Hive needs to perform multiple disk reads and writes during the query process, and each read and write to the disk will cause a delay

The query engine Presto is a memory-based distributed real-time query engine, which cannot execute queries that require connection operations on multiple data tables.

As a result, there is a problem of long query delay or limited query data volume in the process of big data query

Images