Fusion query method based on heterogeneous data source and distributed file system

Abstract

The present invention discloses a fusion query method based on a heterogeneous data source and a distributed file system. The method comprises the steps that: (1) a user initiates a query request to a system, wherein the system comprises a processing node, a source data node, and a maintenance node; (2) the processing node receives the request from the user, and analyzes the request to generate a grammar execution tree; (3) the processing node is interacted with the source data node to obtain source data information of each table and assign tasks according to the different types of source data; (4) the target data source carries out data extraction and analysis according to the request, and returns the filtered data; and (5) the processing node carries out transmission, aggregation and connection operations on the returned data, and returns the processed results to the user. According to the method disclosed by the present invention, the user can easily query the heterogeneous data source, and query different structured databases and the full engine data; and the user can use the distributed query technology to realize the fusion query of the distributed file system and the structured database.

Description

technical field [0001] The invention relates to the technical field of querying heterogeneous data sources, in particular to a fusion query method based on heterogeneous data sources and a distributed file system. Background technique [0002] In recent years, with the rapid development of computer technology and the Internet, the era of information explosion has begun. Society is flooded with more data than ever before, leading to the creation of a wide variety of data systems. Traditional data storage methods are mostly based on relational databases such as myspl, oracle, and sqlserver. In the case of a small amount of data, a good user experience has been obtained. However, with the advent of the era of massive data, the new distributed file system HDFS has been favored by more and more people because of its high fault tolerance and cheaper storage expansion. Full-text search is a key application in the era of big data. As a popular enterprise-level search engine, Elast...

AI Technical Summary

Problems solved by technology

Under the general trend of convenience and informatization, the problems existing in these numerous data systems have gradually become prominent, mainly in the following two aspects: 1. Each data source is independent, there is no unified standard, and it is difficult to carry out data processing through a unified interface. Data query; 2. Each application system is established based on different data sources. Due to the inconsistent standards among data sources, it is impossible to realize data sharing, connection and fusion query between systems

Images