Fusion query method based on heterogeneous data source and distributed file system

A technology of distributed files and heterogeneous data sources, applied in the fields of electrical digital data processing, special data processing applications, instruments, etc., can solve difficult data query, inability to achieve system data sharing, connection and fusion query, and inconsistent standards And other issues

Inactive Publication Date: 2017-07-21
南京中新赛克科技有限责任公司
View PDF4 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Under the general trend of convenience and informatization, the problems existing in these numerous data systems have gradually become prominent, mainly in the following two aspects: 1. Each data source is independent, there is no unified standard, and it is difficult to carry

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Fusion query method based on heterogeneous data source and distributed file system
  • Fusion query method based on heterogeneous data source and distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] The present invention will be described in detail below with reference to the accompanying drawings and in combination with specific embodiments.

[0017] figure 1 It is a structural schematic diagram of a fusion query method for a heterogeneous data source and a distributed file system implemented according to the present invention. It contains three basic logic modules: processing node, source data node and maintenance node.

[0018] The source data node is responsible for storing source data information, such as hdfs file block location and structured database information, and caches the source data.

[0019] The maintenance node is responsible for monitoring the health status of the source data node and the processing node. If an abnormal situation occurs on a certain node during the operation, the system will deal with the abnormal situation.

[0020] The processing node is responsible for receiving user requests and interacting with source data nodes to obtain s...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention discloses a fusion query method based on a heterogeneous data source and a distributed file system. The method comprises the steps that: (1) a user initiates a query request to a system, wherein the system comprises a processing node, a source data node, and a maintenance node; (2) the processing node receives the request from the user, and analyzes the request to generate a grammar execution tree; (3) the processing node is interacted with the source data node to obtain source data information of each table and assign tasks according to the different types of source data; (4) the target data source carries out data extraction and analysis according to the request, and returns the filtered data; and (5) the processing node carries out transmission, aggregation and connection operations on the returned data, and returns the processed results to the user. According to the method disclosed by the present invention, the user can easily query the heterogeneous data source, and query different structured databases and the full engine data; and the user can use the distributed query technology to realize the fusion query of the distributed file system and the structured database.

Description

technical field [0001] The invention relates to the technical field of querying heterogeneous data sources, in particular to a fusion query method based on heterogeneous data sources and a distributed file system. Background technique [0002] In recent years, with the rapid development of computer technology and the Internet, the era of information explosion has begun. Society is flooded with more data than ever before, leading to the creation of a wide variety of data systems. Traditional data storage methods are mostly based on relational databases such as myspl, oracle, and sqlserver. In the case of a small amount of data, a good user experience has been obtained. However, with the advent of the era of massive data, the new distributed file system HDFS has been favored by more and more people because of its high fault tolerance and cheaper storage expansion. Full-text search is a key application in the era of big data. As a popular enterprise-level search engine, Elast...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/182G06F16/245
Inventor 何海峰夏飞鹏周艳
Owner 南京中新赛克科技有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap