Data query method and system suitable for hybrid back end

A data query and metadata technology, applied in the field of big data, can solve problems such as the lack of perfect data authority control mechanism

Inactive Publication Date: 2021-02-09
SHANGHAI NETIS TECH
View PDF8 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When providing external services as TableApi, there is only a relatively simple metadata management method, and there is no perfect data permission control mechanism

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data query method and system suitable for hybrid back end

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0044] Such as figure 1 , according to the data query method applicable to the hybrid backend provided by the present invention, including:

[0045] Step 1: API, through API operation instructions, SQL syntax analysis, or other syntax analysis, such as Mongodbpipeline, etc., can realize the syntax tree conversion of different types of query APIs. When querying, first create the query execution environment (including metadata warehouse, custom method warehouse, and configuration information of the data processing engine), and each query will create a query session. Then the query statement is converted into an operation tree and handed over to the query session for execution.

[0046] Step 2: Generate an abstract syntax tree through syntax parsing. This process is processed in the same program as the previous step. Different query APIs need to realize the conversion of these two processes into an operation tree of execution steps at the same time.

[0047] Step 3: The abstrac...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data query method and system suitable for a hybrid back end, and the method comprises the steps: 1, carrying out the SQL grammar analysis through an API operation instruction;2, generating an abstract syntax tree through SQL syntax analysis; 3, converting the abstract syntax tree into a logic table operation tree, and querying and acquiring metadata information of tablesand columns through a verifier; and 4, creating a query time domain through the configured metadata information, and executing the logic operation tree of the query. According to the method, servicescan be directly provided for the final user, the final data storage information is hidden, authority control, data access control and other functions can be provided, and the user can conveniently inquire the data only by knowing the structure of the logic table.

Description

technical field [0001] The present invention relates to the technical field of big data, in particular to a data query method and system applicable to hybrid backends. Background technique [0002] Existing systems such as HIVE, Spark, Flink, and Impala do not provide or only provide PushDown operations for filtering and field extraction when using SQL-like statement computing engines to process data. When providing external services as TableApi, there is only a relatively simple metadata management method, and there is no perfect data permission control mechanism. When providing Api, it is also necessary to understand the format and location of data storage in order to process the data. [0003] Patent document CN1858744A (application number: CN200610064845.2) discloses a data query system and data query method, the data query system includes: multiple databases, each database is connected to a query terminal, and each query terminal and database are connected to a communi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/21G06F16/23G06F16/242G06F16/245G06F16/25
CPCG06F16/2246G06F16/217G06F16/23G06F16/2433G06F16/245G06F16/258
Inventor 朱晓峰蔡晓华杨光辉
Owner SHANGHAI NETIS TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap