Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Distributed OLAP ad-hoc query method based on Spark and Hive

A query method and distributed technology, applied in the field of distributed OLAP ad hoc query, can solve problems such as insatiable performance and soaring

Pending Publication Date: 2020-11-20
北京海致科技集团有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In recent years, with the development of technology, enterprises have increasingly high demand for query and analysis systems for various reports. Most of the traditional analysis systems are based on relational databases such as MySQL, and the performance is difficult to meet the increasing query and analysis of massive data.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed OLAP ad-hoc query method based on Spark and Hive
  • Distributed OLAP ad-hoc query method based on Spark and Hive
  • Distributed OLAP ad-hoc query method based on Spark and Hive

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0017] like figure 1 As shown, this patent includes seven steps:

[0018] 1, figure 1 It is the execution process of the entire OLAP engine, starting from OLAP Query, which can trigger OLAP Query in any way, and is the first stage of engine execution.

[0019] 2. After the Query obtains the Query Keyword, the Validator (verification detection) of the keyword will be performed to verify the correctness and security of the keyword to avoid malicious Query.

...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a distributed OLAP ad-hoc query method based on Spark and Hive, which comprises the following steps: 1, starting from OLAP Query, triggering OLAP Query in any mode, which is the first stage of engine execution; 2, after the Query is collected by the Query, Validators of the keywords are carried out, namely, verification and detection are carried out; 3, after verification,if the verification result is passed, performing subsequent Cache operation; 4, carrying out Router route matching; 5, carrying out SQL (Structured Query Language) Rewrite; and 6, executing by the execution engine at the current stage, and obtaining a corresponding result. The method has the advantages that the method not only supports mass data, but also can support efficient statistical query ofany dimension and time span under the mass data, and a flexible multi-dimensional combined query tool is provided and corresponds to service analysis of different dimensions.

Description

technical field [0001] The invention relates to the technical field of big data multidimensional analysis engines, in particular to a distributed OLAP impromptu query method based on Spark and Hive. Background technique [0002] In recent years, with the development of technology, enterprises have increasingly high demand for query and analysis systems for various reports. Most of the traditional analysis systems are based on relational databases such as MySQL, and their performance cannot meet the growing query and analysis of massive data. Due to the rapid development and improvement of the big data technology platform, real-time core business data is gradually applied to the big data platform. OLAP is Online Analytical Processing. The core of OLAP is analysis, focusing on analysis and decision-making, and the magnitude of the analyzed data is very large. The technical difficulties of OLAP implementation are completely matched with the Hadoop-based ecological big data pla...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2458G06F16/28
CPCG06F16/2471G06F16/283
Inventor 刘启涛
Owner 北京海致科技集团有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com