Hybrid query optimization method and device based on big data

A query optimization and big data technology, applied in the computer field, can solve problems such as increasing storage overhead, reducing processing efficiency, and wasting storage engine computing resources, and achieves the effect of avoiding storage overhead and reducing interaction frequency.

Pending Publication Date: 2020-06-02
BEIJING QIHOO TECH CO LTD
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, given that some storage engines still have partial computing power, using the existing processing method can easily cause a waste of computing resources of the storage engine; Increased storage overhead and reduced processing efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Hybrid query optimization method and device based on big data
  • Hybrid query optimization method and device based on big data
  • Hybrid query optimization method and device based on big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0028] The hybrid query processing method and device based on big data provided by the present invention can be applied to a preset big data processing system. Such as figure 1 As shown, the preset big data processing system includes: a service interface 11 , an analysis module 12 , a routing module 13 , multiple computing engines 14 and multiple storage engines 15 .

[0029] Wherein, the service interface 11 provides at least one...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a hybrid query optimization method and device based on big data. The method comprises the following steps: converting a received hybrid query statement into a corresponding logic tree; determining a first sub-statement and a second sub-statement according to the connection relationship between the nodes in the logic tree and the information of the nodes; routing the first sub-statement to a corresponding storage engine for the storage engine to execute the corresponding first sub-statement, and obtaining an intermediate query result corresponding to the first sub-statement; and routing the second sub-statement to a corresponding calculation engine, so as to enable the calculation engine to perform calculation processing according to the intermediate query result corresponding to the first sub-statement and the second sub-statement to obtain a final query result. By the adoption of the scheme, the computing capacity of the storage engine can be fully utilized, the interaction frequency between the storage engine and the computing engine is reduced, and the defect that in the prior art, storage expenditure is increased due to the fact that a large amount of source data which are not processed by the storage engine need to be read in for reprocessing can be overcome.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a big data-based mixed query optimization method and device. Background technique [0002] With the continuous development of science and technology and society, all kinds of data are growing in a blowout style, thus providing a data basis for people's various needs. [0003] Due to the diversity of business requirements, it is usually necessary to jointly process data in different storage engines during actual business processing. At present, when jointly processing data in different storage engines, it is usually necessary to export required data from each storage engine to Excel, and process the output data of each storage engine in Excel. [0004] It can be seen that the existing processing method is to read all the source data in the storage engine, and then further perform unified processing on the read data. However, given that some storage engines still have partial c...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2455G06F9/50
CPCG06F9/5027
Inventor 刘思源朱海龙李铭徐胜国徐皓李铮
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap