Query statement optimization method and apparatus

A query statement and optimization device technology, applied in the field of data query, can solve problems such as occupation and large storage space

Active Publication Date: 2016-06-22
HUAWEI TECH CO LTD +1
View PDF3 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Embodiments of the present invention provide a query statement optimization method and device to solve the problem of oc...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Query statement optimization method and apparatus
  • Query statement optimization method and apparatus
  • Query statement optimization method and apparatus

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0107] The embodiment of the present invention provides a query optimization method, such as figure 1 shown, including:

[0108] 101. Receive n query statements; wherein, n≥2, and n is an integer.

[0109] The execution body of the method for optimizing query statements provided by the embodiments of the present invention may be a big data query system, and the big data query system may include but not limited to Hive, Shark, or Impala. The method for optimizing a query statement provided by the embodiment of the present invention can be applied in the process of batch querying a data warehouse by a big data query system, wherein the data warehouse can be composed of one or more operands. The query statement optimization method provided by the embodiment of the present invention is specifically an optimization method between query statements.

[0110] In the embodiment of the present invention, in one optimization process, n query statements are used as basic units, and the ...

Embodiment 2

[0158] In this embodiment, an optimization process is taken as an example to illustrate the method for optimizing a query statement provided in Embodiment 1. For an explanation of relevant content in this embodiment, reference may be made to Embodiment 1. The "first type of optimization method" in Embodiment 1 is: in this embodiment, operators whose corresponding operands have the same version number, have fixed value-reference relationship, and belong to different logical query trees are established Precursor-successor relationship; the "second type of optimization method" in the first embodiment in this embodiment is: combining two logical query trees into one logical query tree through a common subtree. The execution subject of this embodiment is a big data query system.

[0159] Such as Image 6 As shown, the optimization method of the query statement provided in this embodiment includes the following steps:

[0160] 601. Receive n query statements, where n≥2, and n is a...

Embodiment 3

[0215] The embodiment of the present invention provides a query optimization device 150 for executing figure 1 The optimization method of the query statement shown, such as Figure 15 As shown, the optimization device 150 of the query statement includes:

[0216] A receiving unit 1501, configured to receive n query statements; wherein, n≥2, n is an integer;

[0217] A generating unit 1502, configured to generate n logical query trees from the n query statements;

[0218] Execution unit 1503, configured to add version numbers to operands corresponding to operators in the n logical query trees according to preset rules; wherein, the preset rules include: different version numbers of different operands, each operator The version numbers of the operands of the second specified value are different, and the version numbers of the same operands corresponding to the operators in different operator groups are different; a group obtained by grouping operators in the n logical query t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Embodiments of the invention disclose a query statement optimization method and apparatus, which relate to the technical field of data query and are used for solving the problem of occupation of a large amount of storage spaces caused by caching of historical logic query trees and corresponding query results of the query trees. The query statement optimization method provided by the embodiment of the invention comprises the steps of receiving n query statements; generating n logic query trees by the n query statements; according to a preset rule, adding version numbers for operands corresponding to operators in the n logic query trees; and optimizing the operators to reduce the number of physical tasks generated by the n logic query trees, wherein the operators belong to different logic query trees and the version numbers of the operands corresponding to the operators are same. The technical scheme provided by the embodiments of the invention can be used in a process that a big data query system performs batch query on a data warehouse.

Description

technical field [0001] The invention relates to the technical field of data query, in particular to a query sentence optimization method and device. Background technique [0002] Currently, the basic unit of big data query systems (such as Hive, Shark, Impala, etc.) to parse and optimize query statements is a single query statement. The process of parsing a query statement by a big data query system generally includes: the big data query system generates a logical query tree (also called a logical query plan) from the query statement, and generates the logical query tree into a query execution engine in the big data query system (for example, MapReduce, Spark, etc.) identifiable physical query tree (also called physical query plan); the query execution engine executes physical tasks in the physical query tree to generate query results; wherein, the logical query tree is composed of multiple operators. [0003] In order to improve query efficiency, a method for optimizing qu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 李丰王蕾张赟冯晓兵
Owner HUAWEI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap