Unlock instant, AI-driven research and patent intelligence for your innovation.

A method and device for querying massive data in a distributed system

A distributed system and mass data technology, applied in the field of mass data query, can solve problems such as no technical solutions, and achieve the effect of improving query efficiency

Active Publication Date: 2021-04-02
SHANXI CHINA MOBILE COMM CORP
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Regardless of which of the above HIVE and Impala query schemes is used, there will be their own problems, however, there is no effective solution for this in related technologies

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and device for querying massive data in a distributed system
  • A method and device for querying massive data in a distributed system
  • A method and device for querying massive data in a distributed system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] The implementation of the technical solution will be further described in detail below in conjunction with the accompanying drawings.

[0061] The query method of massive data in the distributed system of the embodiment of the present invention, such as figure 1 As shown, the method includes:

[0062] Step 101, analyzing the received query request to obtain execution tasks generated by statements used to characterize business analysis requirements;

[0063] Step 102, select HIVE query engine or Impala query engine to perform distributed query on the execution task according to the size of the data file in the execution task and the available memory of the cluster, so as to generate the statement used to represent the business analysis requirements in the execution task The corresponding new execution path.

[0064] In an embodiment of the present invention, the method further includes: storing the data required by the query request in a distributed manner. Specifical...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a query method of mass data in a distributed system and a query device. The method comprises the following steps: resolving a received query request to obtain an execution task generated by the statement for presenting the service analysis demand; selecting a HIEV query engine or an Impala query engine to perform the distributed query on the execution task according to the data file size in the execution task and the cluster available memory, thereby generating a new execution path corresponding to the statement for representing the service analysis demand in the execution task.

Description

technical field [0001] The invention relates to data query technology, in particular to a method and device for querying massive data in a distributed system. Background technique [0002] Big data is now reaching every sector of the global economy. Like other essential elements of production (for example, hard assets and human capital), many modern economic activities simply cannot take place without it. The use of big data is becoming an important way for leading companies to outperform their peers in terms of performance. Businesses can use data to design products that better match customer needs. Data can even be used to improve products in use. One such example is that a cell phone that learns about the user's habits and preferences, loaded with apps and data tailored to that specific user's needs, is more valuable than a new, non-customized device. [0003] In order to make more effective use of these data and enhance the competitiveness of enterprises, there must ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/2458
CPCG06F16/2471
Inventor 卢山
Owner SHANXI CHINA MOBILE COMM CORP