Data aggregation query method and apparatus

A data aggregation and database technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of high network overhead and low efficiency, and achieve the effect of improving efficiency and reducing network overhead

Active Publication Date: 2015-11-11
国家超级计算深圳中心(深圳云计算中心) +1
View PDF5 Cites 17 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In view of this, embodiments of the present invention provide a method and device for data aggregatio

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data aggregation query method and apparatus
  • Data aggregation query method and apparatus
  • Data aggregation query method and apparatus

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0023] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0024] figure 1 It shows the implementation flowchart of the data aggregation query method provided by the embodiment of the present invention, and the details are as follows:

[0025] In step S101, when a query request for a database cluster is received, a hash table corresponding to the query request is determined, and multiple partition tables corresponding to the hash table are determined, and the multiple partition tables are associated in the database cluster.

[0026] In the embodiment of the present invention, the database cluster includes at least two database servers. The query request ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention is suitable for the technical field of large-scale data processing and particularly relates to a data aggregation query method and apparatus. The method comprises: when receiving a query request for a database cluster, determining a hash table corresponding to the query request, determining a plurality of partition tables corresponding to the hash table, and generating a MapReduce query task; through scheduling nodes in an Hadoop Yarn framework, according to the MapReduce query task, determining a plurality of subtasks, and distributing the subtasks to a plurality of computing nodes; through the computing nodes, performing the subtasks, obtaining a plurality of computing results, and through the computing nodes, feeding the computing results back to the scheduling nodes; and through the scheduling nodes, simplifying the computing results, and obtaining a query result corresponding to the query request. The method and the apparatus realize relational query and statistics of the relevant partition tables in the database cluster, reduce network overhead, and improve data aggregation query efficiency.

Description

technical field [0001] The invention belongs to the technical field of large-scale data processing, and in particular relates to a data aggregation query method and device. Background technique [0002] In a database cluster, aggregation query is one of the main means of data query and analysis. The query of the database cluster involves multiple nodes in the database cluster. In the existing method of performing aggregation query on the database cluster, the data distributed on multiple nodes is aggregated to the master node, and then the master node executes the aggregation query. [0003] The existing data aggregation query method needs to transmit a large amount of data in the process of aggregating the data of multiple nodes to the master node, and the network overhead is very large. In addition, in the existing data aggregation query method, only the master node aggregates a large amount of data, and the execution of the data aggregation query is limited by the data ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/24553G06F16/24578
Inventor 胡伟黄晓慧黄齐仁李浩陈晓攀熊志强
Owner 国家超级计算深圳中心(深圳云计算中心)
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap