Data processing method and system for aggregate function in group query

A data processing system and aggregation function technology, applied in the field of big data processing, can solve problems such as low value density, limited computing power of computing nodes, increased processing volume and complexity, and achieve the effect of improving computing power

Active Publication Date: 2021-01-29
INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In today's era of information explosion, data gradually presents the characteristics of massive volume, multiple types, and low value density, which greatly increases the processing volume and complexity of SQL group query calculation operations in the database.
Compared with the explosive growth rate of data, the computing power of computing nodes in the existing technology is limited, so the processing time is also greatly increased

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and system for aggregate function in group query
  • Data processing method and system for aggregate function in group query
  • Data processing method and system for aggregate function in group query

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0044] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Apparently, the described embodiments are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0045] Please refer to figure 1 , figure 1 It is a schematic diagram of the hardware scene of the data processing method of the aggregation function in the group query provided in this application, including a main server 101 , a sub-server 102 , and an FPGA (Field Programmable Gate Array, Field Programmable Gate Array) 103 .

[0046]The main CPU of the main server 101 ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present application discloses a data processing method and system for aggregation functions in group query, including the main CPU dividing the data file to be queried into data blocks, and distributing the data blocks to multiple computing nodes; each computing node CPU respectively The data block is transmitted to the FPGA, and the FPGA is correspondingly connected to the computing node CPU through the PCIe interface; the FPGA obtains the data block, and performs SQL group query on the data block according to the calculation algorithm of the aggregation function in the group query Calculate the aggregation function, and return the calculation result to the main CPU through the calculation node CPU; the calculation algorithm is realized by using Opencl language parallel optimization, and transplanted on the FPGA; the main CPU merges the calculation results to obtain the final query result. The present application improves the processing capability of computing nodes through FPGA parallel acceleration, thereby improving the computing speed of group query aggregation functions.

Description

technical field [0001] The present application relates to the field of big data processing, in particular to a data processing method and system for aggregate functions in group query. Background technique [0002] In recent years, with the development of information technology, big data processing technology, as a high-tech product of the new era, has been widely used in various industries. [0003] Distributed processing technology is used in big data processing, which can connect computing nodes of different types of instruction sets and architectures in different locations using communication networks to form a distributed processing system. Under the unified control and management of the main control unit, each computing node coordinates to complete the data processing task. [0004] Group query calculation is a common data processing task. It can divide the processing objects of the same category into a group for calculation, and obtain the target aggregation function...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/242G06F16/2453
CPCG06F16/244G06F16/24532G06F16/27
Inventor 王丽
Owner INSPUR SUZHOU INTELLIGENT TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap