Multi-dimensional grouping operation method and system

A grouping operation, multi-dimensional technology, applied in the field of data processing, can solve the problems of poor database system processing capacity, inflexible dimensional combination conditions, and high latency

Inactive Publication Date: 2014-09-24
ALIBABA GRP HLDG LTD
View PDF5 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The main purpose of this application is to provide a multi-dimensional grouping operation method and system that can provide online multi-dimensional grouping of massive data in real time, so as to solve the problem of poor processing capacity of the database system in the prior art, and the multi-dimensional grouping operation can only be performed offline. Problems such as high latency and inflexible dimension combination conditions caused by calculations, among which:

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-dimensional grouping operation method and system
  • Multi-dimensional grouping operation method and system
  • Multi-dimensional grouping operation method and system

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0046] [Example 1] Suppose there are 5 documents as follows:

[0047] 1. (A, a, I)

[0048] 2. (A, b, I)

[0049] 3. (B,c,II)

[0050] 4. (C, D, III)

[0051] 5. (C, D, III)

[0052] Among them, 1~5 represent the serial number of the document, (A~C), (a~d), (I~III) represent the dimension values ​​corresponding to three different dimensions, then the inverted vocabulary established by these 5 documents is :

[0053] A→1,2 a→1 I→1,2

[0054] B→3 b→2 II→3

[0055] C→4,5 c→3 III→4,5

[0056] d→4,5

[0057] As mentioned above, for example, there is a mapping relationship between dimension value A and document serial numbers 1 and 2 in the inverted vocabulary. By using the inverted vocabulary, the specific location information of each dimension value can be obtained directly, thus making the query faster.

[0058] Next, in step S103, a query request related to a plurality of predetermined dimensions is received from the user terminal, and the query request is sent to each...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multi-dimensional grouping operation method and system. The multi-dimensional grouping operation method comprises steps as follows: fragmenting mass data in a distributed manner to form a plurality of data fragments; creating indexes for data according to each dimensionality in each data fragment, and generating a plurality of created indexes into an index file; performing online multi-dimensional grouping operation on the data by using the index file; and performing combination operation on an operation result of each data fragment. With the adoption of the method and system, online multi-dimensional grouping operation on the mass data can be realized, and a multi-dimensional grouping operation result of ten hundred million grade mass data can be returned in milliseconds.

Description

technical field [0001] The present application relates to the technical field of data processing, in particular to a multi-dimensional grouping operation method and system for massive data. Background technique [0002] At present, in the application system of the data warehouse, there are various analysis requirements for multi-dimensional grouping of massive data. The so-called multi-dimensional grouping is to group massive data (1 billion-level data) in multiple dimensions, and perform summary calculations on the grouped results. The summary operations include sum, max, min, avg, etc. In the face of various analysis requirements, distributed offline computing methods have been used in the past. For example, the traditional Map-Reduce computing method has high latency because it is offline computing. As an online application system, it has high requirements for service response time, especially for online systems that apply multi-dimensional grouping, it is required to r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/2264
Inventor 郑博文袁俊强
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap