Multidimensional cross data processing method and device

A technology of cross data and processing methods, applied in the field of data processing, can solve problems such as high computing costs, affecting efficiency, hashmap data conflicts, etc., and achieve the effect of improving processing efficiency

Active Publication Date: 2017-09-12
ALIBABA GRP HLDG LTD
View PDF12 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0014] (1) High storage cost
[0015] For example, in the above example, each piece of data has 4 dimensions, with an average of 35 bytes; taking 34 provinces, 4 operators, 4 network standards, and 3 operating systems as an example to perform multi-dimensional cross calculation, the data of each dimension If the value includes all values, the total number of combinations of different values ​​is 35×5×5×4=3500, that is, when writing calculation results to the database, an average of 3500 records are written at a time, and 3500× 35 = 122,500 bytes. In massive real-time analysis, multi-dimensional cross-computing is performed at the granularity of minutes. On average, 5.3G is written to the database every month. In th

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multidimensional cross data processing method and device
  • Multidimensional cross data processing method and device
  • Multidimensional cross data processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0064] Embodiment 1. A multi-dimensional cross data processing method, such as image 3 As shown, including steps S110-S120:

[0065] S110. Perform the following operations on each piece of data to be subjected to multi-dimensional cross calculation: for multiple combinations of dimension values, convert each dimension value combination into Corresponding values, the multiple combinations of dimension values ​​include: the combination of dimension values ​​in this piece of data, and the combination of dimension values ​​when one or more dimension values ​​are changed to all values; the converted values ​​are respectively As the key name of different key-value pairs, the index of the data is used as the key value of each key-value pair;

[0066] S120. For the key-value pairs generated by the data to be subjected to multidimensional cross calculation, respectively calculate the average value of the key value corresponding to each key name.

[0067] In this embodiment, if one o...

other Embodiment approach

[0081] In other implementation manners, it is also possible that only part of the dimension takes all values ​​and the corresponding value is 0; or, when each dimension takes all values, the corresponding value is 1, so that the bit operation is performed on the corresponding bit Set to 1 for operation.

[0082] In an alternative solution of this embodiment, for the key-value pairs generated by the data to be multi-dimensional cross calculation, calculating the average value of the key value corresponding to each key name includes:

[0083]In the key-value pairs generated by the data to be multi-dimensional cross calculation, the key-value accumulation and the number of key-value pairs are counted according to the key name; for each key name, the accumulation result corresponding to the key name is divided by the corresponding number, and get the average value of the key value corresponding to the key name.

[0084] In this alternative, if the operation is performed on a sing...

Embodiment 2

[0102] Embodiment 2. A multi-dimensional cross data processing device, such as Figure 6 shown, including:

[0103] The key-value pair generation module 21 is used to perform the following operations on each piece of data to be multi-dimensional cross calculation: for various combinations of dimension values, each The various combinations of dimension values ​​include: the combination of dimension values ​​in this piece of data, and the combination of dimension values ​​when one or more dimension values ​​are changed to all values ;Use the converted values ​​as the key names of different key-value pairs, and use the index of the data as the key value of each key-value pair;

[0104] The calculation module 22 is configured to calculate the average value of the key value corresponding to each key name for the key-value pairs generated by the data to be subjected to multi-dimensional cross calculation.

[0105] In this embodiment, the key-value pair generating module 21 is a pa...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a multidimensional cross data processing method and device. The method comprises the steps that each piece of data which is about to be subjected to multidimensional cross computation is operated in the mode that various combinations of dimension values are converted into corresponding numerical values respectively according to the one-to-one corresponding relations between dimension value combinations and the numerical values, wherein the various combinations of the dimension values comprise the dimension value combinations in the piece of data and the dimension value combinations obtained when one or more dimension values are changed into all values; the numerical values obtained after conversion serve as key names of different key-value pairs respectively, and the index of the piece of data serves as the key value of each key-value pair; according to the key-value pairs generated by the data which is about to be subjected to the multidimensional cross computation, the average value of the key values corresponding to the key names is calculated. By means of the multidimensional cross data processing method and device, the processing efficiency can be improved.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a multi-dimensional cross data processing method and device. Background technique [0002] When processing data, it is often necessary to perform multi-dimensional cross calculation, that is, to calculate the average value of the combination of different values ​​​​of the data dimension (including the case where the value of the dimension is "all"), and store it in the database for easy query. [0003] An example such as figure 1 Shown: Each piece of raw data includes four dimensions and indicators, the four dimensions are: province, operator, network standard, and operating system. Each dimension has one or more values. For example, the network standard has four values: 2G, 3G, 4G, and WIFI; all values ​​will be added during the calculation process, which is not included in the original data. The value of all values ​​will appear; other dimensions will also increase the value of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/2264
Inventor 熊桥
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap