Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Data processing method and apparatus

A data processing and data technology, applied in the Internet field, can solve the problems of low real-time processing efficiency and different number of threads, and achieve the effect of self-adaptive segmentation processing, improving processing speed and improving efficiency.

Active Publication Date: 2018-06-22
ALIBABA GRP HLDG LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0009] The purpose of this application is to provide an improved data processing technology to solve the problem of low real-time processing efficiency caused by the different number of threads actually used by concurrent processing.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and apparatus
  • Data processing method and apparatus
  • Data processing method and apparatus

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] The main idea of ​​this application is to replace the prior art that the processing of the same dimension identifier is assigned to the same thread for execution, and for those concurrent processing of the multiple concurrent processing of the same event stream data, the number of dimension identifiers is equivalent to the number of predetermined threads, By assigning the processing identified by the same dimension to multiple different threads for parallel execution, the processing speed of these concurrent processing is accelerated, threads are avoided, and all concurrent processing is completed as much as possible at the same time, thereby improving the overall real-time processing efficiency of event stream data. It is convenient for users to use.

[0021] On this basis, the idea of ​​this application is to realize adaptive adjustment of the number of threads allocated to each dimension identifier corresponding to processing according to the amount of processing invo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data processing method and device. The method comprises the steps that for all dimensionality processing in multi-dimensional processing, whether the absolute value of the difference between the dimensionality identification number and a preset thread number is smaller than or equal to a preset threshold is judged; when the absolute value of the difference between the dimensionality identification number and the preset thread number is smaller than or equal to the preset threshold, all pieces of processing corresponding to all dimensionality identifications are distributed to a preset number of multiple threads to be executed, and therefore the pieces of processing corresponding to the same dimensionality identification are completed on multiple threads in parallel; furthermore, when the absolute value of the difference between the dimensionality identification number and the preset thread number is larger than the preset threshold, all pieces of processing corresponding to all dimensionality identifications are distributed to corresponding single threads in a preset number of threads to be executed, and therefore the pieces of processing corresponding to the same dimensionality identification are completed on the same thread. In this way, all pieces of concurrent processing of event flow data can be completed at the same time, so that the real-time processing efficiency is improved, and use of a user is facilitated.

Description

technical field [0001] The present application relates to the Internet field, and more specifically relates to a data processing method and device for multi-dimensional processing of event stream data. Background technique [0002] With the rapid development of the Internet, online data interaction is becoming more and more common, and it has become the main way of data interaction in people's lives. [0003] In the process of online data interaction, due to the need for efficient and real-time processing of massive online interactive data, event streaming technology is generally used to process massive interactive data. The so-called event flow means that the same kind of events occurring at different times in the same data source constitute a strictly ordered flow in time. That is, the same type of data is formed into multiple events, and when multiple events are triggered in a certain order, an event flow is formed. Statistical processing on multiple dimensions can be p...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30G06Q10/08
Inventor 陈岳阳李经纬
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products