Real-time task processing method and device

A technology of real-time tasks and processing methods, applied in the field of big data processing, which can solve problems such as increasing system processing time, increasing developer workload, and low development efficiency

Active Publication Date: 2017-09-15
BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The existing real-time processing method of streaming tasks increases the system processing time and the developer's workload for the same or similar logical relationship operations at different times or in different states, and the overall development efficiency is not high.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time task processing method and device
  • Real-time task processing method and device
  • Real-time task processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0049] In order to achieve the purpose of the above invention, the embodiment of the present invention discloses a real-time task processing method, such as figure 1 shown. figure 1 It is a flowchart of a real-time task processing method according to an embodiment of the present invention, including:

[0050] S101. When the stream processing task is currently executing the aggregate function, load the stored encapsulated data packet of the aggregate function ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses a real-time task processing method and device. The real-time task processing method comprises the steps that when a streaming processing task currently executes an aggregate function, a stored encapsulated data packet of the aggregate function executed by the streaming processing task is loaded, wherein the encapsulated data packet comprises operation objects of the aggregate function of the streaming processing task and a function relationship between the operation objects; and aggregate function operation is executed on current values of the operation objects of the aggregate function according to the function relationship between the operation objects, the aggregate function and the encapsulated data packet. Through the embodiment, state management on a real-time data processing process is realized, updating operation is performed conveniently, system processing time is shortened, the workload of developers is lowered, and development efficiency of real-time tasks is improved.

Description

technical field [0001] The invention relates to the technical field of big data processing, in particular to a real-time task processing method and device. Background technique [0002] With the prosperity of the Internet industry, especially the continuous rise of the mobile Internet, more and more data are generated by machines and users, and various big data processing systems have also been born, such as Hadoop, Spark, Storm, etc. Some of these systems are batch processing engines for batch processing tasks, such as Hadoop MapReduce, some are real-time processing engines for streaming data processing, such as Storm, and some systems can simultaneously implement batch processing tasks and real-time processing tasks for streaming data processing , such as Spark. [0003] In the batch big data processing system, all the data is ready when the calculation starts, waiting for the input calculation; while the streaming data processing engine, the data is not completely ready ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/30
CPCG06F9/3005G06F9/30101
Inventor孙琦
OwnerBEIJING QIYI CENTURY SCI & TECH CO LTD