Unlock instant, AI-driven research and patent intelligence for your innovation.

Stream computing system, data processing method thereof, electronic device and medium

A data processing and stream computing technology, applied in the field of data processing, can solve problems such as the impact of computing timeliness, and achieve high timeliness effects

Active Publication Date: 2021-02-12
北京智慧星光信息技术有限公司
View PDF9 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, when there is a data flood in the data source of the input stream computing engine, the amount of data arriving is greater than the amount of data passing through, and the system will enter a state of back pressure, which will have a serious impact on the timeliness of computing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stream computing system, data processing method thereof, electronic device and medium
  • Stream computing system, data processing method thereof, electronic device and medium
  • Stream computing system, data processing method thereof, electronic device and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] Specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0036] figure 1 is a schematic flowchart of a data processing method using a flow computing system according to the present invention, figure 2 It is a schematic diagram of the data flow direction of the data processing method using the stream computing system in the present invention, such as figure 1 with 2 As shown, the data processing method includes:

[0037] Step S1, constructing a data access layer connecting the data source and the stream computing engine of the stream computing system, the data access layer includes a time-sensitive queue and a stacking queue, and the stream computing engine includes a time-sensitive data flow graph and a stacked data flow graph;

[0038] Step S2, receiving the data of the data source through the data access layer;

[0039] Step S3, checking the length of data cached in the aging queue of the dat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a stream computing system, a data processing method thereof, an electronic device and a medium. The stream computing system comprises a stream computing engine and a data accesslayer. The data access layer comprises an aging queue and a stacking queue, the stream computing engine comprises an aging data flow diagram and a stacking data flow diagram, the data access layer receives data of a data source, checks the length of the cached data in the aging queue, and if the aging queue has residual space, the data is directly enqueued into the aging queue; if the aging queuehas no residual space, the data is directly enqueued into a stacking queue; wherein the aging data flow diagram consumes the data of the aging queue, and the accumulation data flow diagram consumes the data of the accumulation queue. According to the invention, the flow calculation engine has the characteristics of aging sensitivity and accumulation resistance at the same time.

Description

technical field [0001] The present invention relates to the technical field of data processing, and more specifically, to a stream computing system and a data processing method thereof, electronic equipment, and a computer-readable storage medium. Background technique [0002] Stream computing is a computing model widely used in real-time big data processing. The stream computing model regards the data source as an infinite flow in terms of time distribution and quantity, and the data is continuously generated continuously, and its value gradually decreases as time goes by. In the flow computing model, different operators are connected in series in the form of a directed acyclic graph (DAG), and the incoming data is processed sequentially from top to bottom. [0003] In the current mainstream stream computing engine (such as Flink), the same operator processes data in parallel in the form of multiple instances, and different operators are connected in the form of remote cal...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/2455G06F16/2453G06F9/50G06F9/54
CPCG06F9/5016G06F9/5027G06F9/546G06F2209/548G06F16/24532G06F16/24552G06F16/24568
Inventor 赵冲赵自波李青龙骆飞
Owner 北京智慧星光信息技术有限公司