A topological structure updating system and method for stream processing

A technology of topology structure and update method, applied in the field of data processing, can solve the problem that the big data stream processing architecture is not suitable for online topology update, etc., and achieve the effect of realizing hot deployment, solving online topology update, and realizing online update.

Active Publication Date: 2019-05-07
BEIJING GRIDSUM TECH CO LTD
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present invention provides a stream processing topology update system and method to solve the technical problem that the existing big data stream processing architecture is not suitable for online topology update

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A topological structure updating system and method for stream processing
  • A topological structure updating system and method for stream processing
  • A topological structure updating system and method for stream processing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0036] The embodiment of the present invention provides an embodiment of a stream processing topology update system, figure 2 is a schematic structural diagram of a topology update system according to an embodiment of the present invention, such as figure 2 As shown, the system includes: a task manager 10 and a task executor 12, wherein,

[0037] The task manager 10 is configured to detect the first state of the first state machine, and generate a control message according to the above state, wherein the above first state is used to trigger the management of the topology structure of the stream processing; the task executor 12 is used to receive The aforementioned control message; and triggered by the aforementioned control message, triggering a second state of the second state machine, and performing an action corresponding to the aforementioned second state on the aforementioned topological structure according to the second state.

[0038] In the embodiment of the present...

Embodiment 2

[0073] According to an embodiment of the present invention, an embodiment of a topology update method for stream processing is provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0074] Figure 8 A flowchart of the steps of a stream processing topology update method according to an embodiment of the present invention, as shown in Figure 8 The method includes the following steps:

[0075] Step S102, acquiring a control message, wherein the above control message is a message generated according to the first state of the first state machine, and the above first state is used to trigger the management of the topology structure of the stream processing;

[0076] In step S...

Embodiment 3

[0088] The embodiment of the present invention also provides a device for implementing the above topology update method for stream processing, Figure 9 is a schematic structural diagram of a stream processing topology update device according to an embodiment of the present invention, as shown in Figure 9 As shown, the control device for the above range hood includes: an acquisition module 100 and a trigger module 102, wherein,

[0089] The obtaining module 100 is configured to obtain a control message, wherein the above-mentioned control message is a message generated according to the first state of the first state machine, and the above-mentioned first state is used to trigger the management of the topology structure of the stream processing; the triggering module 102 is used for Triggered by the control message, a second state of the second state machine is triggered, and an action corresponding to the second state is performed on the topology structure according to the se...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a topological structure updating system and method for stream processing. The system comprises a task manager used for detecting a first state of a first state machine and generating a control message according to the state, wherein the first state is used for triggering a topological structure for stream processing to be managed; The task executor used for receiving the control message; And triggering a second state of a second state machine under the triggering of the control message, and executing an action corresponding to the second state on the topological structure according to the second state. The technical problem that an existing big data stream processing architecture is not suitable for online stream topology updating is solved.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a topology update system and method for stream processing. Background technique [0002] With the development of Internet technology, more and more industries require fast and efficient data processing, and the timeliness of data processing is getting higher and higher. The forms of information are continuously entering the system, and need to be processed at the fastest speed; another example is the timeliness of news, which also requires that major events be recommended to users as soon as possible, so as to increase user stickiness and bring considerable traffic. [0003] Figure 1a is a DAG topology diagram of an application in the prior art according to an embodiment of the present invention, such as Figure 1a As shown, the currently deployed and running DAGs include Task1, Task2, Source, and Sink. After the existing stream processing framework submits the applicatio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F8/65G06F8/20G06F9/445G06F11/07
Inventor 赵懿
Owner BEIJING GRIDSUM TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap