Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Topology update system and method for stream processing

A technology of topology structure and update method, which is applied in the field of data processing, can solve the problems that the big data flow processing architecture is not suitable for online topology update, etc., and achieve the effect of hot deployment, online topology update, and online update

Active Publication Date: 2022-04-26
BEIJING GRIDSUM TECH CO LTD
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The embodiment of the present invention provides a stream processing topology update system and method to solve the technical problem that the existing big data stream processing architecture is not suitable for online topology update

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Topology update system and method for stream processing
  • Topology update system and method for stream processing
  • Topology update system and method for stream processing

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0036] The embodiment of the present invention provides an embodiment of a stream processing topology update system, figure 2 is a schematic structural diagram of a topology update system according to an embodiment of the present invention, such as figure 2 As shown, the system includes: a task manager 10 and a task executor 12, wherein,

[0037] The task manager 10 is configured to detect the first state of the first state machine, and generate a control message according to the above state, wherein the above first state is used to trigger the management of the topology structure of the stream processing; the task executor 12 is used to receive The aforementioned control message; and triggered by the aforementioned control message, triggering a second state of the second state machine, and performing an action corresponding to the aforementioned second state on the aforementioned topological structure according to the second state.

[0038] In the embodiment of the present...

Embodiment 2

[0073] According to an embodiment of the present invention, an embodiment of a topology update method for stream processing is provided. It should be noted that the steps shown in the flow chart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0074] Figure 8 A flowchart of the steps of a stream processing topology update method according to an embodiment of the present invention, as shown in Figure 8 The method includes the following steps:

[0075] Step S102, acquiring a control message, wherein the above control message is a message generated according to the first state of the first state machine, and the above first state is used to trigger the management of the topology structure of the stream processing;

[0076] In step S...

Embodiment 3

[0088] The embodiment of the present invention also provides a device for implementing the above topology update method for stream processing, Figure 9 is a schematic structural diagram of a stream processing topology update device according to an embodiment of the present invention, as shown in Figure 9 As shown, the control device for the above range hood includes: an acquisition module 100 and a trigger module 102, wherein,

[0089] The obtaining module 100 is configured to obtain a control message, wherein the above-mentioned control message is a message generated according to the first state of the first state machine, and the above-mentioned first state is used to trigger the management of the topology structure of the stream processing; the triggering module 102 is used for Triggered by the control message, a second state of the second state machine is triggered, and an action corresponding to the second state is performed on the topology according to the second state...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a stream processing topology update system and method. Wherein, the system includes: a task manager, configured to detect a first state of the first state machine, and generate a control message according to the above state, wherein, the above first state is used to trigger the management of the topology structure of stream processing; the task executor , for receiving the above-mentioned control message; and triggering the second state of the second state machine under the trigger of the above-mentioned control message, and performing an action corresponding to the above-mentioned second state on the above-mentioned topology structure according to the second state. The invention solves the technical problem that the existing large data flow processing framework is not suitable for updating the flow topology online.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a topology update system and method for stream processing. Background technique [0002] With the development of Internet technology, more and more industries require fast and efficient data processing, and the timeliness of data processing is getting higher and higher. The forms of information are continuously entering the system, and need to be processed at the fastest speed; another example is the timeliness of news, which also requires that major events be recommended to users as soon as possible, so as to increase user stickiness and bring considerable traffic. [0003] Figure 1a is a DAG topology diagram of an application in the prior art according to an embodiment of the present invention, such as Figure 1a As shown, the currently deployed and running DAGs include Task1, Task2, Source, and Sink. After the existing stream processing framework submits the applicatio...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F8/65G06F8/20G06F9/445G06F11/07
Inventor 赵懿
Owner BEIJING GRIDSUM TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com