Method, device and system for scaling operator nodes

By constructing the topology diagram of the operator node and transmitting the adjustment identification information, the expansion and capacity adjustment of the operator nodes in the stream computing system is realized, which solves the problem of improper resource allocation caused by task traffic changes, and improves resource utilization and system stability .

CN113626192BActive Publication Date: 2024-11-22ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110912385.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-10
Publication Date
2024-11-22
Estimated Expiration
2041-08-10

AI Technical Summary

Technical Problem

In a stream computing system, real-time changes in task traffic lead to improper resource allocation, resulting in waste of resources or high load pressure. The existing scaling capacity processing requires stopping the task and reallocating resources, affecting the task's throughput and stability.

Method used

By constructing the topology diagram of the operator node, the adjustment identification information is transmitted, and the operator node is expanded and adjusted without stopping the task, reducing the impact on the task.

Benefits of technology

It avoids the overall task process delay caused by restarting after task stops, and only adjusts the topology diagram of operator nodes, which reduces the impact on the task and improves resource utilization and system stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113626192B_ABST
    Figure CN113626192B_ABST
Patent Text Reader

Abstract

The embodiments of this specification provide a method, device and system for scaling operator nodes. In this method, in response to the concurrent operator nodes to be adjusted determined according to the real-time data stream, the control node constructs an operator node topology map according to the upstream and downstream operator nodes of the concurrent operator nodes to be adjusted; the control node sends the graph state update information for the operator node to each operator node to be adjusted in the operator node topology map, so that each operator node to be adjusted updates the connection; the control node sends the adjustment identification information to the source operator node in the operator node topology map; at each operator node, the adjustment identification information is obtained according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; and the control node sends the adjustment completion information to each operator node in response to receiving the adjustment identification information, so as to notify the completion of the operator node topology map.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of this specification relate to the field of big data computing technology, and specifically, to a method, device, and system for scaling operator nodes. Background Art

[0002] In current stream computing systems, when a task is submitted for execution, resources are allocated based on the default parameters or the parameters adjusted by user experience, and then the task calculation is performed according to the allocated resources until the task is completed or stopped.

[0003] For stream computing tasks, the task flow in different time periods will change. For example, the task is to count the traffic volume in the city every day. The traffic volume in the morning and evening peak hours will be significantly higher than other time periods, so the flow data in the morning and evening peak hours is higher than other time periods. In this case, since the task flow changes in real time, the initially allocated resources are always used for task calculation. When the task flow is small, only part of the resources are needed to execute, which leads to resource waste. When the task flow is large, even if all the allocated resources are used to execute the task, it will cause high load pressure and reduced throughput. Based on this, for stream computing tasks, resources need to be adjusted according to real-time traffic to match the current task flow.

[0004] At present, one resource adjustment method is that during the execution of stream computing tasks, when the task traffic changes, corresponding expansion and contraction processing is required. At this time, a checkpoint can be triggered to complete a snapshot of the task status. Then the task is stopped, and resources such as concurrent operators are reallocated according to the current task traffic. After the reallocation is completed, the task is restarted to continue running. Summary of the invention

[0005] In view of the above, the embodiments of this specification provide a method, device and system for scaling up and down the operator nodes. In the technical solution provided in the embodiments of this specification, by constructing an operator node topology map and transmitting adjustment identification information between each operator node in the operator topology map, there is no need to stop the task, thereby avoiding the problem of overall task process delay caused by the need to repeatedly consume a section of data from the previous checkpoint when restarting the task after stopping, and only each operator node involved in the operator node topology map can achieve scaling up and down adjustment, thereby reducing the impact on the task.

[0006] According to one aspect of an embodiment of the present specification, a method for scaling an operator node is provided, comprising: in response to a concurrent operator node to be adjusted determined according to a real-time data stream, a control node constructs an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted; the control node sends graph state update information for the operator node to each operator node to be adjusted in the operator node topology graph, so that each operator node to be adjusted updates its connection, wherein the graph state update information of each operator node includes the upstream and downstream operator nodes of the operator node. The control node transmits adjustment identification information to the source operator node in the operator node topology map; at each operator node, the adjustment identification information is obtained according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; and the control node responds to receiving the adjustment identification information from the tail operator node in the operator node topology map, and sends adjustment completion information to each operator node, so as to notify each operator node that the operator node topology map after the expansion and contraction adjustment is completed.

[0007] According to another aspect of the embodiments of this specification, a method for scaling an operator node is also provided, which is executed by a control node, and the method includes: in response to a concurrent operator node to be adjusted determined according to a real-time data stream, constructing an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted; sending graph state update information for the operator node to each operator node to be adjusted in the operator node topology graph, so that each operator node to be adjusted updates its connection, wherein the graph state update information of each operator node includes the graph state update information of the operator node. The adjustment information of the upstream and downstream operator nodes of the operator node and the connection adjustment information for the operator node; sending adjustment identification information to the source operator node in the operator node topology map, so that each operator node obtains the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; and in response to receiving the adjustment identification information from the tail operator node in the operator node topology map, sending adjustment completion information to each operator node, so as to notify each operator node that the operator node topology map after the expansion and contraction adjustment is completed.

[0008] According to another aspect of the embodiments of this specification, there is also provided a device for scaling up and down an operator node, the device being applied to a control node, the device comprising: at least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, the at least one processor executing the computer program to implement: in response to a concurrent operator node to be adjusted determined according to a real-time data stream, constructing an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted; sending graph state update information for the operator node to each operator node to be adjusted in the operator node topology graph, so that the operator node to be adjusted Update the connection of each operator node, wherein the graph state update information of each operator node includes the upstream and downstream operator node adjustment information of the operator node and the connection adjustment information for the operator node; send adjustment identification information to the source operator node in the operator node topology graph, so that each operator node obtains the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; and in response to receiving the adjustment identification information from the tail operator node in the operator node topology graph, send adjustment completion information to each operator node, so as to notify each operator node that the operator node topology graph after the expansion and contraction adjustment is completed.

[0009] According to another aspect of the embodiments of this specification, a system for scaling up and down an operator node is also provided, the system comprising a control node and a plurality of operator nodes included in an operator node topology graph constructed by the control node, the control node being used to respond to a concurrent operator node to be adjusted determined according to a real-time data stream, construct an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted; and send graph state update information for the operator node to each operator node to be adjusted in the operator node topology graph, so that each operator node to be adjusted Update the connection, wherein the graph state update information of each operator node includes the upstream and downstream operator node adjustment information of the operator node and the connection adjustment information for the operator node; each operator node is used to obtain the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; the control node is also used to respond to receiving the adjustment identification information from the tail operator node in the operator node topology graph, and send adjustment completion information to each operator node to notify each operator node that the operator node topology graph after the expansion and contraction adjustment is completed.

[0010] According to another aspect of an embodiment of this specification, a computer-readable storage medium is also provided, which stores a computer program, and when the computer program is executed by a processor, the method for scaling an operator node as described above is implemented.

[0011] According to another aspect of the embodiments of this specification, a computer program product is also provided, including a computer program, which, when executed by a processor, implements any of the above methods for scaling operator nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] A further understanding of the nature and advantages of the embodiments of the present specification can be achieved by referring to the following drawings. In the drawings, similar components or features may have the same reference numerals.

[0013] Figure 1 A block diagram of an example of a system for scaling an operator according to an embodiment of this specification is shown.

[0014] Figure 2 A flowchart of an example of a method for scaling an operator according to an embodiment of this specification is shown.

[0015] Figure 3 A flowchart of an example of constructing an operator node topology graph according to an embodiment of this specification is shown.

[0016] Figure 4 A schematic diagram showing an example of a pipeline subgraph including concurrent operator nodes to be adjusted according to an embodiment of the present specification.

[0017] Figure 5A A schematic diagram showing an example of a control node constructing an operator node topology graph according to an embodiment of this specification.

[0018] Figure 5B A schematic diagram showing an example of a transmission path of adjustment identification information in an operator node topology graph according to an embodiment of this specification.

[0019] Figure 5C A schematic diagram shows an example of a tail operator node in an operator node topology diagram feeding back adjustment identification information to a control node according to an embodiment of this specification.

[0020] Figure 5D A schematic diagram shows an example of a control node sending adjustment completion information to each operator node in an operator node topology graph according to an embodiment of this specification.

[0021] Figure 5EA schematic diagram showing an example of an operator node topology graph formed after a control node clears operator nodes to be deleted according to an embodiment of this specification.

[0022] Figure 6 A schematic diagram showing an example of thermal migration according to an embodiment of the present specification.

[0023] Figure 7 A flowchart of an example of reallocating state key groups when a stateful operator node is expanded according to an embodiment of this specification is shown.

[0024] Figure 8 A schematic diagram shows an example of reallocating state key groups when a stateful operator node is expanded according to an embodiment of this specification.

[0025] Fig. 9 A flowchart of an example of a method for scaling an operator node according to an embodiment of this specification is shown.

[0026] Fig.10 A block diagram of an example of an apparatus for scaling an operator node according to an embodiment of this specification is shown.

[0027] Fig.11 A block diagram of an electronic device for implementing a method for adjusting the capacity of an operator node according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0028] The subject matter described herein will be discussed below with reference to example implementations. It should be understood that the discussion of these implementations is only to enable those skilled in the art to better understand and implement the subject matter described herein, and is not a limitation of the scope of protection, applicability or examples set forth in the claims. The functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the embodiments of this specification. Various processes or components may be omitted, substituted or added to each example as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0029] As used herein, the term "including" and its variations represent open terms, meaning "including but not limited to". The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other definitions may be included below, whether explicit or implicit. Unless the context clearly indicates otherwise, the definition of a term is consistent throughout the specification.

[0030] In current stream computing systems, when a task is submitted for execution, resources are allocated based on the default parameters or the parameters adjusted by user experience, and then the task calculation is performed according to the allocated resources until the task is completed or stopped.

[0031] For stream computing tasks, the task flow in different time periods will change. For example, the task is to count the traffic volume in the city every day. The traffic volume in the morning and evening peak hours will be significantly higher than other time periods, so the flow data in the morning and evening peak hours is higher than other time periods. In this case, since the task flow changes in real time, the initially allocated resources are always used for task calculation. When the task flow is small, only part of the resources are needed to execute, which leads to resource waste. When the task flow is large, even if all the allocated resources are used to execute the task, it will cause high load pressure and reduced throughput. Based on this, for stream computing tasks, resources need to be adjusted according to real-time traffic to match the current task flow.

[0032] At present, one resource adjustment method is that during the execution of stream computing tasks, when the task traffic changes, corresponding expansion and contraction processing is required. At this time, a checkpoint can be triggered to complete a snapshot of the task status. Then the task is stopped, and resources such as concurrent operators are reallocated according to the current task traffic. After the reallocation is completed, the task is restarted to continue running. However, in this resource adjustment method, the entire task needs to be stopped before resource adjustment can be performed, and the overall stop of the task has a large impact, which will affect all operator nodes of the task. In addition, it will also affect the throughput and stability of the task.

[0033] In view of the above, the embodiments of this specification provide a method, device and system for scaling an operator. In the method, the control node responds to the concurrent operator nodes to be adjusted determined according to the real-time data flow, and constructs an operator node topology graph for the concurrent operator nodes to be adjusted according to the upstream and downstream operator nodes of the concurrent operator nodes to be adjusted; the control node sends graph state update information for each operator node to be adjusted in the operator node topology graph, so that each operator node to be adjusted updates the connection, wherein the graph state update information of each operator node includes the upstream and downstream operator node adjustment information of the operator node and the connection adjustment information for the operator node; the control node sends adjustment identification information to the source operator node in the operator node topology graph; at each operator node, the adjustment identification information is obtained according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; and the control node responds to receiving the adjustment identification information from the tail operator node in the operator node topology graph, and sends adjustment completion information to each operator node, so as to notify each operator node that the operator node topology graph adjusted after expansion and contraction is completed. By constructing an operator node topology map and transmitting adjustment identification information between each operator node in the operator node topology map, there is no need to stop the task, thus avoiding the problem of overall task process delay caused by the need to repeatedly consume a section of data from the previous checkpoint when the task is stopped and restarted. In addition, only the operator nodes involved in the operator node topology map can be expanded or reduced, thereby reducing the impact on the task.

[0034] The following describes in detail the method, device and system for scaling operators provided in the embodiments of this specification in conjunction with the accompanying drawings.

[0035] Figure 1 A block diagram of an example of a system 100 for scaling an operator according to an embodiment of this specification is shown.

[0036] like Figure 1 As shown, the system 100 may include at least one control node 110 and multiple operator nodes 120 , and each control node 110 is communicatively connected to each operator node 120 .

[0037] The control node 110 can be used to coordinate and manage the operation of each operator node 120, for example, to allocate resources such as operator nodes to tasks, manage checkpoints, and control the execution path of tasks between various operator nodes. When the system 100 includes one control node 110, the control node 110 performs the above operations in the system 110; when the system 100 includes multiple control nodes 110, one of the multiple control nodes 110 serves as the main control node to perform the above operations in the system 110, and the other control nodes 110 serve as backup control nodes. When the main control node fails, the backup control node is upgraded to the main control node to continue to support the normal operation of the system 100, avoiding the system 100 from shutting down due to the failure of the main control node, thereby improving the high availability of the system 100.

[0038] The operator node 120 may be a working node for executing tasks. Each operator node 120 may execute tasks in parallel, and each operator node 120 may execute multiple tasks in parallel. In one example, each operator node 120 may have multiple task slots, each task slot represents a set of resources in the operator node 120. For example, if the operator node 120 has 4 task slots, each task slot is allocated 25% of the memory resources of the operator node. Each task slot may run one or more threads, and each task slot may execute multiple tasks in parallel.

[0039] In one example, the system 100 can be applied in the framework of FLINK. In the framework of FLINK, the task client can accept the user's program code, then create a data stream, and then submit the data stream and the task to the control node 110, so that the control node 110 can obtain the task and the data stream of the task. After that, the control node 110 assigns the task to each operator node 120, which is specifically executed by each operator node 120.

[0040] In the embodiment of the present specification, after acquiring the task, the control node 110 can determine the operator node for executing the task from the multiple operator nodes 120 according to the task. The control node 110 then deploys the execution order of each operator node 120 for executing the task to form a pipeline composed of each operator node 120, which is used to completely execute the task.

[0041] The control node 110 in the system 100 may perform relevant operations of the embodiments of this specification, and each operator node 120 may also perform relevant operations of the embodiments of this specification.

[0042] Figure 2 A flowchart of an example 200 of a method for scaling an operator according to an embodiment of this specification is shown.

[0043] like Figure 2 As shown, at 210, in response to the concurrent operator node to be adjusted determined according to the real-time data stream, the control node constructs an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted.

[0044] In an embodiment of the present specification, during the execution of a task, the data flow of the task is submitted in real time to the pipeline formed by each operator node 120 for executing the task for processing. The flow of the data flow can change in real time. When the flow of the data flow changes, the data processing volume of the concurrent operator nodes in the pipeline will be affected due to the unchanged structure of the pipeline. When the flow of the data flow decreases, the data processing volume of the concurrent operator nodes decreases. When the data processing capacity of the concurrent operator nodes far exceeds the flow of the data flow, it will lead to a waste of resources of the operator nodes. When the flow of the data flow increases, the data processing volume of the concurrent operator nodes increases, which will increase the load pressure of the concurrent operator nodes and reduce the throughput. Based on this, it can be judged whether the concurrent operator nodes in the pipeline need to be adjusted according to the real-time changes of the data flow. The adjustment includes expansion adjustment and contraction adjustment. The concurrent operator nodes to be adjusted can be determined as concurrent operator nodes to be adjusted. In a pipeline for executing a complete task, there can be one or more concurrent operator nodes to be adjusted.

[0045] In the embodiments of this specification, a concurrent operator node may be an operator node running in a stage of a pipeline, and a concurrent operator node may include one or more operator nodes. Multiple operator nodes in a concurrent operator node may be executed in parallel, and the multiple operator nodes executed in parallel are in the same stage of the pipeline.

[0046] In one example, it can be determined whether to determine the concurrent operator node as a concurrent operator node to be adjusted based on the real-time data flow and the data processing capability of the concurrent operator node. Specifically, when the flow of the data flow is reduced, when the real-time data flow can be processed by some of the concurrent operator nodes, it means that the concurrent operator node can be scaled down to reduce the number of operator nodes that match the real-time data flow to process the data flow. At this time, the concurrent operator node can be determined as a concurrent operator node to be adjusted.

[0047] When the flow of data flow increases, when the pressure on the concurrent operator node increases and the throughput decreases, it means that the concurrent operator node can be expanded and adjusted, and a certain number of operator nodes can be added to the concurrent operator node to cope with the increased data flow. At this time, the concurrent operator node can be determined as a concurrent operator node to be adjusted.

[0048] After determining the concurrent operator nodes to be adjusted, when the adjustment of the concurrent operator nodes to be adjusted is an expansion adjustment, the control node can add an operator node, and the newly added operator node is used as an operator node in the concurrent operator nodes to be adjusted, and is executed in parallel with other operator nodes in the concurrent operator nodes to be adjusted. Then, the control node initiates a new process in the newly added operator node for the newly added operator node to perform subsequent related operations. When the adjustment of the concurrent operator nodes to be adjusted is a reduction adjustment, if there are operator nodes to be deleted in the concurrent operator nodes to be adjusted, the control node can mark the operator nodes to be deleted.

[0049] After determining the concurrent operator node to be adjusted, the control node can determine the upstream and downstream operator nodes of the concurrent operator node to be adjusted according to the pipeline used to execute the complete task. The upstream and downstream operator nodes may include upstream operator nodes and / or downstream operator nodes. For example, when the concurrent operator node to be adjusted is the source concurrent operator node in the pipeline, the downstream operator node of the concurrent operator node to be adjusted can be determined; when the concurrent operator node to be adjusted is the tail concurrent operator node in the pipeline, the upstream operator node of the concurrent operator node to be adjusted can be determined; when the concurrent operator node to be adjusted is a concurrent operator node in the middle stage of the pipeline, the upstream operator node and downstream operator node of the concurrent operator node to be adjusted can be determined.

[0050] The concurrent operator nodes to be adjusted are the operator nodes affected by the expansion and contraction adjustment. Accordingly, the upstream and downstream operator nodes of the concurrent operator nodes to be adjusted are also the operator nodes affected by the expansion and contraction adjustment. For example, when the concurrent operator nodes to be adjusted are scaled down, some operator nodes will be deleted, and the upstream and downstream operator nodes connected to these operator nodes also need to delete the connections with these operator nodes. For another example, when the concurrent operator nodes to be adjusted are expanded, it is necessary to add operator nodes to the concurrent operator nodes, and accordingly, it is also necessary to increase the connections between the operator nodes and the upstream and downstream operator nodes.

[0051] The affected concurrent operator nodes to be adjusted and the upstream and downstream operator nodes of the concurrent operator nodes to be adjusted can constitute an operator node topology map for the concurrent operator nodes to be adjusted, that is, each operator node in the operator node topology map is an affected operator node that needs to be adjusted accordingly in this scaling adjustment. The operator nodes included in the operator node topology map can be some operator nodes in the entire pipeline used to fully execute the task. In this way, the scaling adjustment of the concurrent operator nodes to be adjusted only needs to adjust some operator nodes in the operator node topology map without affecting other operator nodes, thereby reducing the impact on the running task.

[0052] Figure 3 A flowchart of an example 300 of constructing an operator node topology graph according to an embodiment of this specification is shown.

[0053] like Figure 3 As shown, at 211, in response to the concurrent operator node to be adjusted determined according to the real-time data stream, the control node may determine the upstream operator node and / or downstream operator node of the concurrent operator node to be adjusted.

[0054] When the concurrent operator node to be adjusted has no upstream operator node, only the downstream operator node of the concurrent operator node to be adjusted can be determined. For example, if the concurrent operator node to be adjusted is a starting operator node, only the downstream operator node of the concurrent operator node to be adjusted can be determined. When the concurrent operator node to be adjusted has no downstream operator node, only the upstream operator node of the concurrent operator node to be adjusted can be determined. For example, if the concurrent operator node to be adjusted is an ending operator node, only the upstream operator node of the concurrent operator node to be adjusted can be determined. When the concurrent operator node to be adjusted has upstream operator nodes and downstream operator nodes, the upstream operator nodes and downstream operator nodes of the concurrent operator node to be adjusted can be determined at the same time.

[0055] In the embodiments of this specification, the starting operator node may be the operator node that first executes the task in the pipeline. Therefore, in the pipeline, the starting operator node has no upstream operator node, and the data flow of the task executed by the starting operator node can be obtained from the control node. The ending operator node may be the operator node that last executes the task in the pipeline. Therefore, in the pipeline, the ending operator node has no downstream operator node, and the ending operator node can directly output the result of the task.

[0056] After determining the upstream and downstream operator nodes (including: upstream operator nodes and / or downstream operator nodes) of the concurrent operator node to be adjusted, the concurrent operator node to be adjusted and the upstream and downstream operator nodes may be operator nodes used to construct an operator node topology graph.

[0057] At 212 , the control node may determine a first start operator node and a first end operator node according to the connection relationship between the concurrent operator nodes to be adjusted, the upstream operator nodes, and / or the downstream operator nodes.

[0058] The connection relationship between each operator node in the concurrent operator node to be adjusted, the upstream operator node and / or the downstream operator node is determined by the pipeline composed of all operator nodes used to execute the task. Therefore, the connection relationship between each operator node in the concurrent operator node to be adjusted, the upstream operator node and / or the downstream operator node can be determined according to the pipeline used to execute the complete task. The various operator nodes can be connected into a pipeline subgraph according to the connection relationship. The pipeline subgraph can be a part of the pipeline, so that the execution process of the pipeline subgraph is part of the entire task execution process.

[0059] After the pipeline subgraph is obtained, the starting operator node in the pipeline subgraph may be determined as the first starting operator node, and the ending operator node in the pipeline subgraph may be determined as the first ending operator node.

[0060] Figure 4 FIG. 1 is a schematic diagram showing an example of a pipeline subgraph including concurrent operator nodes to be adjusted according to an embodiment of the present specification. Figure 4 As shown, the concurrent operator nodes to be adjusted include concurrent operator nodes B and C. The adjustment of concurrent operator node B is an expansion adjustment, that is, it is expanded from the original one operator node B1 to a concurrent operator node consisting of two operator nodes B1 and B2. The adjustment of concurrent operator node C is a contraction adjustment, that is, it is contracted from two concurrent operator nodes C1 and C2 to one operator node C1. The upstream operator nodes of the concurrent operator node B include operator nodes A1 and A2, and the downstream operator nodes include operator nodes C1 and C2. The upstream operator nodes of the concurrent operator node C include operator nodes B1 and B2, and the downstream operator nodes include operator node D1. Therefore, operator nodes A1, A2, B1, B2, C1, C2 and D1 are operator nodes that constitute the operator node topology graph, so that the pipeline subgraph formed by the operator nodes A1, A2, B1, B2, C1, C2 and D1 according to the connection relationship is as shown in the figure. Figure 4 As shown, Figure 4 The solid and dotted lines in the figure represent the connections between operator nodes. The solid lines represent the connections that remain unchanged before and after the adjustment, and the dotted lines represent the connections that need to be adjusted during the expansion and contraction adjustment process. The adjustments here can include adding and deleting connections. Figure 4 In the pipeline subgraph shown, operator nodes A1 and A2 are starting operator nodes, and operator nodes A1 and A2 can be determined as the first starting operator nodes; operator node D1 is the ending operator node, and operator node D1 can be determined as the first ending operator node.

[0061] At 213 , the control node may determine a source operator node.

[0062] The source operator node may be the starting operator node among the determined concurrent operator nodes to be adjusted, the upstream operator nodes, and the downstream operator nodes, that is, the source operator node may be the operator node that first receives the data stream for processing according to the execution order of the pipeline among the determined concurrent operator nodes to be adjusted, the upstream operator nodes, and the downstream operator nodes.

[0063] In one way of determining the source operator node, the control node may determine the source operator node according to the first starting operator node. Specifically, when the concurrent operator node to be adjusted does not have an upstream operator node, the control node may directly determine the first starting operator node as the source operator node.

[0064] In another manner of determining the source operator node, the control node may determine the source operator node according to the first starting operator node, the concurrent operator nodes to be adjusted, and the upstream operator nodes.

[0065] Specifically, the control node may determine other operator nodes in the upstream operator nodes except the concurrent operator node to be adjusted as the second starting operator node, that is, it can be expressed by the following formula: V 2S =V UPsTREAM -V CHANGED , where V 2S Represented as the second starting operator node, V UPSTREAM represents the upstream operator node, V CHANGED Indicates the concurrent operator nodes to be adjusted, V UPSTREAM -V CHANGED Indicates other operator nodes in the upstream operator nodes except the concurrent operator node to be adjusted.

[0066] Then, the control node may combine the first starting operator node and the second starting operator node to determine the source operator node. That is, it can be expressed as follows: V SOURCE =V 1S ∪V 2S , where V 1S Represented as the first starting operator node, V SOURCE represents the source operator node, V 1S ∪V 2S Indicates that the first start operator node and the second start operator node are merged.

[0067] by Figure 4 For example, in Figure 4 In the various operator nodes used to construct the operator node topology graph shown in FIG. , the first starting operator node includes operator nodes A1 and A2, then V 1S ={A1, A2}; the concurrent operator nodes to be adjusted include concurrent operator nodes B and C, then V CHANGED={B1, B2, C1, C2}; the upstream operator nodes of concurrent operator node B include operator nodes A1 and A2, and the upstream operator nodes of concurrent operator node C include operator nodes B1 and B2, then V UPSTREAM ={A1, A2, B1, B2}. Then, the other operator nodes in the upstream operator nodes except the concurrent operator node to be adjusted include {A1, A2}, that is, it can be determined that the second starting operator node includes {A1, A2}, V 2S =V UPSTREAM -V CHANGED ={A1, A2}, the source operator nodes obtained by merging the first starting operator node and the second starting operator node include operator nodes A1 and A2, that is, V SOURCE =V 1S ∪V 2S ={A1, A2}.

[0068] At 214, the control node may determine the tail operator nodes.

[0069] The tail operator node may be the end operator node among the determined concurrent operator nodes to be adjusted, the upstream operator nodes, and the downstream operator nodes, that is, the tail operator node may be the operator node that finally receives the data flow for processing according to the execution order of the pipeline among the determined concurrent operator nodes to be adjusted, the upstream operator nodes, and the downstream operator nodes.

[0070] In one way of determining the tail operator node, the control node may determine the tail operator node according to the first end operator node. Specifically, when the concurrent operator node to be adjusted does not have a downstream operator node, the control node may directly determine the first end operator node as the tail operator node.

[0071] In another manner of determining the tail operator node, the control node determines the tail operator node according to the first ending operator node, the concurrent operator nodes to be adjusted, and the downstream operator nodes.

[0072] Specifically, the control node may determine other operator nodes in the downstream operator nodes except the concurrent operator node to be adjusted as the second end operator node, that is, it can be expressed by the following formula: 2E =V DOWNSTREAM -V CHANGED , where V 2E Represented as the second end operator node, V DOWNSTREAM represents the downstream operator node, V CHANGED Indicates the concurrent operator nodes to be adjusted, V DOWNSTREAM -V CHANGED Indicates other operator nodes in the downstream operator nodes except the concurrent operator node to be adjusted.

[0073] Then, the control node may combine the first end operator node and the second end operator node to determine the end operator node. That is, it can be expressed as follows: V SINK =V 1E ∪V 2E , where V 1E Represented as the first end operator node, V SINK represents the tail operator node, V 1E ∪V 2R Indicates that the first end operator node and the second end operator node are merged.

[0074] by Figure 4 For example, in Figure 4 In the various operator nodes used to construct the operator node topology graph shown in FIG. , the first end operator node includes the operator node D1, then V 1E = {D1}; the concurrent operator nodes to be adjusted include concurrent operator nodes B and C, then V CHANGED ={B1, B2, C1, C2}; the downstream operator nodes of the concurrent operator node B include operator nodes C1 and C2, and the downstream operator nodes of the concurrent operator node C include operator node D1, then V DOWNSTREAM ={C1, C2, D1}. Then, the downstream operator nodes, except for the concurrent operator node to be adjusted, include {D1}, that is, it can be determined that the second end operator node includes {D1}, V 2E =V DOWNSTREAM -V CHANGED = {D1}, the tail operator node obtained by merging the first end operator node and the second end operator node includes the operator node D1, that is, V SINK =V 1R ∪V 2E ={D1}.

[0075] It should be noted that the execution order of the operation 213 and the operation 214 is not limited. The operation 213 may be executed first and then the operation 214, or the operation 214 may be executed first and then the operation 213, or the operations 213 and 214 may be executed simultaneously.

[0076] After determining the source operator node and the tail operator node, at 215 , the control node may construct an operator node topology graph.

[0077] In one construction method, when the concurrent operator node to be adjusted has both upstream operator nodes and downstream operator nodes, the control node can construct an operator node topology graph based on the source operator node, the tail operator node, the concurrent operator node to be adjusted, the downstream operator node, and the upstream operator node. Specifically, the control node can arrange the source operator node, the tail operator node, the concurrent operator node to be adjusted, the downstream operator node, and the upstream operator node in the execution order of the pipeline, and arrange connections between each operator node and its upstream and downstream operator nodes.

[0078] When the concurrent operator nodes to be adjusted include a newly added operator node, the control node may arrange a connection between the newly added operator node and the operator nodes existing in the adjusted operator node topology graph in the upstream and downstream operator nodes of the newly added operator node, wherein the operator nodes existing in the adjusted operator node topology graph may include the operator nodes existing before and after the adjustment and the newly added operator nodes, so that Figure 4 For example, if the operator nodes A1, A2, and C1 in the upstream and downstream operator nodes of the newly added operator node B2 are operator nodes that always exist before and after the adjustment, connections can be arranged between the newly added operator node B2 and the operator nodes A1, A2, and C1 respectively. When there is an operator node to be deleted in the upstream and downstream operator nodes of the newly added operator node, the operator node to be deleted will not exist in the adjusted operator node topology map, so the control node may not arrange a connection between the newly added operator node and the operator node to be deleted.

[0079] When the concurrent operator nodes to be adjusted include an operator node to be deleted, the control node may arrange a connection between the operator node to be deleted and the operator nodes existing in the operator node topology graph before adjustment in the upstream and downstream operator nodes of the operator node to be deleted. However, when there is a newly added operator node in the upstream and downstream operator nodes of the operator node to be deleted, the control node may not arrange a connection between the deleted operator node and the newly added operator node. Figure 4 For example, if operator nodes B1 and D1 in the upstream and downstream operator nodes of the to-be-deleted operator node C2 are operator nodes existing in the operator node topology graph before adjustment, the control node may arrange connections between the to-be-deleted operator node C2 and operator nodes B1 and D1. If operator node B2 is a newly added operator node in the upstream and downstream operator nodes of the to-be-deleted operator node C2, the control node may not arrange connections between operator node B2 and the to-be-deleted operator node C2.

[0080] In another construction mode, when there is no upstream operator node for the concurrent operator node to be adjusted, the control node can construct an operator node topology map based on the source operator node, the tail operator node, the concurrent operator node to be adjusted, and the downstream operator node. In another construction mode, when there is no downstream operator node for the concurrent operator node to be adjusted, the control node can construct an operator node topology map based on the source operator node, the tail operator node, the concurrent operator node to be adjusted, and the upstream operator node.

[0081] Figure 5A FIG. 1 is a schematic diagram showing an example of a control node constructing an operator node topology diagram according to an embodiment of this specification. Figure 5A As shown, M represents the control node, Figure 5A The operator nodes used to construct the operator node topology graph are the above Figure 4 The concurrent operator nodes to be adjusted and the upstream and downstream operator nodes in the example shown, the operator node topology diagram constructed by the control node for the concurrent operator nodes to be adjusted is as follows Figure 5A shown. Figure 5A The connecting line in represents the connection between two operator nodes. The two connected operator nodes can communicate with each other. Figure 5A In the figure, the solid connecting line may represent the connection existing in the operator node topology graph after the expansion and contraction adjustment is completed, and the dotted connecting line may represent the connection to be deleted, which exists in the topology graph before the adjustment but does not exist in the operator node topology graph after the adjustment.

[0082] By constructing an operator node topology map, the control node can determine the concurrent operator nodes that need to be adjusted and use them to construct the operator node topology map. In subsequent expansion and contraction adjustments, only the operator node topology map needs to be operated, which reduces the impact of expansion and contraction adjustments and thus reduces the impact on task execution.

[0083] Back to Figure 2 After the control node constructs the operator node topology graph for the concurrent operator nodes to be adjusted, the control node can determine the position of each operator node in the operator node topology graph, the upstream and downstream operator nodes connected to each operator node in the operator node topology graph, and other information.

[0084] At 220 , the control node may send graph state update information for each adjusted operator node in the operator node topology graph, so that each adjusted operator node updates its connection.

[0085] In the embodiment of the present specification, the operator nodes that existed before the expansion and contraction adjustment perform tasks according to the formed pipeline, so each operator node that existed before the adjustment can determine its own upstream and downstream operator nodes, and establish a communication connection with the upstream and downstream operator nodes. In the operator node topology map constructed by the control node, there are newly added operator nodes and / or operator nodes to be deleted according to the needs of the expansion and contraction adjustment, so that the operator node topology map is updated compared to the topology map composed of the operator nodes before the adjustment.

[0086] The control node can send graph status update information for each operator node that has been adjusted in the operator node topology graph. The adjusted operator nodes may include newly added operator nodes, operator nodes to be deleted, operator nodes that exist before and after the adjustment and whose communication connections have been increased or decreased, etc.

[0087] The graph state update information of each adjusted operator node is information about changes in the operator node before and after the scaling adjustment. The graph state update information may include adjustment information of upstream and downstream operator nodes of the operator node and connection adjustment information for the operator node.

[0088] The upstream and downstream operator node adjustment information may include newly added operator nodes and / or operator nodes to be deleted in the upstream and downstream operator nodes. When performing expansion adjustment, there may be newly added operator nodes, and when performing reduction adjustment, there may be operator nodes to be deleted. In the graph state update information of the operator node, the upstream and downstream operator node update information of the operator node may represent the newly added and / or to be deleted operator nodes in the upstream and downstream operator nodes.

[0089] The connection adjustment information of an operator node may indicate the connection information of the operator node with other operator nodes that needs to be adjusted. In one example, for capacity expansion adjustment, the connection adjustment information may include newly added connections between the newly added operator node and each other operator node in the upstream and downstream operator nodes except the operator node to be deleted. The other operator nodes are operator nodes that always exist before and after the adjustment. Figure 5A Taking the newly added operator node B2 as an example, among the upstream and downstream operator nodes of the newly added operator node B2, the operator node C2 is the operator node to be deleted, then the other operator nodes except the operator node C2 include operator nodes A1, A2 and C1, and the operator nodes A1, A2 and C1 are the operator nodes that always exist before and after the adjustment, then the connection adjustment information may include the newly added connections between the newly added operator node B2 and the operator nodes A1, A2 and C1 respectively, and the newly added operator node B2 will not establish a connection with the operator node C2 to be deleted.

[0090] For the scaling adjustment, the connection adjustment information may include the connections to be deleted between the operator node to be deleted and each other operator node in the upstream and downstream operator nodes except the newly added operator node, and the other operator nodes include the operator nodes that existed before the adjustment. Figure 5A Take the operator node C2 to be deleted as an example, among the upstream and downstream operator nodes of the operator node C2 to be deleted, the operator node B2 is a newly added operator node, and the operator nodes B1 and D1 are the existing operator nodes before the adjustment. Therefore, the connection adjustment information may include the connections between the operator node C2 to be deleted and the operator nodes B1 and D1 to be deleted.

[0091] After each operator node in the constructed operator node topology graph receives the graph state information sent by the control node, each operator node can determine the upstream and downstream operator nodes connected to it in the constructed operator node topology graph, and whether there are connections that need to be deleted. Each operator node updates the connection with its upstream and downstream operator nodes according to the received graph state update information, and the way to update the connection includes adding a new connection and marking the connection to be deleted. After each operator node updates the connection, a sequential sub-pipeline can be formed, which can be part of the above-mentioned pipeline for executing the entire task.

[0092] Next, at 230, the control node may send adjustment identification information to the source operator node in the operator node topology graph. The adjustment identification information may be transmitted sequentially according to the pipeline formed by the operator node topology graph until it reaches the tail operator node in the operator node topology graph.

[0093] At 240 , at each operator node, adjustment identification information is obtained according to the graph state information received by the operator node, so as to adjust the state of the operator node based on the adjustment identification information.

[0094] In an embodiment of the present specification, the adjustment identification information may be information carrying a specified identification, and the specified identification is used to distinguish the states before and after the adjustment in each operator node. When an operator node receives the adjustment identification information, it indicates that the operator node can be adjusted, and for the data stream before the adjustment identification information, the operator node can process it in the manner before the adjustment, and for the data stream after the adjustment indication information, the operator node can process it in the manner after the adjustment. When an operator node sends out the adjustment identification information, it may indicate that the operator node has completed the adjustment; when an operator node receives the adjustment identification information from an upstream operator node, it indicates that the operator node can determine that the upstream operator node has completed the adjustment, so that the operator node can receive from the upstream operator node the data stream processed by the upstream operator node in the manner after the adjustment.

[0095] Figure 5BFIG. 1 is a schematic diagram showing an example of adjusting the transmission path of identification information in an operator node topology diagram according to an embodiment of this specification. Figure 5B As shown, the arrows connecting the various operator nodes are the transmission directions of the adjustment identification information. The control node sends the adjustment identification information to the source operator nodes A1 and A2 respectively, and the source operator nodes A1 and A2 send the adjustment identification information to the operator nodes B1 and B2 respectively after the adjustment is completed. For example, when the operator node B1 receives the adjustment identification information sent by the source operator node A1, the operator node B1 can determine that the source operator node A1 has completed the adjustment. After the newly added operator node B2 completes the adjustment, it sends the adjustment identification information to the operator node C1. After the operator node B1 completes the adjustment, it sends the adjustment identification information to the operator node C1 and the operator node C2 to be deleted. After the operator node C1 and the operator node C2 to be deleted complete their respective adjustments, they send the adjustment identification information to the tail operator node D1.

[0096] In one example, for an operator node topology map, when there is a capacity reduction adjustment, there are operator nodes to be deleted among the concurrent operator nodes to be adjusted. During the transmission of the adjustment identification information in the operator node topology map, at the operator node to be deleted, the adjustment identification information can be received from the upstream operator node connected to the operator node. At this time, the operator node to be deleted can determine that the upstream operator node has completed the adjustment, so that the operator node to be deleted can disconnect from the upstream operator node. After completing the adjustment, the operator node to be deleted can continue to send the adjustment identification information to the connected downstream operator node.

[0097] At the downstream operator node, after receiving the adjustment identification information sent by the operator node to be deleted, the downstream operator node can determine that the adjustment of the operator node to be deleted has been completed, so that the downstream operator node can disconnect from the operator node to be deleted.

[0098] At each operator node, it is necessary to collect the adjustment identification information sent by all upstream operator nodes connected to the operator node before making adjustments locally. After the adjustment itself is completed, the adjustment identification information can be sent to the downstream operator nodes.

[0099] In an embodiment of the present specification, the adjustment identification information may be transmitted to the tail operator node in the operator node topology diagram. The tail operator node may receive the adjustment identification information from each upstream operator node according to the upstream operator node to which it is connected. When the tail operator node collects the adjustment identification information sent by all the upstream operator nodes to which it is connected, the collected adjustment identification information may be fed back to the control node. Figure 5C A schematic diagram shows an example of a tail operator node in an operator node topology diagram feeding back adjustment identification information to a control node according to an embodiment of this specification.

[0100] At 250 , in response to receiving adjustment identification information from the tail operator node in the operator node topology graph, the control node sends adjustment completion information to each operator node to notify each operator node that the operator node topology graph adjusted by expansion and contraction is completed.

[0101] When the control node receives adjustment identification information from the tail operator node in the operator node topology graph, it indicates that the adjustment identification information has been transmitted in sequence in the operator node topology graph according to the order of the sub-pipeline, and each operator node in the operator node topology graph has also completed the adjustment. If all operator nodes in the operator node topology graph have completed the adjustment, it can be determined that the adjustment for the entire operator node topology graph is completed. Thus, when the control node receives adjustment identification information from the tail operator node in the operator node topology graph, it can be determined that the adjustment for the entire operator node topology graph is completed.

[0102] When the control node can determine that the adjustment of the entire operator node topology graph is completed, the control node can send adjustment completion information to each operator node, where the adjustment completion information is used to indicate that the expansion and contraction adjustment of the operator node topology graph has been completed. Figure 5D A schematic diagram shows an example of a control node sending adjustment completion information to each operator node in an operator node topology graph according to an embodiment of this specification.

[0103] When each operator node receives the adjustment completion information, it can determine that the adjustment of the entire operator node topology is completed, so that data flow processing can be performed according to the pipeline represented by the adjusted operator node topology. Each operator node in the operator node topology is combined with other operator nodes in the pipeline according to the arrangement represented by the adjusted operator node topology to form a pipeline that has been adjusted for expansion and contraction.

[0104] In one example, when the expansion and contraction adjustment includes a shrinking adjustment, after the operator node topology map is completed, the connections between the operator node to be deleted that has undergone the shrinking adjustment and other operator nodes have been disconnected. At this time, the control node can clear the operator node to be deleted and only retain the operator nodes in the adjusted operator node topology map. Figure 5E A schematic diagram showing an example of an operator node topology graph formed after a control node clears operator nodes to be deleted according to an embodiment of this specification.

[0105] Through the above-mentioned scaling adjustment method provided in the embodiment of this specification, the scaling adjustment can be completed without stopping the task, reducing the impact on the task execution. In addition, compared with the pipeline that executes the entire task, the operator nodes in the operator node topology diagram can only be part of the operator nodes in a part of the pipeline. In this case, only the operator nodes in the operator node topology diagram need to be operated to achieve scaling adjustment, reducing the impact on the pipeline.

[0106] In an example of an embodiment of this specification, the scaling method provided in the embodiment of this specification can be applied to the hot migration operation of an operator node. Hot migration is to replace an existing operator node with a newly added operator node. Therefore, it is equivalent to adding a new operator node in a concurrent operator node through a scaling operation, and then deleting the corresponding operator node through a scaling operation.

[0107] Figure 6 FIG. 2 is a schematic diagram showing an example of thermal migration according to an embodiment of the present specification. Figure 6 As shown, the state shown in the left figure is the topological graph formed by each operator node before the hot migration operation. In this example, the operator node C2 needs to be hot migrated, so the control node can perform a capacity expansion operation on the concurrent operator node C, that is, add an operator node C3, and the newly added operator node C3 establishes a connection with the upstream operator node B1 and the downstream operator node D1. The control node also performs a capacity reduction operation on the operator node C2, and the control node sends a graph status update information to the operator node C2 to inform the operator node C2 that it is the operator node to be deleted. The operator node topology at this time is shown in the middle figure, and the connection represented by the dotted line is the connection to be deleted.

[0108] The control node sends adjustment identification information to the operator node topology diagram shown in the middle figure to complete the adjustment of the operator node topology diagram. The adjusted operator node C2 is deleted, and the connection of the operator node C2 is also deleted, so that the adjusted operator node topology diagram can be obtained, as shown in FIG. Figure 6 As shown in the right figure.

[0109] By adopting the expansion and contraction adjustment method provided in the embodiment of this specification to perform hot migration, it is avoided to stop the task during the hot migration process, and the hot migration is completed during the execution of the task, thereby reducing the impact of the hot migration operation on the task.

[0110] Figure 7 A flowchart of an example 700 of reallocating state key groups when a stateful operator node is expanded according to an embodiment of this specification is shown.

[0111] In this example, the concurrent operator nodes to be adjusted for expansion adjustment may include stateful concurrent operator nodes, and the stateful concurrent operator nodes may include at least one stateful operator node. The stateful operator nodes maintain operator states, and the stateful operator nodes may be stored in the form of state key values ​​(Key), and each state key value is used to maintain a state instance. In each stateful operator node, the state key value may be stored in the form of a state key group (Key Group), and each state key group may include at least one state key value.

[0112] In this example, the stateful concurrent operator nodes may include a first stateful operator node existing before the expansion adjustment and a newly added second stateful operator node. The first stateful operator node may be an operator node that always exists before and after the expansion adjustment.

[0113] like Figure 7 As shown, at 710, when the control node is performing capacity expansion adjustment, a new second stateful operator node can be added to the concurrent operator nodes to be adjusted, and then a first state key group to be assigned to each first stateful operator node and each second stateful operator node is determined, where the first state key group is the key group after adjustment of each stateful operator node in the concurrent operator nodes to be adjusted.

[0114] Then, at 720 , when adjustment identification information is received from an upstream operator node to which the stateful concurrent operator node is connected, each stateful operator node in the stateful concurrent operator node may be adjusted, and the adjustment includes reallocation of the state key group.

[0115] At each first stateful operator node, an original state key group is stored locally. The original state key group of each first stateful operator node is the state key group before the expansion adjustment. The redistribution of the state key group is based on the original state key group of each first stateful operator node.

[0116] At each first stateful operator node, the original state key group of the first stateful operator node can be matched with each determined first state key group, and the first state key group with the highest matching degree can be determined as the second state key group. Each first stateful operator node can determine the second state key group with the highest matching degree. The matching method can be to compare the number of the same state key value, the more the number of the same state key value, the higher the matching degree, and the fewer the number of the same state key value, the lower the matching degree.

[0117] After determining multiple second state key groups, each second state key group corresponds to a stateful operator node. At 730, at the stateful operator nodes corresponding to each second state key group, the state data of the original state key group corresponding to the second state key group can be obtained. In one example, when the original state key group of the stateful operator node corresponding to the second state key group has the highest matching degree with the second state key group of the stateful operator node, the stateful operator node can directly obtain the state data of the original state key group locally. In another example, the state data of each operator node included in the concurrent operator node can be accessed by each other. When the original state key group of the stateful operator node corresponding to the second state key group has the highest matching degree with the second state key group of other stateful operator nodes, the other stateful operator nodes can access the state data of the stateful operator node, so that the state data of the corresponding matching original state key group can be obtained from the stateful operator node.

[0118] For example, the second state key group is KeyGroup(1, 2), the stateful operator node corresponding to the second state key group is the first stateful operator node A1, the original state key group of the second stateful operator node A2 is KeyGroup(1, 2, 3), and the original state key group has the highest matching degree with the second state key group of the first stateful operator node A1, then the first stateful operator node A1 can obtain the state data corresponding to the original state key group KeyGroup(1, 2, 3) from the second stateful operator node A2.

[0119] In one example, when the original state key group obtained by the stateful operator node corresponding to the second state key group includes other state key values ​​​​except the key group in the second state key group, the stateful operator node can delete the state data corresponding to other unnecessary state key values ​​​​and update the state key group so that the updated state key group is the same as the second state key group.

[0120] At 740 , for each stateful operator node corresponding to the first state key group other than the second state key group, the first state key group of the stateful operator node does not have the highest matching degree with any group of original state key values.

[0121] When the stateful operator node receives the adjustment completion information, the stateful operator node can determine that the expansion and contraction adjustment of other operator nodes has been completed, and then the stateful operator node obtains the state data corresponding to the required state key value from other operator nodes in the stateful concurrent operator node to form a first state key group. A stateful operator node can obtain the state data corresponding to the required state key value from one or more other operator nodes. For example, the first state key group of a stateful operator node includes multiple state key values, and the stateful operator node can obtain part of the state key values ​​in the multiple state key values ​​from a stateful operator node, and then obtain another part of the state key values ​​in the multiple state key values ​​from another stateful operator node. The two parts of the state key value can constitute the state data corresponding to the state key value.

[0122] In one example, for each stateful operator node corresponding to the second key group, the obtained original state key group may include part of the state key values ​​in the second key group, and the state data corresponding to the other part of the state key values ​​need to be obtained from other operator nodes in the stateful concurrent operator node. Then, when the stateful operator node receives the adjustment completion information, the state data corresponding to the state key values ​​not included in the original state key group are obtained from other operator nodes in the stateful concurrent operator node to form the second state key group.

[0123] Figure 8 A schematic diagram of an example 800 of reallocating state key groups when a stateful operator node is expanded according to an embodiment of this specification is shown.

[0124] like Figure 8 As shown, the operator node A to be adjusted includes the first stateful operator nodes A1 and A2 that existed before the expansion adjustment, and the newly added second stateful operator node A3. The original state key group of the first stateful operator node A1 is KeyGroup A 11 ,KeyGroup A 11 The state key values ​​in include state key values ​​1, 2 and 3. The original state key group of the first stateful operator node A2 is KeyGroup A 21 ,KeyGroup A 21 The state key values ​​in include state key values ​​4, 5, and 6. Figure 8 The first stage shown is the stage before the expansion adjustment, and each first stateful operator node A1 and A2 maintains the original state key group.

[0125] When the state key group redistribution starts, the control node determines that the first state key group to be allocated to the first stateful operator node A1 is KeyGroup A. 13 ,KeyGroup A 13The state key values ​​in include state key values ​​1 and 2. The first state key group to be assigned to the first stateful operator node A2 is KeyGroup A. 23 ,KeyGroup A 23 The state key values ​​in include state key values ​​3 and 4. The first state key group to be assigned to the second stateful operator node A3 is KeyGroup A 33 ,KeyGroupA 33 The state key values ​​in include state key values ​​5 and 6.

[0126] Then, when the adjustment identification information is received from the upstream operator node connected to the stateful concurrent operator node, at each first stateful operator node (i.e., A1 and A2), the original state key group of each first stateful operator node is matched with each determined first state key group, and the first state key group with the highest matching degree is determined as the second state key group. Specifically, at the first stateful operator node A1, the original state key group KeyGroup A 11 The first state key group is KeyGroup A 13 、KeyGroup A 23 and KeyGroup A 33 By matching, we can determine KeyGroup A 11 With KeyGroup A 13 If there are two state keys with the same value, then KeyGroup A 11 With KeyGroup A 13 The matching degree is the highest, and KeyGroup A can be 13 At the first stateful operator node A2, the original state key group KeyGroup A can be determined as the second state key group. 21 The first state key group is KeyGroup A 13 、KeyGroup A 23 and KeyGroup A 33 By matching, we can determine KeyGroup A 21 With KeyGroup A 33 If there are two state keys with the same value, then KeyGroup A 21 With KeyGroup A 33 The matching degree is the highest, and KeyGroup A can be 33 Determine the second state key group.

[0127] After determining the second state key group, the second state key group KeyGroup A 13The corresponding first stateful operator node A1 can obtain the original state key group KeyGroup A locally 11 The status data of KeyGroup A 11 As KeyGroup A 12 Another second state key group KeyGroup A 33 The corresponding first stateful operator node A3 can obtain the original state key group KeyGroup A from the first stateful operator node A2 21 The status data of KeyGroupA 21 As KeyGroup A 32 .

[0128] For the first stateful operator node A2, when the first stateful operator node A2 receives the adjustment completion information, the state data corresponding to the required state key value can be obtained from the first stateful operator node A1 and the second stateful operator node A3 to form the first state key group. Specifically, the first stateful operator node A2 needs the state data corresponding to the state key value 3 and the state data corresponding to the state key value 4 to form the state key group KeyGroup A. 22 After the state adjustment of the first stateful operator node A1 and the second stateful operator node A3 is completed, the first stateful operator node A2 can obtain the state data of the state key value 3 from the first stateful operator node A1, obtain the state data of the state key value 4 from the second stateful operator node A3, and then form the state key value 3 and the state key value 4 into a state key group KeyGroup A. 22 . At this point, the second stage is completed.

[0129] In the third stage, the first stateful operator node A1 may delete the state data of the state key value 3 to obtain a state key group KeyGroup A including only the state key values ​​1 and 2. 13 The second stateful operator node A3 may delete the state data of the state key value 4 to obtain a state key group KeyGroup A including only the state key values ​​5 and 6. 33 .

[0130] Through the above example of state key group redistribution, each operator node in the concurrent operator node can query each other's state data, so that the required state data can be directly obtained from the local or other concurrent operator nodes, avoiding the splitting and merging of the state data, thereby improving the efficiency of state key group redistribution.

[0131] Fig. 9 A flowchart of an example 900 of a method for scaling an operator node according to an embodiment of this specification is shown.

[0132] Fig. 9 The expansion and contraction adjustment method shown in FIG. 1 can be executed by a control node. Fig. 9 As shown, at 910, in response to a concurrent operator node to be adjusted determined according to a real-time data stream, an operator node topology graph for the concurrent operator node to be adjusted is constructed according to upstream and downstream operator nodes of the concurrent operator node to be adjusted.

[0133] At 920, graph state update information for each adjusted operator node in the operator node topology graph is sent to the operator node so that each adjusted operator node updates its connection, wherein the graph state update information of each operator node includes adjustment information of upstream and downstream operator nodes of the operator node and connection adjustment information for the operator node.

[0134] At 930, the adjustment identification information is sent to the source operator node in the operator node topology graph, so that each operator node obtains the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information.

[0135] At 940 , in response to receiving adjustment identification information from the tail operator node in the operator node topology graph, adjustment completion information is sent to each operator node to notify each operator node that the operator node topology graph after expansion and contraction adjustment is completed.

[0136] In addition, the control node may also perform other operations performed by the control node in the embodiments of this specification.

[0137] Fig.10 A block diagram of an example of an apparatus for scaling an operator node (hereinafter referred to as a scaling adjustment apparatus 1000 ) according to an embodiment of this specification is shown.

[0138] The expansion and contraction adjustment device 1000 can be applied to a control node. Fig.10 As shown, the expansion and contraction adjustment device 1000 includes: a topology map construction unit 1010, a map state update information sending unit 1020, an adjustment identification information sending unit 1030 and an adjustment completion information sending unit 1040.

[0139] The topology map construction unit 1010 is configured to construct an operator node topology map for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted in response to the concurrent operator node to be adjusted determined according to the real-time data flow.

[0140] The graph state update information sending unit 1020 is configured to send graph state update information for each adjusted operator node in the operator node topology graph, so that each adjusted operator node updates the connection, wherein the graph state update information of each operator node includes the upstream and downstream operator node adjustment information of the operator node and the connection adjustment information for the operator node.

[0141] The adjustment identification information sending unit 1030 is configured to send the adjustment identification information to the source operator node in the operator node topology graph, so that each operator node obtains the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information.

[0142] The adjustment completion information sending unit 1040 is configured to send adjustment completion information to each operator node in response to receiving adjustment identification information from the tail operator node in the operator node topology map, so as to notify each operator node that the operator node topology map after expansion and contraction adjustment is completed.

[0143] In one example, the topology map construction unit 1010 can also be configured to: determine the upstream operator node and / or downstream operator node of the concurrent operator node to be adjusted in response to the concurrent operator node to be adjusted determined according to the real-time data stream; determine the first starting operator node and the first ending operator node according to the connection relationship between the concurrent operator node to be adjusted, the upstream operator node and / or the downstream operator node; determine the source operator node according to the first starting operator node, the concurrent operator node to be adjusted and the upstream operator node, or according to the first starting operator node; determine the tail operator node according to the first ending operator node, the concurrent operator node to be adjusted and the downstream operator node, or according to the first ending operator node; and construct an operator node topology map based on the source operator node, the tail operator node, the concurrent operator node to be adjusted, the upstream operator node and / or the downstream operator node.

[0144] In one example, the topology map construction unit 1010 can also be configured to: determine other operator nodes in the upstream operator nodes except the concurrent operator node to be adjusted as the second starting operator node; and the control node merges the first starting operator node and the second starting operator node to determine them as the source operator node; or, when there is no upstream operator node for the concurrent operator node to be adjusted, determine the first starting operator node as the source operator node.

[0145] In one example, the topology map construction unit 1010 can also be configured to: determine the other operator nodes in the downstream operator nodes except the concurrent operator node to be adjusted as the second end operator node; and the control node merges the first end operator node and the second end operator node into a tail operator node; or, when there is no downstream operator node for the concurrent operator node to be adjusted, determine the first end operator node as the tail operator node.

[0146] Reference above Figures 1 to 10 , describes embodiments of a method, device, and system for scaling operator nodes according to embodiments of this specification.

[0147] The device for scaling the operator nodes in the embodiments of this specification can be implemented in hardware, or in software, or in a combination of hardware and software. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the device in which it is located reading the corresponding computer program instructions in the memory into the memory and running them. In the embodiments of this specification, the device for scaling the operator nodes can be implemented, for example, using electronic devices.

[0148] Fig.11 A block diagram of an electronic device 1100 for implementing a method for scaling an operator node according to an embodiment of this specification is shown.

[0149] like Fig.11 As shown, the electronic device 1100 may include at least one processor 1110, a memory (e.g., a non-volatile memory) 1120, a memory 1130, and a communication interface 1140, and the at least one processor 1110, the memory 1120, the memory 1130, and the communication interface 1140 are connected together via a bus 1150. At least one processor 1110 executes at least one computer-readable instruction stored or encoded in the memory (i.e., the above-mentioned element implemented in the form of software).

[0150] In one embodiment, computer executable instructions are stored in a memory, which, when executed, enable at least one processor 1110 to: in response to concurrent operator nodes to be adjusted determined according to real-time data streams, construct an operator node topology graph for the concurrent operator nodes to be adjusted according to the upstream and downstream operator nodes of the concurrent operator nodes to be adjusted; send graph state update information for each operator node to be adjusted in the operator node topology graph, so that each operator node to be adjusted updates the connection, wherein the graph state update information of each operator node includes the upstream and downstream operator node adjustment information of the operator node and the connection adjustment information for the operator node; send adjustment identification information to the source operator node in the operator node topology graph, so that each operator node obtains the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; and in response to receiving the adjustment identification information from the tail operator node in the operator node topology graph, send adjustment completion information to each operator node, so as to notify each operator node that the operator node topology graph adjusted after expansion and contraction is completed.

[0151] It should be understood that the computer executable instructions stored in the memory, when executed, cause at least one processor 1110 to perform the above combined operations in various embodiments of the present specification. Figure 1-10 Describes the various operations and functions.

[0152] According to one embodiment, a program product such as a machine-readable medium is provided. The machine-readable medium may have instructions (i.e., the above-mentioned elements implemented in the form of software), which, when executed by a machine, causes the machine to perform the above-mentioned combination of various embodiments of this specification. Figure 1-10 Describes the various operations and functions.

[0153] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0154] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above-mentioned embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute a part of the present invention.

[0155] The computer program code required for the operation of each part of this specification can be written in any one or more programming languages, including object-oriented programming languages, such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB, NET and Python, conventional procedural programming languages ​​such as C language, Visual Basic 2003, Perl, COBOL2002, PHP and ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages. The program code can be run on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and another part on the remote computer, or all on the remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service, such as software as a service (SaaS).

[0156] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.

[0157] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0158] Not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined according to needs. The device structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0159] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described techniques. However, the techniques may be implemented without these specific details. In some instances, in order to avoid obscuring the concepts of the described embodiments, well-known structures and devices are shown in block diagram form.

[0160] The optional implementation modes of the embodiments of the present specification are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present specification are not limited to the specific details in the above implementation modes. Within the technical concept of the embodiments of the present specification, various simple modifications can be made to the technical solutions of the embodiments of the present specification, and these simple modifications all belong to the protection scope of the embodiments of the present specification.

[0161] The above description of the contents of this specification is provided to enable any person of ordinary skill in the art to implement or use the contents of this specification. Various modifications to the contents of this specification will be apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other variations without departing from the scope of protection of the contents of this specification. Therefore, the contents of this specification are not limited to the examples and designs described herein, but are consistent with the widest range of principles and novel features disclosed herein.

Claims

1. A method for scaling an operator node, comprising: In response to the concurrent operator node to be adjusted determined according to the real-time data stream, the control node constructs an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted; The control node sends graph state update information for each operator node that has been adjusted in the operator node topology graph, so that each operator node that has been adjusted updates its connection, wherein the graph state update information of each operator node includes upstream and downstream operator node adjustment information of the operator node and connection adjustment information for the operator node; The control node sends adjustment identification information to the source operator node in the operator node topology graph, where the adjustment identification information is used to distinguish the states before and after the adjustment in each operator node. When the operator node sends the adjustment identification information, it indicates that the operator node has completed the adjustment, and the adjustment identification information is transmitted sequentially according to the pipeline formed by the operator node topology graph; At each operator node, the adjustment identification information is obtained according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; as well as In response to receiving the adjustment identification information from the tail operator node in the operator node topology graph, the control node sends adjustment completion information to each operator node to notify each operator node that the operator node topology graph after the expansion and contraction adjustment is completed.

2. The method of claim 1, wherein: In response to the concurrent operator node to be adjusted determined according to the real-time data stream, the control node constructs an operator node topology map for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted, including: In response to the concurrent operator node to be adjusted determined according to the real-time data stream, the control node determines an upstream operator node and / or a downstream operator node of the concurrent operator node to be adjusted; The control node determines a first starting operator node and a first ending operator node according to a connection relationship between the concurrent operator node to be adjusted, the upstream operator node and / or the downstream operator node; The control node determines a source operator node according to the first starting operator node, the concurrent operator node to be adjusted, and the upstream operator node, or according to the first starting operator node; The control node determines a tail operator node according to the first end operator node, the concurrent operator node to be adjusted, and the downstream operator node, or according to the first end operator node; and The control node constructs the operator node topology graph based on the source operator node, the tail operator node, the concurrent operator node to be adjusted, the upstream operator node and / or the downstream operator node.

3. The method of claim 2, wherein: The control node determines the source operator node according to the first starting operator node, the concurrent operator node to be adjusted, and the upstream operator node, or according to the first starting operator node, including: The control node determines the other operator nodes in the upstream operator nodes except the concurrent operator node to be adjusted as the second starting operator node; and the control node combines the first starting operator node and the second starting operator node to determine them as the source operator node; or, When the concurrent operator node to be adjusted does not have an upstream operator node, the control node determines the first starting operator node as the source operator node.

4. The method of claim 2, wherein: The control node determines the tail operator node according to the first end operator node, the concurrent operator node to be adjusted, and the downstream operator node, or according to the first end operator node, including: The control node determines the other operator nodes in the downstream operator nodes except the concurrent operator node to be adjusted as the second end operator node; and the control node merges the first end operator node and the second end operator node into the tail operator node; or, When the concurrent operator node to be adjusted does not have a downstream operator node, the control node determines the first end operator node as the tail operator node.

5. The method of claim 1, wherein: The connection adjustment information includes: For capacity expansion adjustment, there are new connections between the newly added operator node and all other operator nodes in the upstream and downstream operator nodes except the operator node to be deleted; and / or, For the scaling-down adjustment, there are connections to be deleted between the operator node to be deleted and all other operator nodes in the upstream and downstream operator nodes except the newly added operator node.

6. The method of claim 1, wherein: At each operator node, acquiring the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information includes: At the operator node to be deleted, receiving the adjustment identification information from an upstream operator node connected to the operator node; and, At the operator node to be deleted, disconnect the connection with the upstream operator node.

7. The method of claim 1, wherein: The concurrent operator nodes to be adjusted for the expansion adjustment include stateful concurrent operator nodes, and the stateful concurrent operator nodes include a first stateful operator node existing before the expansion adjustment and a newly added second stateful operator node. The method further comprises: The control node determines a first state key group to be assigned to each first stateful operator node and each second stateful operator node; When the adjustment identification information is received from the upstream operator node connected to the stateful concurrent operator node, at each of the first stateful operator nodes, the original state key groups of each of the first stateful operator nodes are matched with the determined first state key groups, and the first state key group with the highest matching degree is determined as the second state key group; At the stateful operator nodes corresponding to each second state key group, obtaining the state data of the original state key group corresponding to the second state key group; and For the stateful operator nodes corresponding to each first state key group other than the second state key group, when the stateful operator node receives the adjustment completion information, the state data corresponding to the required state key value is obtained from other operator nodes in the stateful concurrent operator nodes to form a first state key group.

8. The method of claim 7, wherein: For each stateful operator node corresponding to the second key group, the obtained original state key group includes part of the state key values ​​in the second key group. The method further comprises: When the stateful operator node receives the adjustment completion information, the stateful operator node obtains state data corresponding to the state key value not included in the original state key group from other operator nodes in the stateful concurrent operator node to form the second state key group.

9. The method of claim 1, wherein: The adjustment of the concurrent operator node to be adjusted includes expansion adjustment and / or reduction adjustment.

10. A method for scaling an operator node, performed by a control node, the method comprising: In response to the concurrent operator node to be adjusted determined according to the real-time data stream, constructing an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted; Sending graph state update information for each adjusted operator node in the operator node topology graph to each adjusted operator node so that each adjusted operator node updates its connection, wherein the graph state update information of each operator node includes adjustment information of upstream and downstream operator nodes of the operator node and connection adjustment information for the operator node; Sending adjustment identification information to the source operator node in the operator node topology graph, so that each operator node obtains the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information, wherein the adjustment identification information is used to distinguish the states before and after the adjustment in each operator node, and when the operator node sends the adjustment identification information, it indicates that the operator node has completed the adjustment, and the adjustment identification information is transmitted sequentially according to the pipeline formed by the operator node topology graph; and In response to receiving the adjustment identification information from the tail operator node in the operator node topology graph, adjustment completion information is sent to each operator node to notify each operator node that the operator node topology graph after the expansion and contraction adjustment is completed.

11. A device for scaling an operator node, the device being applied to a control node, the device comprising: at least one processor, a memory coupled to the at least one processor, and A computer program stored on the memory, the at least one processor executing the computer program to implement: In response to the concurrent operator node to be adjusted determined according to the real-time data stream, constructing an operator node topology graph for the concurrent operator node to be adjusted according to the upstream and downstream operator nodes of the concurrent operator node to be adjusted; Sending graph state update information for each adjusted operator node in the operator node topology graph to each adjusted operator node so that each adjusted operator node updates its connection, wherein the graph state update information of each operator node includes adjustment information of upstream and downstream operator nodes of the operator node and connection adjustment information for the operator node; Sending adjustment identification information to the source operator node in the operator node topology graph, so that each operator node obtains the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information, wherein the adjustment identification information is used to distinguish the states before and after the adjustment in each operator node, and when the operator node sends the adjustment identification information, it indicates that the operator node has completed the adjustment, and the adjustment identification information is transmitted sequentially according to the pipeline formed by the operator node topology graph; and In response to receiving the adjustment identification information from the tail operator node in the operator node topology graph, adjustment completion information is sent to each operator node to notify each operator node that the operator node topology graph after the expansion and contraction adjustment is completed.

12. A system for scaling an operator node, the system comprising a control node and a plurality of operator nodes included in an operator node topology graph constructed by the control node, The control node is used to respond to the concurrent operator nodes to be adjusted determined according to the real-time data flow, construct an operator node topology graph for the concurrent operator nodes to be adjusted according to the upstream and downstream operator nodes of the concurrent operator nodes to be adjusted; send graph state update information for the operator nodes to be adjusted in the operator node topology graph, so that the respective operator nodes to be adjusted update their connections, wherein, The graph state update information of each operator node includes the adjustment information of the upstream and downstream operator nodes of the operator node and the connection adjustment information for the operator node; and sending adjustment identification information to the source operator node in the operator node topology graph, wherein the adjustment identification information is used to distinguish the states before and after the adjustment in each operator node. When the operator node sends the adjustment identification information, it indicates that the operator node has completed the adjustment, and the adjustment identification information is transmitted sequentially according to the pipeline formed by the operator node topology graph; Each operator node is used to obtain the adjustment identification information according to the connection between the operator node and the upstream and downstream operator nodes, so as to adjust the state of the operator node based on the adjustment identification information; The control node is further configured to, in response to receiving the adjustment identification information from the tail operator node in the operator node topology graph, send adjustment completion information to each operator node to notify each operator node that the operator node topology graph after the expansion and contraction adjustment is completed.

13. A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.