Shrinking processing method, computing cluster system, electronic device and readable storage medium

Abstract

The application discloses a method for shrinkage processing, a computing cluster system, an electronic device and a readable storage medium. The method comprises the following steps: obtaining a computing node to be offline; marking an operator running on each computing node in the computing node to be offline as a to-be-transferred operator; wherein the operator is used for distributed computing of a computing task on the computing node; for all marked operators belonging to the same computing task in the computing node to be offline, triggering a transfer, and in the transfer, at least multiple operators are transferred for one computing task; and after all operators in the computing node to be offline are transferred to other computing nodes, the computing node to be offline is offline. The application solves the problem that in the prior art, when the distributed computing needs to be shrunk, the computing task is interrupted multiple times, thereby affecting the processing efficiency of the computing task, and further reduces the number of transfers in the shrinkage process, and improves the processing efficiency of the computing task.

Description

Technical Field

[0001] This application relates to the field of distributed computing, and more specifically, to methods for scaling down, computing cluster systems, electronic devices, and readable storage media. Background Technology

[0002] With the development of technology, the demand for computing resources for a computing task is increasing, and some computing tasks also have high real-time requirements. In such cases, using a single computing node to run a computing task will not meet the requirements. To solve this problem, distributed computing has been introduced into existing technologies.

[0003] Distributed computing is a computing method that is the opposite of centralized computing. Distributed computing breaks down a computational task into many smaller parts and distributes them to multiple computing nodes for processing. This saves overall computation time and greatly improves computational efficiency.

[0004] In distributed computing, scaling down computing nodes is often necessary. For example, if a computing node has too much data and provides excess computing power, it can be scaled down. In distributed computing, each computing node runs a portion of the computational task. Scaling down requires transferring the task from one node to another, and then taking that node offline. Each time a computing node is taken offline, a task failover occurs, causing an interruption of the computational task. Because distributed computing tasks are spread across multiple computing nodes, scaling down can lead to multiple interruptions of the computational task, impacting processing efficiency. Summary of the Invention

[0005] This application provides a scaling-down processing method, a computing cluster system, an electronic device, and a readable storage medium to at least solve the problem in the prior art where multiple interruptions of computing tasks occur when scaling down distributed computing requires taking computing nodes offline, thus affecting the processing efficiency of computing tasks.

[0006] According to one aspect of this application, a scaling-down processing method is provided, comprising: acquiring computing nodes to be taken offline, wherein the computing nodes are used for distributed streaming computing; marking operators running on each computing node to be taken offline as operators to be transferred; wherein the operators are used to perform distributed computing tasks on the computing nodes; triggering a transfer for all marked operators belonging to the same computing task in the computing nodes to be taken offline, wherein the transfer is used to transfer operators under the computing task to other computing nodes besides the computing nodes to be taken offline, wherein the transfer involves at least one transfer of multiple operators for the same computing task; and taking the computing nodes to be taken offline after all operators in the computing nodes to be taken offline have been transferred to the other computing nodes.

[0007] According to another aspect of this application, a computing cluster system is also provided, wherein the computing cluster system comprises multiple computing nodes, and the above-described method is used when taking computing nodes offline.

[0008] According to another aspect of this application, an electronic device is also provided, including a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the above-described method steps.

[0009] According to another aspect of this application, a readable storage medium is also provided, on which computer instructions are stored, wherein the computer instructions, when executed by a processor, implement the above-described method steps.

[0010] In this embodiment, the following methods are employed: First, obtain computing nodes to be taken offline; mark the operators running on each of these nodes as operators to be transferred; wherein the operators are used to perform distributed computation on the computing nodes; for all marked operators belonging to the same computing task within the computing nodes to be taken offline, a transfer is triggered, wherein the transfer is used to transfer the operators under the computing task to other computing nodes besides the computing nodes to be taken offline, and at least one transfer of multiple operators is performed for the same computing task; after all operators in the computing nodes to be taken offline have been transferred to the other computing nodes, the computing nodes to be taken offline are then taken offline. This application solves the problem in the prior art where multiple interruptions of computing tasks occur when scaling down distributed computing and computing nodes need to be taken offline, thus affecting the processing efficiency of computing tasks, thereby reducing the number of transfers during the scaling down process and improving the processing efficiency of computing tasks. Attached Figure Description

[0011] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0012] Figure 1 This is a schematic diagram of a distributed streaming computing scenario according to an embodiment of this application;

[0013] Figure 2 This is a flowchart of a scaling-down processing method according to an embodiment of this application;

[0014] Figure 3 This is a schematic diagram of operators in a computing node according to an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of the offline computing center time transfer operator according to an embodiment of this application; and,

[0016] Figure 5 This is a schematic diagram illustrating Kubernetes scaling down according to an embodiment of this application. Detailed Implementation

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0019] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0020] The following implementation methods can be used to handle scaling down in various distributed computing scenarios, especially in distributed streaming computing. This is because streaming computing has high timeliness requirements; the following implementation methods, when applied to streaming computing, can reduce the duration of computing task interruptions caused by scaling down, thereby improving the execution efficiency of streaming computing tasks. First, we will explain what streaming computing is. The following optional implementation methods can be applied to big data computing scenarios, especially distributed streaming computing for big data.

[0021] In traditional data processing workflows, data is first collected and then stored in a database. When users need it, they query the database to get answers or perform related processing. For example, data is typically stored in a table before processing and analysis, which raises the issue of timeliness. If processing data at the yearly or monthly level, the timeliness requirement is not high; however, if users are dealing with data at the day, hour, or even minute level, the timeliness requirement becomes much higher. In scenarios with high timeliness requirements, if the traditional data processing method of uniformly collecting data, storing it in a database, and then analyzing it may not meet the timeliness requirements.

[0022] To address the challenge of high timeliness in data processing, stream computing (or streaming computation) has been introduced into existing technologies. Stream computing processes data streams (or streaming data) in real-time. A data stream refers to a dynamic collection of data in terms of time distribution and quantity. Since the value of data streams generally decreases over time (e.g., collected alarm information), real-time computation is necessary to provide a rapid response.

[0023] To further improve the response time of streaming computing, distributed computing is used in existing technologies, where a streaming computing task is distributed across different computing nodes for processing. Figure 1 This is a schematic diagram of a distributed streaming computing scenario according to an embodiment of this application, such as... Figure 1 As shown, a computing cluster can include multiple servers, which can be of two types: physical servers (simply referred to as servers) and virtual servers, which are the opposite of physical servers. Virtual servers are servers that run on physical servers using virtual machines. Figure 1 The computing cluster can include virtual servers and / or servers, for example, including N servers (i.e., server 1 to server N), and can also include M virtual servers (virtual server 1 to virtual server M). Both servers and virtual servers are referred to as computing nodes in the computing cluster. Figure 1 Midstream computing tasks are distributed across multiple computing nodes for distributed computation.

[0024] In distributed computing, if the number of computing nodes is large, scaling down is performed to reduce the amount of data on each node. Current technology involves failover of streaming tasks on each node to other nodes. This means that each node failure causes an interruption in the streaming task. Since a streaming task is distributed across multiple nodes, scaling down a large number of nodes can lead to multiple failovers, resulting in prolonged interruptions in streaming task data flow and impacting its normal operation. It's important to note that distributed computing is managed by computing nodes. Therefore, scaling up or down computing nodes is typically done node-by-node. Especially during scaling down, considering that each node may be running multiple computing tasks, each node failure affects multiple tasks. If these tasks are streaming tasks, given the real-time requirements of streaming computing, scaling down can significantly impact them.

[0025] To address the aforementioned problems, a volume reduction processing method is provided in the following embodiments. Figure 2 This is a flowchart of a scaling-down processing method according to an embodiment of this application, such as... Figure 2 As shown below, Figure 2 The steps involved in the method are explained.

[0026] Step S202: Obtain the computing nodes to be taken offline.

[0027] In this step, a computing node can be a computing node within a computing cluster, which can be distributed across the same or different geographical regions. Alternatively, it can be described as a computing node performing distributed computing alongside other computing nodes. Both computing nodes within a cluster (which essentially performs distributed computing) and those performing distributed computing can be dynamically adjusted. That is, if computing power is insufficient, more computing nodes can be added; conversely, if computing power is abundant, fewer computing nodes can be removed. Removing a computing node means taking it offline from the distributed computing framework or the computing cluster. The offline computing node can then be used for other tasks.

[0028] There are many ways to determine which computing nodes should be taken offline. For example, one can obtain the average utilization rate of computing resources across all nodes in the computing cluster over a predetermined period. If the average utilization rate is less than a pre-set threshold, it indicates that there is surplus computing resources, and the nodes to be taken offline can be determined based on this average utilization rate. Alternatively, one can obtain the computational load of the task at different times each day and find the time period with the lowest computational load. If the computational load is consistently the lowest within the same time period each day, the nodes to be taken offline can be determined based on this time period. For instance, if the computing cluster is used to find traffic violations in video streams, statistics show that traffic flow is usually lowest between 1 AM and 5 AM, requiring the least amount of computation. Therefore, an automated plan can be set up to take a predetermined number of computing nodes offline at 1 AM each day and bring them back online at 5 AM.

[0029] Step S204: Mark the operator running on each computing node in the computing nodes to be taken offline as the operator to be transferred, wherein the operator is used to perform distributed computing on the computing node.

[0030] An operator is a commonly used term in computing, referring to a computational unit that executes a certain algorithm. An operator can be a software program, which allows an operator to run on a device. For computing nodes, which provide computing resources (such as processors and memory), operators are the computational units that run on these resources. Different computational tasks may use different algorithms, and different algorithms correspond to different operators.

[0031] Figure 3 This is a schematic diagram of the operators in the computation node according to an embodiment of this application, such as... Figure 3 As shown, there are two computational tasks ( Figure 3 (Taking streaming computing tasks as an example) These are streaming computing task A and streaming computing task B. Operator A processes streaming computing task A, and operator B processes streaming computing task B. Figure 3 There are four computing nodes: Computation Node 1, Computation Node 2, Computation Node 3, and Computation Node 4. Computation Nodes 1, 2, and 3 are used for distributed processing of streaming computing task A, and each of these three nodes runs operator A. Computation Nodes 2, 3, and 4 are used for distributed processing of streaming computing task B, and each of these three nodes runs operator B. Because Computation Nodes 2 and 3 have executed both streaming computing tasks A and B, they contain both operator A and operator B. Figure 3 This clearly explains the relationship between operators and computation nodes.

[0032] Step S206: For all operators marked under the same computing task in the computing node to be taken offline, a transfer is triggered. The transfer is used to transfer the operators under the computing task to other computing nodes besides the computing node to be taken offline. In the transfer, at least one transfer of multiple operators is performed for the same computing task.

[0033] In steps S206 and S208, the term "transfer" is involved. There are many ways to transfer data; for example, failover is one method. Failover uses the failover capability of streaming computing tasks to transfer operators from a failed computing node to a normal settlement node. Of course, other types or methods of transfer are also applicable. In the following implementation, failover will be used as an example for explanation.

[0034] In step S206, operator fault transfer is performed based on the computation task. When transferring all operators to be transferred under the same computation task, multiple operators can be transferred at once. As a preferred implementation, all operators to be transferred can be transferred in one fault transfer, thus completing the transfer of all operators to be transferred in the current computation task in one fault transfer. This transfer method differs from operator transfer based on the computation center. Figure 4 This is a schematic diagram of the offline computing center time transfer operator according to an embodiment of this application, and... Figure 3 The difference is that, in Figure 4 Two new compute nodes, 4 and 5, have been added to the system. These two compute nodes are used to process streaming task C. Now, compute nodes 4 and 5 need to be taken offline. If we take them offline one by one, first taking compute node 4 offline requires moving operator C to compute node 1, which will cause an interruption to streaming task C. Then, taking compute node 5 offline requires moving operator C to compute node 3, which will also cause another interruption to streaming task C. Figure 4 The example only shows the offline status of two computing nodes. If multiple computing nodes go offline, multiple interrupts will be required, which will inevitably affect the execution of the streaming computing task C.

[0035] In this step, operators are transferred according to the computational task. During the transfer process, there is at least one fault transfer that transfers multiple operators. Figure 4For example, if operator transfer is performed according to step S206, it is first determined that both computing node 4 and computing node 5 have operator C. In this case, operator C is directly transferred to computing node 1 and computing node 3 respectively. This transfer method results in fewer interruptions than the transfer performed node by node. When the number of nodes that need to be taken offline is large, the transfer step in step S206 will achieve better results.

[0036] Step S208: After all the operators in the computing node to be taken offline have been transferred to the other computing nodes, the computing node to be taken offline is taken offline.

[0037] In this step, after all operators in the computing node to be taken offline have been transferred to other nodes, the node can be taken offline. It should be noted that when selecting other computing nodes to receive operators, the load of each node capable of receiving operators can be considered; nodes with lower loads can be prioritized. Alternatively, nodes with the same type of operator can be selected to receive the transferred operators. For example, assuming in... Figure 4 There is also a computing node 6, which runs operator C. When computing nodes 4 and 5 are taken offline, the operators of these two computing nodes can be moved to computing node 6. This approach makes the operators running on the computing nodes more singular, which is beneficial to improving efficiency. It is also beneficial to summarize the calculation results.

[0038] In the above steps, when a computing node is taken offline, the transfer can be performed based on operators belonging to the same computing task within the computing nodes to be taken offline. Multiple operators can be transferred at once during the transfer. Therefore, this step solves the problem in existing distributed computing technologies where multiple interruptions of computing tasks occur when scaling down and computing nodes need to be taken offline, thus affecting the processing efficiency of computing tasks. It reduces the number of transfers during the scaling-down process and improves the processing efficiency of computing tasks. Compared to the original method of taking offline each computing node individually, this step only requires one transfer, regardless of the number of computing nodes involved in a computing task. For example, assuming 10 computing nodes are to be taken offline, and each computing node runs operator A, taking offline each computing node individually would require 10 transfers. However, the above method... Figure 2 The process, which involves taking computational tasks offline, requires only one transfer, reducing the time consumption to just one-tenth of the original. If more computational nodes are to be taken offline, even more time can be saved, significantly improving the efficiency and real-time performance of stream computing.

[0039] By analyzing the above Figure 2Analysis of the steps reveals that if a single transfer completes all marked operators, the computational tasks containing these operators will only be interrupted once. This is a relatively good implementation method. Specifically, in this implementation, triggering a transfer for all marked operators belonging to the same computational task can include the following steps: triggering a transfer for all operators belonging to the same computational task, wherein the single transfer transfers all operators under the computational task to the other nodes. Still using... Figure 4 For example, suppose we need to take compute nodes 1, 2, and 3 offline. All three nodes contain operator A. Therefore, we can transfer all three operators at once, for example, to compute nodes 4 and 5. Compute nodes 2 and 3 both contain operator B. In this case, we can also transfer operator B at once to compute nodes 4 and 5. This approach minimizes the number of interruptions for each compute task, resulting in minimal impact on streaming compute tasks A and B.

[0040] The above implementation method does indeed have the least impact on computing tasks, but another situation may arise in practical applications. For example, there are 20 computing nodes executing streaming computing task A. Now, 15 of these computing nodes need to be taken offline. If other computing nodes that can receive these operators can be found, the operator A on these computing nodes can be directly transferred. However, if it is necessary to first transfer the operator A from a portion of the 15 computing nodes, and then determine which computing nodes to transfer the operators of the other portion of computing nodes to based on the characteristics of the other nodes in the computing cluster besides the 15 computing nodes to be taken offline, then in this case, all operators belonging to the same computing task can be grouped, and a transfer can be triggered for each group. In this case, all operators in each group are transferred to the other nodes at once, and at least one group includes multiple operators. For example, operators A from 15 compute nodes awaiting shutdown can be divided into two groups: a first group with 7 operators and a second group with 8 operators. Initially, the compute nodes with the lowest load are selected to transfer the first group of operators. At this point, some compute nodes receive operators A, increasing the load. Then, the load is reordered, and the compute nodes with the lowest load are selected to transfer the second group of operators A. Although this approach involves two interruptions, it ensures that less loaded compute nodes receive the operators, and the impact of these two interruptions on the computational task is much smaller than the multiple interruptions caused by each node transfer.

[0041] The above optional embodiments illustrate the operator transitions. It should be noted that the transition steps in these optional embodiments are applicable to any computation node to be taken offline, for example... Figure 4 In the scenario shown, the above-described transfer method can be used to take offline compute nodes 1, 2, and 3; similarly, the above-described transfer method can also be used to take offline compute nodes 2, 3, and 4. Although the above implementation method achieves the goal of reducing the number of interruptions regardless of which compute node is taken offline, a scaling-down process can achieve better results if the appropriate compute node to be taken offline is selected.

[0042] There are several scenarios when selecting compute nodes to be taken offline, which will be explained below.

[0043] Scenario 1: A certain computing node needs maintenance, for example, Figure 4 Computing node 1 needs to be upgraded, and the upgrade process must be handled offline; therefore, computing node 1 will be taken offline. In this case, a software interface can be provided where the user can input the name of computing node 1, and then the computing node corresponding to that name will be designated as the computing node to be taken offline. The software interface can provide an input box for the user to directly enter the name of the computing node to be taken offline, or alternatively, it can display the names of all computing nodes for the user to select. While displaying all computing node names, the interface can also show the current running status of each computing node and the operators currently running on that node. If a computing node's running status is abnormal, the user can also select that computing node to take offline.

[0044] Scenario 2: The user needs K compute nodes to perform other computing tasks. In this case, the user can directly input the names of the compute nodes to be taken offline to determine the nodes to be taken offline. However, if the number of K compute nodes to be taken offline is large, inputting them one by one is time-consuming. In this case, the user can directly input a number, and then the system will automatically select K compute nodes to be taken offline based on the number of user-inputted data and a predefined algorithm (minimum load, minimum number of running operators, etc.).

[0045] Scenario 3: The user is unsure how many compute nodes need to be migrated; the user only knows the required amount of computing resources, such as 512MB of memory and an 8-core CPU. In this case, the user's input requirements can be obtained, and the compute nodes to be decommissioned can be determined based on these requirements. The requirements indicate the amount of computing resources to be released, and the sum of the computing resources occupied by the determined decommissioned compute nodes satisfies the requirements. After selecting the compute nodes to be decommissioned based on the user's input requirements, the selected compute nodes can be displayed to the user for confirmation.

[0046] All three scenarios described above can select suitable computing nodes to be taken offline. In practical applications, at least one of these three scenarios can be flexibly selected as needed. When comparing the number of computing nodes to be taken offline, it's possible that computing node 1 decides to transfer its operator to computing node 6, which is also a node to be taken offline. While this can still transfer the operator to a working node, the transfer process is much more complex, requiring a transfer from computing node 1 to computing node 6, and then from computing node 6 to a working settlement node, reducing efficiency. To address this, in an optional implementation, after obtaining the computing nodes to be taken offline, they can be marked as "disabled from scheduling." Newly created operators will not be scheduled to nodes marked as "disabled from scheduling." This indication clearly identifies which settlement nodes cannot receive operators; this approach is similar to a blacklist, hence the term "blacklisting." The blackening operation can be used not only to set indication information for compute nodes that are about to go offline, but also to set indication information for compute nodes that are not offline but whose computing load is already too large to receive new operators.

[0047] This blacking operation clearly indicates which nodes can receive transferred operators and which cannot. In this case, the indication information of each computing node can be obtained first (to determine whether the computing node is blacked out). Then, for all operators belonging to the same computing task among the computing nodes to be taken offline, the transfer is triggered to computing nodes not marked as prohibited from scheduling. This optional implementation method can accurately select computing nodes that can receive operators. When blacking out computing nodes, as an optional implementation method, indication information can also be set for specific computing tasks (i.e., specific operators) to indicate that the computing node will not receive operators transferred from one or more computing tasks. For example, computing node 1 blacking out operator C indicates that computing node 1 will not receive the transferred operator C; for example, computing node 4 blacking out operator A indicates that computing node 4 will not receive the transferred operator A, and so on. When blacking out a certain computing task or a certain type of operator, the implementation principle is the same as blacking out the computing node, and will not be repeated here. The appropriate blacking operation can be flexibly selected according to the needs in application.

[0048] In distributed computing, typically only computing nodes are managed, while the computing tasks running on those nodes are not. This node-based management approach prevents computing personnel from considering operator transfer at the task-level granularity. After introducing the task-based operator transfer approach described in the above implementation, it's possible to add management of computing tasks. In one optional implementation, besides highlighting computing nodes, tasks, or operators, specific functional modules can be set up to manage each computing task for better maintenance. In one optional implementation, a computing task includes a Jobmanager component and at least one Taskmanager component. The Taskmanager component runs on the computing node performing the computation, and the operators run within it. The Jobmanager component manages at least one Taskmanager component. Through this optional implementation, the Jobmanager component can obtain all the operators of the computing task through the Taskmanager components it manages. That is, it can obtain the operators running on each computing node of the computing node to be taken offline through the Taskmanager components within the computing task. In this optional implementation, each computing task has a work management component, and each operator resides within a task management component. This is highly helpful for identifying operators belonging to the same computing task on different computing nodes. Alternatively, this method can be used to manage each computing task. For example, operators on computing nodes can be configured with identification information of their respective computing tasks. Statistics based on the identification information of the computing tasks to which operators belong can also be used to identify operators belonging to the same computing task on different computing nodes.

[0049] As mentioned above, operators belonging to the same computing task can exist on different computing nodes, and similarly, operators belonging to different computing tasks can exist on the same computing node. To facilitate the operation of different operators and to provide a certain degree of isolation between them, in an optional implementation, the task management component containing the operator can be configured to run in a container. Specifically, in this optional implementation, obtaining the operators running on each computing node to be taken offline can include the following steps: obtaining the container with the task management component set up on the computing node to be taken offline, and identifying the obtained container; obtaining the operator from the task management component in the container with the identified identifier. A container is a technology used to bundle an application and all its necessary files into a runtime environment. As a unit, a container can be easily moved and run on any operating system in any environment. Using containers can isolate software, enabling it to run independently in different operating systems, hardware, networks, storage systems, and security policies. After running operators within a container, each operator is independent of the others, minimizing their mutual impact.

[0050] As can be seen from the above optional implementation methods, the above methods can be applied to computing clusters (also known as computing systems). In a computing cluster system with multiple computing nodes, the method described in the above implementation methods can be used when taking a computing node offline. The above implementation methods can not only be applied to computing clusters, but also to some existing software used for distributed stream computing, to supplement the offline functionality of these software programs. The following uses Kubernetes as an example. In Kubernetes, a pod is used as a special type of container. Unlike containers in the usual sense, a pod can contain multiple containers; therefore, a pod can be considered a special type of container.

[0051] Kubernetes, or K8s for short, is an abbreviation of "ubernete," using the number 8 to represent the eight characters in its name. Kubernetes is used to manage containerized applications across multiple hosts on a cloud platform. Its goal is to make deploying containerized applications simple and efficient. Kubernetes provides deployment through containers, where each container is isolated from the others, has its own file system, and processes within containers do not interfere with each other, allowing for the differentiation of computing resources. Compared to virtual machines, containers can be deployed quickly. Because containers are decoupled from the underlying infrastructure and machine file system, they can migrate between different clouds and different operating system versions. Containers consume fewer resources and deploy quickly. Each application can be packaged into a container image, and the one-to-one relationship between each application and container gives containers a significant advantage. Using containers allows for the creation of container images for applications at different development stages, as each application does not need to be combined with other application stacks or depend on the production environment infrastructure, providing a consistent environment from development to testing and production. In Kubernetes, the JobManager is the central controller for managing and scheduling operators within streaming computing tasks; the TaskManager is the smallest unit for allocating computing resources for streaming computing tasks and is used for job execution.

[0052] In a Kubernetes environment, scaling down compute nodes evicts existing pods. If the application has multiple replicas configured, Kubernetes uses its internal mechanisms for failover, thus providing some scalability. However, it lacks a specific scaling mechanism for streaming tasks where TaskManagers are distributed across different compute nodes. When streaming tasks run in a Kubernetes cluster, they rely on compute node scaling capabilities. Due to inconsistent scaling paces and the distribution of TaskManagers across different compute nodes, streaming tasks often experience multiple failovers. Failover here refers to the rapid activation of redundant or backup servers, systems, hardware, or networks to take over the work when a running streaming job unexpectedly terminates.

[0053] To address this issue, we can first blacklist the compute nodes that need to be taken offline, then eviction tags can be applied to the pods running Taskmanagers on the blacklisted compute nodes. Furthermore, we can design an eviction tag awareness mechanism within the stream computing task. When it detects a list of pods with eviction tags for the current stream computing Taskmanager, it immediately separates all Taskmanagers that need to be evicted (i.e., removes the Taskmanager from the compute node to be taken offline). Then, it calls the Kubernetes service to create a new Taskmanager pod on an unblacklisted node. This ensures that only one failover occurs during the entire scaling-down process, shortening the impact on data traffic during the scaling-down of the stream computing cluster.

[0054] Figure 5 This is a schematic diagram illustrating Kubernetes scaling down according to an embodiment of this application, as shown below. Figure 5 As shown, the stream computing cluster has multiple computing nodes. Figure 5 The diagram only shows four compute nodes participating in the scaling down: compute node A, compute node B, compute node C, and compute node D. Compute node A runs Jobmanager, and the Taskmanager (represented as T in the diagram) from compute node C needs to be moved to compute nodes B and D. Figure 5 The following modules are shown: blacking module, marking module, offline module, sensing module, and separation module. The sensing module and separation module are located in the computing cluster, while the blacking module, marking module, and offline module can be located outside the computing cluster. They can be implemented by network elements with management capabilities or other devices with computing capabilities.

[0055] Figure 5 This paper presents a solution for the rapid scaling down of streaming computing clusters, ensuring that failover of streaming jobs within the cluster occurs only once during the scaling down process, thus shortening the interruption time of data flow during cluster scaling down. For example... Figure 5 As shown, it mainly consists of four parts: a highlighting module, a marking module, a perception module, and a separation module, and these four modules are executed sequentially. The highlighting and marking modules are external to the stream computing cluster. The perception and separation modules are built into the stream computing task, i.e., internal modules of Jobmanager. These modules will be described in detail below.

[0056] The blacklisting module takes a list of compute nodes to be scaled down or the number of machines to be scaled down as input, and calls the blacklisting application programming interface (API) exposed by the stream computing cluster to perform the blacklisting operation. If the input is the number of compute nodes, the cluster's query compute node API needs to be called to convert it into a list of compute node names before calling the blacklisting API to perform the blacklisting operation. Once a compute node is blacklisted, no operators can be scheduled to it.

[0057] The marking module takes a list of blacked-out compute nodes as input. It retrieves the list of pods for the Taskmanager streaming task on the corresponding compute node and marks them as pods that need to be evicted.

[0058] The perception module is a module built into the Jobmanager component inside the streaming computing task. The perception module will only perceive the list of Taskmanagers that are to be evicted associated with the current streaming computing task and pass the list to the separation module.

[0059] The detach module, which is also a module of the Jobmanager component inside the streaming task, receives a list of Taskmanagers with eviction tags and performs a one-time detach operation. This will trigger a failover of the streaming task and cause a one-time interruption of the processing flow of the streaming task.

[0060] Once the above four modules have completed their steps, the compute nodes being scaled down will no longer have any related Taskmanager pods running stream computing. The compute nodes in the scaling-down list will then be removed from the cluster without further failover, thus completing all scaling-down operations.

[0061] exist Figure 5 In this process, external modules—a blacklisting module and a marking module—notify nodes that they can no longer schedule Taskmanager pods and mark pods running Taskmanager that need to be detached. Then, the awareness and detachment modules built into the Jobmanger perform a single detachment operation for the same job. This ensures that during the entire scaling down process, the same streaming task only undergoes one failover. Finally, the offline module removes the compute node from the streaming task. Since the entire scaling down process involves only one failover for the streaming task, the data flow interruption time is shortened.

[0062] In this embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the methods described in the above embodiments.

[0063] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0064] These computer programs may also be loaded onto a computer or other programmable data processing equipment, causing a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented using different modules, and different steps can be implemented using different modules.

[0065] This embodiment provides such an apparatus, referred to as a scaling-down processing method, comprising: an acquisition module for acquiring computing nodes to be taken offline; a marking module for marking operators running on each computing node to be taken offline as operators to be transferred; wherein the operators are used to perform distributed computing tasks on the computing nodes; a triggering module for triggering a transfer for all marked operators belonging to the same computing task in the computing nodes to be taken offline, wherein the transfer is used to transfer operators under the computing task to other computing nodes besides the computing nodes to be taken offline, and the transfer involves at least one transfer of multiple operators for the same computing task; and a shutdown module for taking the computing nodes to be taken offline after all operators in the computing nodes to be taken offline have been transferred to the other computing nodes.

[0066] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0067] Optionally, the triggering module is configured to trigger a transfer for all operators belonging to the same computing task, wherein the transfer transfers all operators under the computing task to the other nodes; or, to group all operators belonging to the same computing task, and trigger a transfer for each group, wherein all operators under each group are transferred to the other nodes at once, and at least one group includes multiple operators.

[0068] Optionally, the acquisition module is used to acquire the name of the computing node that needs to be taken offline, input by the user, and take the computing node corresponding to the name of the computing node as the computing node to be taken offline; and / or, acquire the user's input requirement or the number of computing nodes that need to be taken offline, and determine the computing nodes to be taken offline based on the requirement or the number of computing nodes that need to be taken offline; wherein, the requirement is used to indicate the number of computing resources that need to be released, and the sum of the number of computing resources occupied by the determined computing nodes to be taken offline satisfies the requirement.

[0069] Optionally, the acquisition module is further configured to mark the computing nodes to be taken offline as prohibited from scheduling, wherein newly created operators will not be scheduled to computing nodes marked as prohibited from scheduling.

[0070] Optionally, the triggering module is used to trigger a transfer to computing nodes that are not marked as prohibited from scheduling for all operators belonging to the same computing task among the computing nodes to be taken offline.

[0071] Optionally, the marking module is used to obtain the operators running on each computing node of the computing node to be taken offline through the task management component in the computing task, wherein a computing task includes a work management component and at least one task management component, the task management component runs on the computing node that performs the computing task, the operators run inside the task management component, and the work management component is used to manage at least one task management component.

[0072] Optionally, the marking module is used to obtain containers with task management components in the computing nodes to be taken offline, and to mark the obtained containers; and to obtain operators in the task management components from the containers with the marks.

[0073] Optionally, the computing node to be taken offline is a computing node in the same computing cluster, and the other computing nodes are computing nodes in the computing cluster other than the computing node to be taken offline; and / or, the computing node to be taken offline and the other computing nodes are used for streaming computing.

[0074] Optionally, the transfer is a fault transfer under streaming computing, which is one of the types of transfers in which the operator is transferred to other computing nodes after the computing task is interrupted.

[0075] It should be noted that the first acquisition module implements the functions of the aforementioned blackening and marking modules, while the second acquisition and triggering modules implement the functions of the sensing and separation modules. The offline module in this device functions similarly to... Figure 5 The offline module has the same function.

[0076] The above optional implementation methods solve the problem in the prior art where multiple interruptions of computing tasks are caused when computing nodes need to be taken offline during distributed computing downsizing, thus affecting the processing efficiency of computing tasks. This reduces the number of transfers during the downsizing process and improves the processing efficiency of computing tasks.

[0077] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for reducing volume, comprising: Obtain multiple computing nodes to be taken offline, wherein the computing nodes are used for distributed streaming computing; The operator running on each of the plurality of computing nodes to be taken offline is marked as the operator to be transferred; wherein, the operator is used to perform distributed computing on the computing node; The operator transfer is performed according to the computing task. For all operators marked under the same computing task among the multiple computing nodes to be taken offline, the transfer is triggered. The transfer is used to transfer the operators under the computing task to other computing nodes besides the computing nodes to be taken offline. In the transfer, at least one transfer of multiple operators is performed for the same computing task. After the operators in the plurality of computing nodes to be taken offline have been transferred to the other computing nodes, the plurality of computing nodes to be taken offline are taken offline.

2. The method according to claim 1, wherein, For all marked operators belonging to the same computational task, the transition is triggered as follows: A transition is triggered for all marked operators belonging to the same computational task, wherein the transition transfers all operators under the computational task to the other nodes; or, All marked operators belonging to the same computation task are grouped together, and a transition is triggered for each group. In this case, all operators in each group are transferred to the other nodes at once, and at least one group includes multiple operators.

3. The method according to claim 1, wherein, Obtaining the compute nodes to be taken offline includes: Obtain the name of the compute node that needs to be taken offline, input by the user, and designate the compute node corresponding to that name as the compute node to be taken offline; and / or, The user input request or the number of computing nodes that need to be taken offline is obtained, and the computing nodes to be taken offline are determined based on the request or the number of computing nodes that need to be taken offline; wherein, the request is used to indicate the amount of computing resources that need to be released, and the sum of the amount of computing resources occupied by the determined computing nodes to be taken offline satisfies the request.

4. The method according to claim 1, wherein, After obtaining the compute nodes to be taken offline, the method further includes: The compute nodes to be taken offline are marked as prohibited from scheduling, meaning that newly created operators will not be scheduled to compute nodes marked as prohibited from scheduling.

5. The method according to claim 4, wherein, Triggering a transfer for all operators belonging to the same computing task within the computing nodes to be taken offline includes: For all operators belonging to the same computing task among the computing nodes to be taken offline, a transfer is triggered to computing nodes that are not marked as prohibited from scheduling.

6. The method according to any one of claims 1 to 5, wherein, Obtaining the operators running on each of the compute nodes to be taken offline includes: The task management component in the computing task obtains the operators running on each computing node of the computing node to be taken offline. A computing task includes a work management component and at least one task management component. The task management component runs on the computing node that performs the computing task. The operators run inside the task management component. The work management component is used to manage at least one task management component.

7. The method according to claim 6, characterized in that, Obtaining the operators running on each of the compute nodes to be taken offline includes: Obtain the container with the task management component in the computing node to be taken offline, and identify the obtained container; Retrieve the operator from the task management component from the container with the aforementioned identifier.

8. The method according to any one of claims 1 to 5, wherein, The computing nodes to be taken offline are computing nodes in the same computing cluster, and the other computing nodes are computing nodes in the computing cluster other than the computing nodes to be taken offline. And / or, the computing nodes to be taken offline and the other computing nodes are used for streaming computing.

9. The method according to claim 8, wherein, The transfer is a fault transfer under streaming computing, which is one of the types of transfers. In the fault transfer, after the computing task is interrupted, the operator is transferred to other computing nodes.

10. A computing cluster system, wherein, The computing cluster system comprises multiple computing nodes, and when taking a computing node offline, the computing cluster system employs the method described in any one of claims 1 to 9.

11. An electronic device comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1 to 9.

12. A readable storage medium having computer instructions stored thereon, wherein, When executed by a processor, the computer instructions implement the steps of the method described in any one of claims 1 to 9.

Images