Methods and apparatus for scheduling stream processing tasks

Abstract

This invention discloses a method and apparatus for scheduling stream processing tasks. The method includes: receiving a task migration request initiated by a user; determining the task migration trigger time and task migration strategy based on the task migration request; acquiring each computing resource node involved in the task migration and controlling the coordinator responsible for task migration to schedule each computing resource node according to the task migration time and task migration strategy to execute the task migration; injecting a special delimiter event into the data stream of the stream processing job during the task migration process; using the special delimiter event to disconnect the task to be migrated from all its upstream and downstream tasks; and simultaneously deploying the task to be migrated to the computing resource node where the task needs to be migrated. This method can achieve the scheduling of stream processing tasks, solving the problems of long-term pauses in data stream processing caused by restarting jobs in existing technologies, resulting in significant performance fluctuations and instability.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a method and apparatus for scheduling stream processing tasks. Background Technology

[0002] In various industrial sectors, numerous tasks require data-intensive decision-making based on large amounts of diverse data. Big data processing can be broadly categorized into two types: batch processing and stream processing. Batch processing technology refers to taking a set of files as input and generating a new set of files as output. Batch processing relies on the crucial assumption that the input data is bounded, containing a finite amount of known information. However, in reality, data is often generated infinitely, such as sensor measurements, GPS signals, and financial transaction events. Since changes caused by the input only become apparent in the output an hour or a day later, batch processors must manually divide the data into blocks of fixed time intervals, such as processing data from the previous day or hour at the end of each day or hour. However, the value of the data itself diminishes over time, and excessive processing delays can lead to an inability to react promptly to data changes, resulting in delayed decision-making.

[0003] To address the problems in the aforementioned processing, stream processing technology can be used. Stream processing can begin processing whenever an event occurs, handling unbounded data continuously and incrementally. It allows for complete control of the data processing process within memory, ensuring low latency and making it suitable for large-scale data from various data sources. Stream processing task scheduling can be divided into static scheduling and dynamic scheduling. Dynamic scheduling focuses on already deployed and running stream processing jobs, addressing further optimization issues after the job's execution. Static scheduling often has limitations, and its actual performance cannot be predicted during the static scheduling phase. Therefore, subsequent dynamic scheduling is needed to optimize and adjust the stream processing job accordingly. Compared to static scheduling, dynamic scheduling provides additional runtime quality attribute information about the stream processing job, allowing for adjustments based on this information.

[0004] Currently, the main implementation mechanism for dynamic scheduling is based on job restarting. Due to the inherent characteristics of stream processing jobs, most dynamic scheduling methods require restarting and deploying the job when optimizing a running stream processing job. Once deployed, a stream processing job is in a continuous running state. When adjusting a running stream processing job, an elastic scaling mechanism based on job restarting can be used. Specifically, this involves manually stopping the stream processing job, restoring the job based on the savepoint created when the job was stopped, and re-specifying the parallelism of operators during restoration. In relevant versions, stream processing jobs will automatically scale up or down based on the increase or decrease of resources in the cluster. While restarting a job can allow for significant adjustments in operator parallelism, resources, and other dimensions with a single restart, it also inevitably pauses data stream processing for a long time, causing significant performance fluctuations and instability. Summary of the Invention

[0005] This invention provides a scheduling method and apparatus for stream processing tasks. By implementing a task migration mechanism, integrating with a stream processing engine, and designing a dynamic scheduling algorithm, the pause time of the data stream is minimized, ensuring the correctness and consistency of data processing. This solves the problem of long pauses in data stream processing caused by restarting jobs in existing technologies, resulting in significant performance fluctuations and instability. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a method for scheduling stream processing tasks, the method comprising:

[0007] Receive a task migration request initiated by a user, determine the task migration trigger time and task migration strategy based on the task migration request, and the task migration request carries the task to be migrated that meets the scheduling trigger conditions.

[0008] Obtain all computing resource nodes involved in the task migration, and control the coordinator responsible for the task migration to schedule each computing resource node according to the task migration time and task migration strategy to execute the task migration;

[0009] During the task migration process, a special delimiter event is injected into the data stream of the streaming job. The special delimiter event is used to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state. At the same time, the migrated task is deployed to the computing resource node to which the task needs to be migrated.

[0010] Optionally, determining the task migration trigger time and task migration strategy based on the task migration request includes:

[0011] According to the task migration request, monitor the operation indicators of the stream processing job. When abnormal indicators are detected in the stream processing job, extract the target stream processing task corresponding to the abnormal indicator.

[0012] If the target stream processing task meets the triggering conditions for dynamic scheduling, a task migration instruction is triggered, and the timing of the task migration trigger and the task migration strategy are determined based on the impact of the target stream processing task on the operation of the stream processing job before and after migration.

[0013] Optionally, determining the task migration trigger time and task migration strategy based on the impact of the target stream processing task on the operation of the stream processing job before and after migration includes:

[0014] Based on the changes in task traffic experienced during the operation of the stream processing job, peak periods of task traffic are selected to collect runtime data, and trough periods of task traffic are selected as the time for triggering task migration.

[0015] Based on the impact of the target stream processing task on the data transfer cost between tasks before and after migration, the migration cost is calculated for tasks on computing resource nodes with CPU utilization higher than a preset value, and the task with the lowest migration cost on the computing resource node is selected for migration.

[0016] Optionally, the step of calculating the migration cost for tasks on computing resource nodes with CPU utilization higher than a preset value based on the impact of the target stream processing task on the data transfer cost between tasks before and after migration, and selecting the task with the lowest migration cost on the computing resource node for migration, includes:

[0017] Based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration, the changes in the task communication cost between all upstream and downstream tasks of the task on the computing resource node after the current task changes its placement position are examined in turn. The communication cost is measured using the parameters defined in the static scheduling problem.

[0018] Based on the changes in the task communication cost, the sum of the communication costs of all upstream and downstream tasks is summarized to obtain the migration cost of the task on the computing resource node. The task with the lowest migration cost on the computing resource node is selected for migration.

[0019] Optionally, the method further includes:

[0020] For non-data transfer-intensive stream processing jobs, if the task graph in the stream processing job has a linear structure, then the chaining method is used to migrate tasks within the same storage unit to different storage units for execution. If a bottleneck task appears in the stream processing job, then the bottleneck task is migrated to a storage unit with fewer upstream operators deployed.

[0021] Optionally, the step of acquiring each computing resource node involved in the task migration and controlling the coordinator responsible for the task migration to schedule each computing resource node according to the task migration time and task migration strategy to execute the task migration includes:

[0022] Obtain the computing resource nodes where the migrated task is located, the computing resource nodes where all upstream and downstream tasks of the migrated task are located, and the computing resource nodes to which the task needs to be migrated.

[0023] A control command is sent to the coordinator responsible for task migration. Based on the control command, the migration plan is notified to each computing resource node according to the task migration time and task migration strategy, so that each computing resource node can execute the task migration according to the migration plan.

[0024] Optionally, after receiving the user-initiated task migration request, the method further includes:

[0025] If the task to be migrated is a single task, then according to the task migration request, the coordinator responsible for task migration will control the single task to perform the task migration process.

[0026] If the task to be migrated consists of multiple tasks, the coordinator responsible for the task migration will control the coordinator to migrate the multiple tasks sequentially in a serial manner, according to the task migration request that requires changing the current deployment location of each of the multiple tasks.

[0027] Optionally, the step of controlling the coordinator responsible for task migration to migrate the multiple tasks sequentially in a serial order, based on the task migration request requiring changes to the current deployment location of each of the multiple tasks, includes:

[0028] The task graph of the stream processing job is topologically sorted, and each task in the multiple tasks is determined to have a migration requirement according to the topological sort.

[0029] If a task requires migration, the coordinator responsible for task migration will be controlled to migrate the task based on the task's current deployment location.

[0030] Optionally, the step of injecting a special delimiter event into the data stream of the streaming job during task migration, using the special delimiter event to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state, and simultaneously deploying the migrated task to the computing resource node to which the task needs to be migrated, includes:

[0031] During the task migration process, a special delimiter event is injected into the data stream of the streaming job, and the computing resource nodes where all upstream tasks of the migrated task are located are notified to send the special delimiter event to the computing resource nodes of the migrated task. At the same time, the special delimiter event is broadcast to the computing resource nodes where all upstream tasks of the migrated task are located.

[0032] When the migrated task receives the special delimiter event sent by all upstream tasks, it confirms that the migrated task is in a migrateable state, deploys a new task on the computer node to which the task is migrated, and then broadcasts the special delimiter event to all downstream tasks of the migrated task, notifying all upstream and downstream computing resource nodes of the migrated task to establish a connection with the new task so as to send the data cached in the migrated task to the new task.

[0033] Secondly, embodiments of the present invention provide a scheduling method apparatus for stream processing tasks, the apparatus comprising:

[0034] The determining unit is used to receive a task migration request initiated by a user, and determine the task migration triggering time and task migration strategy according to the task migration request. The task migration request carries the task to be migrated that meets the scheduling triggering conditions.

[0035] The migration unit is used to acquire each computing resource node involved in the task migration and control the coordinator responsible for task migration to schedule each computing resource node according to the task migration time and task migration strategy to execute the task migration.

[0036] The deployment unit is used to inject a special delimiter event into the data stream of the streaming job during the task migration process, use the special delimiter event to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state, and deploy the migrated task to the computing resource node where the task needs to be migrated.

[0037] Optionally, the determining unit includes:

[0038] The monitoring module is used to monitor the operation indicators of the stream processing job according to the task migration request. When abnormal indicators are detected in the stream processing job, the target stream processing task corresponding to the abnormal indicators is extracted.

[0039] The determination module is used to determine if the target stream processing task meets the triggering conditions for dynamic scheduling, and then trigger a task migration instruction. Based on the impact of the target stream processing task on the operation of the stream processing job before and after migration, the module determines the time for triggering the task migration and the task migration strategy.

[0040] Optionally, the determining module is specifically used to select the peak period of task traffic to collect runtime data based on the changes in task traffic experienced during the operation of the stream processing job, and select the trough period of task traffic as the time for triggering task migration.

[0041] The determining module is further configured to calculate the migration cost of tasks on computing resource nodes with CPU utilization higher than a preset value based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration, and select the task with the lowest migration cost on the computing resource node for migration.

[0042] Optionally, the determining module is further configured to, based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration, sequentially check the changes in the task communication cost between all upstream and downstream tasks of the task on the computing resource node after the current task changes its placement position, wherein the communication cost is measured using parameters defined in the static scheduling problem.

[0043] The determining module is further configured to summarize the sum of communication costs of all upstream and downstream tasks based on the changes in the task communication cost, obtain the migration cost of the task on the computing resource node, and select the task with the minimum migration cost on the computing resource node for migration.

[0044] Optionally, the determining module is further configured to, for non-data-transfer-intensive stream processing jobs, if the task graph in the stream processing job is linear, use a chain-breaking method to migrate tasks within the same storage unit to different storage units for execution; if a bottleneck task appears in the stream processing job, the bottleneck task is migrated to a storage unit with fewer upstream operators deployed.

[0045] Optionally, the migration unit includes:

[0046] The acquisition module is used to acquire the computing resource nodes where the task to be migrated is located, the computing resource nodes where all upstream and downstream tasks of the task to be migrated are located, and the computing resource nodes to which the task needs to be migrated.

[0047] The sending module is used to send control commands to the coordinator responsible for task migration, and to notify each computing resource node of the migration plan according to the task migration time and task migration strategy based on the control commands, so that each computing resource node can perform task migration according to the migration plan.

[0048] Optionally, the device further includes:

[0049] The control unit is configured to, after receiving a task migration request initiated by a user, if the task to be migrated is a single task, control the coordinator responsible for task migration to perform the task migration process on the single task according to the task migration request to change the current deployment location of the single task.

[0050] The control unit is further configured to, if the task to be migrated consists of multiple tasks, control the coordinator responsible for task migration to migrate the multiple tasks sequentially in a serial arrangement, according to the task migration request to change the current deployment location of each of the multiple tasks.

[0051] Optionally, the control unit includes:

[0052] The judgment module is used to perform topological sorting on the task graph of the stream processing job, and to determine whether each task in the multiple tasks has a migration requirement according to the topological sorting.

[0053] The control module is used to control the coordinator responsible for task migration to migrate the task based on the task's current deployment location if the task has a migration requirement.

[0054] Optionally, the deployment unit includes:

[0055] The injection module is used to inject special delimiter events into the data stream of the streaming job during the task migration process, and to notify all upstream task computing resource nodes of the migrated task to send the special delimiter events to the computing resource nodes of the migrated task, and at the same time broadcast the special delimiter events to all upstream task computing resource nodes of the migrated task.

[0056] The deployment module is used to confirm that the migrated task is in a migrateable state when the migrated task receives the special delimiter event sent by all upstream tasks, and to deploy a new task on the computer node to which the task is migrated. Then, it broadcasts the special delimiter event to all downstream tasks of the migrated task, notifying all upstream and downstream computing resource nodes of the migrated task to establish a connection with the new task so as to send the data cached in the migrated task to the new task.

[0057] Thirdly, embodiments of the present invention provide a storage medium storing executable instructions thereon, which, when executed by a processor, cause the processor to implement the method described in the first aspect.

[0058] Fourthly, embodiments of the present invention provide an apparatus for scheduling a stream processing task, comprising:

[0059] One or more processors;

[0060] Storage device for storing one or more programs.

[0061] Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect.

[0062] As can be seen from the above, the scheduling method and apparatus for stream processing tasks provided in this embodiment of the invention receive a task migration request initiated by a user, determine the task migration trigger time and task migration strategy based on the task migration request, the task migration request carries the task to be migrated that meets the scheduling trigger conditions, obtain each computing resource node involved in the task migration, and control the coordinator responsible for task migration to schedule each computing resource node according to the task migration time and task migration strategy to execute the task migration, and further inject a special delimiter event into the data stream of the stream processing job during the task migration process, using the special delimiter event to disconnect the task to be migrated from all its upstream and downstream tasks, in order to confirm that the task to be migrated is being migrated. The migration task is in a migrateable state and is deployed to the computing resource node where the task needs to be migrated. Compared with the existing technology that uses an elastic scaling mechanism based on restarting the job to adjust the running stream processing job, this embodiment of the invention injects a special delimiter event into the data stream of the stream processing job during the task migration process. This special delimiter event can disconnect the migrated task from all its upstream and downstream tasks, so that the task migration can change the deployment position of the task without restarting the stream processing job. This minimizes the pause time of the data stream and reduces the impact on the job and data stream, while ensuring the correctness and consistency of data processing.

[0063] Furthermore, the technical effects that this embodiment can also achieve include:

[0064] (1) Dynamic scheduling can realize the migration of stream processing tasks. The task migration process can be completed quickly, so as not to have a significant impact on the running stream processing jobs.

[0065] (2) Dynamic scheduling is triggered at the right time and under the right conditions, and the adjusted job can make fuller use of the resources in the cluster, thereby improving performance.

[0066] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0068] Figure 1 A flowchart illustrating a scheduling method for stream processing tasks provided in an embodiment of the present invention;

[0069] Figure 2 This is a schematic diagram illustrating the placement schemes of different tasks before and after chain disassembly, as provided in an embodiment of the present invention.

[0070] Figure 3 This is a schematic diagram of a bottleneck task migration processing scheme provided in an embodiment of the present invention;

[0071] Figure 4 This is a schematic diagram illustrating the interaction relationships between various components after the task migration function provided in this embodiment of the invention is integrated into the processing engine;

[0072] Figure 5 This is a timing diagram of single-task migration provided in an embodiment of the present invention;

[0073] Figure 6 A schematic diagram illustrating the interaction flow of various events during the task migration process provided in this embodiment of the invention;

[0074] Figure 7 This is a block diagram of a scheduling method apparatus for stream processing tasks provided in an embodiment of the present invention. Detailed Implementation

[0075] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0076] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0077] This invention provides a scheduling method and apparatus for stream processing tasks. By implementing a task migration mechanism, integrating it with a stream processing engine, and designing a dynamic scheduling algorithm, the pause time of the data stream is minimized, ensuring the correctness and consistency of data processing. This solves the problem of long pauses in data stream processing caused by restarting jobs in existing technologies, resulting in significant performance fluctuations and instability. Currently, most dynamic scheduling solutions require restarting and deploying the stream processing job when adjusting it. Restarting a job introduces a stop time of several minutes, during which data arriving cannot be processed immediately. To minimize the pause time of the data stream in dynamic scheduling, this invention employs a dynamic scheduling strategy based on a task migration mechanism. This task migration mechanism is highly efficient and ensures the correctness and consistency of data processing. The embodiments of this invention are described in detail below.

[0078] Figure 1 This is a flowchart illustrating a scheduling method for stream processing tasks provided by an embodiment of the present invention. The method may include the following steps:

[0079] S100: Receive a task migration request initiated by the user, and determine the task migration trigger time and task migration strategy based on the task migration request.

[0080] In this context, the task migration request carries the migrated task that meets the scheduling triggering conditions. The first thing to consider in the dynamic scheduling problem of stream processing jobs is the triggering conditions of dynamic scheduling. Once dynamic scheduling is triggered, it will affect the running stream processing job. Even if the method of migrating tasks without restarting the job is used, the processing flow of the migrated task and its downstream tasks needs to be paused for a short period of time. If the triggering conditions for dynamic scheduling are judged to be too low, the job will be scheduled frequently. At this time, the scheduling itself will have a significant impact on the job. However, if the conditions are too high, it may be difficult to trigger dynamic scheduling and the goal of optimizing the stream processing job cannot be achieved.

[0081] Understandably, considering that dynamic scheduling is not always necessary during stream processing, we can monitor the stream processing job and determine whether tasks within it meet the dynamic scheduling triggering conditions when anomalies are detected. Specifically, we can monitor the stream processing job's operational metrics based on task migration requests. When anomalies are detected, we extract the target stream processing task corresponding to the anomaly. If the target task meets the dynamic scheduling triggering conditions, we trigger a task migration instruction. Furthermore, based on the impact of the target task on the stream processing job's operation before and after migration, we determine the timing and strategy for task migration.

[0082] Since abnormal metrics typically affect the normal operation of stream processing jobs—for example, reducing efficiency or consuming cluster resources—dynamic scheduling can be triggered to utilize scheduling resources more efficiently. This can be achieved by considering at least two necessary conditions: Condition 1 indicates backpressure on a stream processing task, and Condition 2 indicates that CPU utilization on a node in the cluster exceeds a specified threshold. Condition 1 indicates a bottleneck in the stream processing job, requiring adjustments to the backpressure-affected task. Condition 2 indicates uneven resource utilization within the cluster, necessitating migration of tasks from nodes with high CPU utilization to nodes with lower utilization to balance overall cluster resource utilization. Conditions 1 and 2 are not contradictory; nodes experiencing backpressure often exhibit excessively high CPU utilization. If no nodes with low CPU utilization exist in the cluster, additional resources may need to be added. It is important to note that both of the above conditions are necessary. Even if one of the two conditions is met, dynamic scheduling may not be triggered. For example, if the node where the task with backpressure is located or the node with high CPU utilization only has a single task deployed, dynamic scheduling will not be triggered. In this case, it is necessary to increase the processing capacity of the individual resource.

[0083] Furthermore, after meeting the scheduling trigger conditions, the timing of task migration needs to be considered. Since task migration requires pausing data transmission for a short period and caching the data in the upstream task of the migrated task, if the data input and transmission rates on the migrated task are too high at any given moment, it is not suitable to perform task migration. Forcing task migration would cause significant fluctuations in the throughput of the stream processing job, and could even lead to the crash of some tasks. Specifically, runtime data can be collected during peak task traffic periods and during low task traffic periods, based on the changes in task traffic experienced during the stream processing job's operation. In real-world scenarios, stream processing jobs often experience peak and low traffic periods, and these traffic changes typically follow a certain pattern. Therefore, a dynamic scheduling mechanism can collect runtime data during peak traffic periods to determine if the scheduling trigger conditions are met. If the conditions are met, the corresponding task migration operation is triggered during low traffic periods to minimize the impact of task migration on the job.

[0084] Furthermore, after the scheduling triggering conditions are met, a task migration strategy needs to be considered, namely, which task to migrate to a resource with lower CPU utilization. The factor considered here is the impact on data transfer costs between tasks before and after migration. The migration cost is calculated for all tasks deployed on nodes with high CPU utilization, and the task with the lowest migration cost is ultimately selected for migration. Specifically, based on the impact of the target stream processing task on data transfer costs between tasks before and after migration, the migration cost of tasks on computing resource nodes with CPU utilization higher than a preset value can be calculated, and the task with the lowest migration cost on the computing resource node can be selected for migration. The migration cost calculation process is as follows: based on the impact of the target stream processing task on data transfer costs between tasks before and after migration, the changes in communication costs between all upstream and downstream tasks on the computing resource node after the current task's placement position is changed are checked sequentially. This communication cost is measured using parameters defined in the static scheduling problem. Furthermore, based on the changes in task communication costs, the sum of the communication costs of all upstream and downstream tasks is summarized to obtain the migration cost of the task on the computing resource node, and the task with the lowest migration cost on the computing resource node is selected for migration.

[0085] It's important to note that, to ensure the job runs as smoothly as possible, the task migration strategy will only migrate one task at a time. Migrating multiple tasks during a single dynamic scheduling process will not only significantly impact the stream processing job but also introduce additional complexity. Therefore, the job's performance after task migration may not necessarily be better than before. In actual operation, after migrating a single task and ensuring the stream processing job is stable, it's advisable to reassess whether further task migration is necessary.

[0086] Furthermore, the task migration strategy can be flexibly adjusted for different types of stream processing jobs. If the amount of data transferred between different operators in a stream processing job is small, there is no need to compare communication costs. Instead, the task with the highest computational complexity on the node with the highest CPU utilization can be directly selected for migration.

[0087] It should be noted that the above task migration strategy has strong versatility and a wide range of applications. However, since it only considers the impact of communication costs on tasks when selecting tasks to be migrated, it cannot provide targeted optimization for non-data-transfer-intensive stream processing jobs. This embodiment of the invention can also, for non-data-transfer-intensive stream processing jobs, if the task graph in the stream processing job has a linear structure, use a chain-breaking approach to migrate tasks within the same storage unit to different storage units for execution. If a bottleneck task appears in the stream processing job, the bottleneck task is migrated to a storage unit with fewer upstream operators deployed.

[0088] Specifically, linear task graphs are a common deployment pattern in stream processing jobs. All tasks in the task graph are arranged in a chain, and except for the data source and data sink, all other tasks have only one upstream task and one downstream task. For such jobs, the default scheduling strategy of the processing engine will schedule all tasks to run in the same storage unit. Since they are all located on the same computing resource node, the data transmission overhead between tasks can be effectively reduced. However, for long-chained task graphs and CPU-intensive tasks, scheduling them to the same storage unit will overload that storage unit, which is not conducive to the effective utilization of resources in the cluster. This invention addresses this by using a chain-breaking method to migrate the corresponding tasks. For example, as shown... Figure 2 As shown, Figure 2 This demonstrates the placement of different tasks before and after chaining. In this case, the first half of the job is located in one storage unit, while the second half is located in another. This allows for more efficient use of CPU resources, while only increasing the communication cost of the middle part. Since machines within the same cluster are typically close together, this has little impact on non-communication-intensive tasks. For longer chained jobs, they can be broken down into more parts, with each part scheduled to a different storage unit.

[0089] In real-world processing jobs, it's common to encounter situations where a particular operator in the job graph can only have a parallelism of 1, while its upstream and downstream operators can have high parallelism. In this case, the task corresponding to the operator with a parallelism of 1 will have multiple upstream and downstream tasks. It needs to receive data from all upstream tasks, process it, and then send it sequentially to all downstream tasks. Due to its excessive processing volume, it can easily become a bottleneck in the job. The default scheduling strategy of the processing engine does not consider this situation and instead schedules the entire task chain to the same storage unit, thus preventing the bottleneck task from receiving sufficient resources. This invention can migrate the bottleneck task to a storage unit with fewer upstream operators deployed. For example, such as... Figure 3 As shown, Figure 3 The migration solution for bottleneck tasks is demonstrated. Before migration, the third task in storage unit 2 is the bottleneck task. In actual processing, the bottleneck task can be migrated to storage unit 3, which has fewer upstream operators deployed. The migration does not change the overall communication cost, but it can make resource utilization more balanced. If the CPU computational complexity of the bottleneck task is high, it can be further considered to migrate it to a new storage unit.

[0090] S110: Obtain the various computing resource nodes involved in the task migration, and control the coordinator responsible for the task migration to schedule the various computing resource nodes according to the task migration time and task migration strategy to execute the task migration.

[0091] To ensure efficient task migration, the pause time in the data stream must be minimized while maintaining data processing accuracy and consistency. The task migration process analyzes the metrics of running jobs and provides targeted migration solutions to optimize key quality attributes of the running stream processing jobs and improve cluster resource utilization. Specifically, it obtains the computing resource nodes where the migrated task resides, the computing resource nodes of all upstream and downstream tasks of the migrated task, and the computing resource nodes to which the task needs to be migrated. Control commands are then sent to the coordinator responsible for task migration. Based on these commands, the migration plan is notified to each computing resource node according to the task migration time and strategy, enabling each computing resource node to execute the task migration according to the plan.

[0092] In practical applications, the entire task migration process can be integrated with the processing engine, taking into account the existing components and functionalities within the processing engine. Figure 4 This demonstrates the interaction between various components after the task migration function is integrated into the processing engine. Here, the task migration coordinator is implemented as a new component in JobMaster, responsible for coordinating the entire task migration process. In the specific implementation, the coordinator is an independent thread that periodically checks for new task migration requests. If a new task migration request is detected, the task migration process begins.

[0093] The coordinator and TaskManager interact via RPC. To enable task migration, both JobManager and TaskManager have added new remote call function interfaces for transmitting data and control information. TaskManager provides the JobManager with interfaces to send migration plans, notify upstream tasks to continue sending data, and notify downstream tasks to reconnect to the migrated task. JobManager provides the TaskManager with interfaces to send task status and notify that special events have been received.

[0094] The coordinator also interacts with existing components in JobManager and JobMaster. After creating the migration task, the coordinator notifies the scheduler to deploy the task to the corresponding storage unit. If the storage unit is already in the JobMaster's storage unit pool, the scheduler will directly deploy it to the corresponding TaskManager's storage unit. If the storage unit is not in the JobMaster's storage unit pool, the coordinator will first request resources from the resource manager. After the resource manager notifies TaskManager to allocate the storage unit to the JobMaster, the scheduler can deploy the new task.

[0095] S120: During the task migration process, a special delimiter event is injected into the data stream of the stream processing job. The special delimiter event is used to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state. At the same time, the migrated task is deployed to the computing resource node to which the task needs to be migrated.

[0096] In this embodiment of the invention, the task to be migrated can be a single task or multiple tasks. If the task to be migrated is a single task, the current deployment location of the single task needs to be changed according to the task migration request, and the coordinator responsible for task migration is controlled to perform the task migration process on the single task. If the task to be migrated is multiple tasks, the current deployment location of each of the multiple tasks needs to be changed according to the task migration request, and the coordinator responsible for task migration is controlled to migrate the multiple tasks in a serial arrangement. Specifically, during task migration, a special delimiter event can be injected into the data stream of the stream processing job, and the computing resource nodes where all upstream tasks of the migrated task reside can be notified to send the special delimiter event to the computing resource nodes of the migrated task. At the same time, the special delimiter event is broadcast to the computing resource nodes where all upstream tasks of the migrated task reside. When the computing resource nodes where all upstream tasks of the migrated task reside receive the special delimiter event, they confirm that the migrated task is in a migrateable state, deploy the new task on the computer node to which the task is migrated, and then broadcast the special delimiter event to all downstream tasks of the migrated task, notifying the computing resource nodes where all upstream and downstream tasks of the migrated task reside to establish a connection with the new task so that the data cached in the migrated task can be sent to the new task.

[0097] Understandably, the task migration process here is user-initiated; specifically, Figure 5 The sequence diagram for a single-task migration is shown. The overall process includes the following steps:

[0098] Step S1: User requests task migration.

[0099] Step S2: The coordinator notifies all upstream task nodes of the migrated task that all upstream tasks send a special delimiter event to the migrated task (a single upstream task may have multiple downstream tasks including the migrated task, but here only the special delimiter event is sent to the migrated task). After sending the special delimiter event, all data events arriving at the upstream task are cached.

[0100] Step S3: After receiving the special delimiter events sent by all upstream tasks, the migrated task sends its current status to the coordinator and broadcasts the special delimiter events to all downstream tasks of that task.

[0101] Step S4: After receiving the special delimiter event, the downstream task notifies the coordinator that it has received the special delimiter event sent by the migrated task.

[0102] Step S5: After confirming the status of the task to be migrated, the coordinator creates a new task and deploys it on the node to which it needs to be migrated. The initial status of the task is set to the status snapshot of the task sent to the coordinator.

[0103] Step S6: After the coordinator waits for the new task to be deployed and confirms receipt of the special delimiter event notification from all downstream tasks, it notifies all downstream nodes of the migrated task to establish a connection with the new task.

[0104] Step S7: The coordinator notifies all upstream task nodes of the migrated task to resume data transmission and sends the data cached during the migration process and subsequent data to the new task.

[0105] Step S8: The coordinator will reclaim the resources in the migrated task nodes, and the task migration will be completed.

[0106] It should be noted that, in order to ensure the correctness of single-task migration, the following points need to be observed in some key steps of the migration process: In step S3, the upstream task must ensure that all subsequent data arrives after sending the special delimiter event; in step S4, the migrated task node must ensure that after receiving the special delimiter events sent by all upstream tasks, it sends its current status to the coordinator and broadcasts the special delimiter events to downstream tasks; in step S6, the coordinator must ensure that after receiving the status of the migrated task, it creates a new task based on the status; in step S7, the coordinator must ensure that the task deployment is complete and confirms that it has received notifications from all downstream tasks of the migrated task before establishing a connection between the downstream tasks and the new task.

[0107] Furthermore, stream processing jobs often contain multiple tasks. A user might request changes to the current deployment locations of multiple tasks in a single request. Therefore, it's necessary to consider how to migrate multiple tasks, i.e., the migration process for multiple tasks. This migration can be composed of multiple single-task migration processes. Specifically, the task graph of the stream processing job can be topologically sorted. Based on this sorted order, each task can be assessed for migration requirements. If a task requires migration, the coordinator responsible for task migration will be controlled to perform the migration based on its current deployment location. In practical applications, multiple tasks can be migrated sequentially. First, the task graph of the stream processing job is topologically sorted. Based on this topological order, each task is assessed for migration requirements. If a task requires migration, it is migrated using a single-task migration method. This topological sorting of the task graph prioritizes tasks closer to the data source and ensures that all upstream tasks have successfully migrated by the time the current task is migrated, thus making the overall migration process of the stream processing job more stable.

[0108] It should be noted that various types of data and events are involved in the task migration process, including control events transmitted between the coordinator and compute resource nodes, as well as events contained in the data stream. Their types and specific descriptions are as follows:

[0109] The task migration plan, which controls the event, contains the necessary information for migrating a single task. It is sent by the coordinator to the node containing the task to be migrated, as well as the nodes containing all upstream tasks of the task to be migrated. The task migration process officially begins after the coordinator sends the migration plan.

[0110] State snapshot of control events: Sent to the coordinator by the migrated task, the content is a snapshot of the local state of the migrated task before migration. The coordinator will create new tasks and deploy them based on this snapshot to ensure that the state of the task will not be changed after migration.

[0111] Reconnectable notification for control events: Sent from the downstream task of the migrated task to the coordinator to notify the coordinator that the task can establish a connection with the new migrated task.

[0112] Ordinary data events in a data stream: These are events transmitted in the normal data stream and that need to be processed by tasks. In order not to affect the exact-once streaming semantics, these events must neither be duplicated nor lost during processing.

[0113] Special delimiter events for data stream events: These events are defined to better control task migrations and are injected into the data stream, similar in mechanism and function to checkpoint delimiters in stream processing frameworks. The special delimiter logically divides the data stream into two parts: all ordinary data preceding the delimiter is processed before task migration, and all ordinary data following the delimiter is processed after task migration.

[0114] For example, Figure 6 The diagram illustrates the interaction flow of various events during task migration. In Figure (a), the coordinator notifies the migrated task and its upstream tasks of the migration plan. After receiving the migration plan, the upstream task begins injecting special delimiter events into the data stream, indicating that it has learned of the plan and will no longer send ordinary data events to the migrated task. In Figure (b), the upstream task has cached the ordinary data events that arrive after the delimiter in the buffer. While the migrated task continues to process ordinary data events, it receives the special delimiter sent by upstream task 1, learning that upstream task 1 has stopped sending ordinary data. At this point, it can be considered that the logical connection between the migrated task and upstream task 1 has been broken. In Figure (c), the migrated task... The migration task receives special delimiters from all upstream tasks. Since it can be guaranteed that no ordinary data events will be received subsequently, a state snapshot can be sent to the coordinator. At the same time, as shown in Figure (d), the migrated task broadcasts special delimiter events to downstream tasks. In Figure (e), downstream task 2 receives the special delimiter event first, which can be considered as having disconnected from the migrated task. It will also notify the coordinator that it will no longer receive data and can establish a connection with a new task at any time. In Figure (f), all downstream tasks receive special delimiter events and notify the coordinator. Logically, it can be considered that the migrated task has disconnected from all its upstream and downstream tasks. The coordinator can safely create new tasks and cancel old tasks.

[0115] Understandably, before and after task migration, ordinary data events in the data stream will not be duplicated or lost, and can be correctly reflected in the task state, ensuring the correctness of the task migration process and not destroying the exact-once stream processing semantics.

[0116] The streaming task scheduling method provided in this embodiment of the invention receives a task migration request initiated by a user, determines the task migration trigger time and task migration strategy based on the task migration request, the task migration request carries the task to be migrated that meets the scheduling trigger conditions, obtains each computing resource node involved in the task migration, and controls the coordinator responsible for task migration to schedule each computing resource node according to the task migration time and task migration strategy to execute the task migration, and further injects a special delimiter event into the data stream of the streaming job during the task migration process, using the special delimiter event to disconnect the migrated task from all its upstream and downstream tasks, in order to confirm that the migrated task is in a state of flux. The migrated task is deployed to the computing resource node where it needs to be migrated. Compared with the existing technology that uses an elastic scaling mechanism based on restarting the job to adjust the running stream processing job, this embodiment of the invention injects a special delimiter event into the data stream of the stream processing job during the task migration process. This special delimiter event can disconnect the migrated task from all its upstream and downstream tasks, so that the task migration can change the deployment position of the task without restarting the stream processing job. This minimizes the pause time of the data stream and reduces the impact on the job and the data stream, while ensuring the correctness and consistency of data processing.

[0117] Furthermore, after task migration, the process can be evaluated. This evaluation includes two aspects: first, assessing the actual performance of the task migration mechanism and its impact on running stream processing jobs; and second, evaluating the effectiveness of the dynamic scheduling strategy. Specifically, the evaluation process involves using software tools within the processing engine to monitor the runtime metrics of the stream processing jobs. The processing engine can push the monitored metrics to the software tools every 5 seconds to display the latest data in the browsing interface. The monitored metrics primarily include CPU utilization and throughput.

[0118] Based on the above embodiments, another embodiment of the present invention provides a scheduling method apparatus for stream processing tasks, such as... Figure 7 As shown, the device includes:

[0119] The determining unit 20 can be used to receive a task migration request initiated by a user, and determine the task migration triggering time and task migration strategy according to the task migration request. The task migration request carries the task to be migrated that meets the scheduling triggering conditions.

[0120] The migration unit 22 can be used to obtain each computing resource node involved in the task migration and control the coordinator responsible for task migration to schedule each computing resource node according to the task migration time and task migration strategy to perform task migration.

[0121] Deployment unit 24 can be used to inject a special delimiter event into the data stream of the stream processing job during the task migration process, use the special delimiter event to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state, and deploy the migrated task to the computing resource node to which the task needs to be migrated.

[0122] In specific application scenarios, the determining unit 20 includes:

[0123] The monitoring module can be used to monitor the operation indicators of the stream processing job according to the task migration request. When abnormal indicators are detected in the stream processing job, the target stream processing task corresponding to the abnormal indicators is extracted.

[0124] The determination module can be used to determine if the target stream processing task meets the triggering conditions for dynamic scheduling, and then trigger a task migration instruction. Based on the impact of the target stream processing task on the operation of the stream processing job before and after migration, the module can determine the time for triggering the task migration and the task migration strategy.

[0125] In specific application scenarios, the determining module can be used to select the peak period of task traffic to collect runtime data based on the changes in task traffic experienced during the operation of the stream processing job, and select the trough period of task traffic as the time for triggering task migration.

[0126] The determining module can also be used to calculate the migration cost of tasks on computing resource nodes with CPU utilization higher than a preset value based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration, and select the task with the lowest migration cost on the computing resource node for migration.

[0127] In specific application scenarios, the determining module can also be used to check the changes in the communication cost between all upstream and downstream tasks of the task on the computing resource node after the current task changes its placement position, based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration. The communication cost is measured using parameters defined in the static scheduling problem.

[0128] Specifically, the determining module can also be used to summarize the sum of communication costs of all upstream and downstream tasks based on the changes in the task communication cost, obtain the migration cost of the task on the computing resource node, and select the task with the minimum migration cost on the computing resource node for migration.

[0129] In specific application scenarios, the determining module can also be used for non-data-transfer-intensive stream processing jobs. If the task graph in the stream processing job has a linear structure, the task in the same storage unit is migrated to different storage units for execution by chaining. If a bottleneck task appears in the stream processing job, the bottleneck task is migrated to a storage unit with fewer upstream operators deployed.

[0130] In specific application scenarios, the migration unit 22 includes:

[0131] The acquisition module can be used to acquire the computing resource nodes where the migrated task is located, the computing resource nodes where all upstream and downstream tasks of the migrated task are located, and the computing resource nodes to which the task needs to be migrated.

[0132] The sending module can be used to send control commands to the coordinator responsible for task migration, and notify each computing resource node of the migration plan according to the task migration time and task migration strategy based on the control commands, so that each computing resource node can execute the task migration according to the migration plan.

[0133] In specific application scenarios, the device further includes:

[0134] The control unit can be used, after receiving a task migration request initiated by a user, if the task to be migrated is a single task, to control the coordinator responsible for task migration to perform the task migration process on the single task according to the task migration request to change the current deployment location of the single task.

[0135] The control unit can also be used to control the coordinator responsible for task migration to migrate the multiple tasks sequentially in a serial manner, if the task to be migrated consists of multiple tasks, according to the task migration request, to change the current deployment location of each of the multiple tasks.

[0136] In specific application scenarios, the control unit includes:

[0137] The judgment module can be used to perform topological sorting on the task graph of the stream processing job, and to determine whether each task in multiple tasks has a migration requirement according to the topological sorting.

[0138] The control module can be used to control the coordinator responsible for task migration to migrate the task based on the task's current deployment location if the task has migration requirements.

[0139] In specific application scenarios, the deployment unit 24 includes:

[0140] The injection module can be used to inject special delimiter events into the data stream of the streaming job during the task migration process, and notify all upstream task computing resource nodes of the migrated task to send the special delimiter events to the computing resource nodes of the migrated task, and at the same time broadcast the special delimiter events to all upstream task computing resource nodes of the migrated task.

[0141] The deployment module can be used to confirm that the migrated task is in a migrateable state when the migrated task receives the special delimiter event sent by all upstream tasks, and deploy a new task on the computer node to which the task is migrated. Then, it broadcasts the special delimiter event to all downstream tasks of the migrated task, notifying the computing resource nodes where all upstream and downstream tasks of the migrated task are located to establish a connection with the new task so as to send the data cached in the migrated task to the new task.

[0142] Based on the above method embodiments, another embodiment of the present invention provides a storage medium storing executable instructions thereon, which, when executed by a processor, cause the processor to implement the above method.

[0143] Based on the above embodiments, another embodiment of the present invention provides a method for scheduling stream processing tasks, including:

[0144] One or more processors;

[0145] Storage device for storing one or more programs.

[0146] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method.

[0147] The above-described apparatus embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The apparatus embodiments are derived from the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0148] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A scheduling method for stream processing tasks, characterized in that, The method includes: Receive a task migration request initiated by a user, determine the task migration trigger time and task migration strategy based on the task migration request, and the task migration request carries the task to be migrated that meets the scheduling trigger conditions. Obtain all computing resource nodes involved in the task migration, and control the coordinator responsible for the task migration to schedule each computing resource node according to the task migration trigger time and task migration strategy to execute the task migration; During the task migration process, a special delimiter event is injected into the data stream of the streaming job. The special delimiter event is used to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state. At the same time, the migrated task is deployed to the computing resource node to which the task needs to be migrated. The step of determining the task migration trigger time and task migration strategy based on the task migration request includes: According to the task migration request, monitor the operation indicators of the stream processing job. When abnormal indicators are detected in the stream processing job, extract the target stream processing task corresponding to the abnormal indicator. If the target stream processing task meets the triggering conditions for dynamic scheduling, a task migration instruction is triggered. Based on the changes in task traffic experienced during the operation of the stream processing job, runtime data is collected during the peak period of task traffic and the trough period of task traffic is selected as the time for triggering task migration. Based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration, the migration cost is calculated for tasks on computing resource nodes with CPU utilization higher than a preset value, and the task with the lowest migration cost on the computing resource node is selected for migration. The process of injecting a special delimiter event into the data stream of the streaming job during task migration, using the special delimiter event to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state, and simultaneously deploying the migrated task to the computing resource node to which the task needs to be migrated, includes: During the task migration process, a special delimiter event is injected into the data stream of the streaming job, and the computing resource nodes where all upstream tasks of the migrated task reside are notified to send the special delimiter event to the computing resource nodes of the migrated task. When the migrated task receives the special delimiter event sent by all upstream tasks, it confirms that the migrated task is in a migrateable state, deploys a new task on the computer node to which the task is migrated, and then broadcasts the special delimiter event to all downstream tasks of the migrated task, notifying all upstream and downstream computing resource nodes of the migrated task to establish a connection with the new task so as to send the data cached in the migrated task to the new task.

2. The method as described in claim 1, characterized in that, The step involves calculating the migration cost of tasks on computing resource nodes with CPU utilization exceeding a preset value based on the impact of the target stream processing task on data transfer costs between tasks before and after migration, and selecting the task with the lowest migration cost on the computing resource node for migration, including: Based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration, the changes in the task communication cost between all upstream and downstream tasks of the task on the computing resource node after the current task changes its placement position are examined in turn. The communication cost is measured using the parameters defined in the static scheduling problem. Based on the changes in the task communication cost, the sum of the communication costs of all upstream and downstream tasks is summarized to obtain the migration cost of the task on the computing resource node. The task with the lowest migration cost on the computing resource node is selected for migration.

3. The method as described in claim 1, characterized in that, The method further includes: For non-data transfer-intensive stream processing jobs, if the task graph in the stream processing job has a linear structure, then the chaining method is used to migrate tasks within the same storage unit to different storage units for execution. If a bottleneck task appears in the stream processing job, then the bottleneck task is migrated to a storage unit with fewer upstream operators deployed.

4. The method as described in claim 1, characterized in that, The process of acquiring the various computing resource nodes involved in the task migration and controlling the coordinator responsible for the task migration to schedule the various computing resource nodes according to the task migration trigger time and task migration strategy to execute the task migration includes: Obtain the computing resource nodes where the migrated task is located, the computing resource nodes where all upstream and downstream tasks of the migrated task are located, and the computing resource nodes to which the task needs to be migrated. A control command is sent to the coordinator responsible for task migration. Based on the control command and the task migration trigger time and strategy, the migration plan is notified to each computing resource node so that each computing resource node can execute the task migration according to the migration plan.

5. The method as described in claim 1, characterized in that, After receiving the user-initiated task migration request, the method further includes: If the task to be migrated is a single task, then according to the task migration request, the coordinator responsible for task migration will control the single task to perform the task migration process. If the task to be migrated consists of multiple tasks, the coordinator responsible for the task migration will control the coordinator to migrate the multiple tasks sequentially in a serial manner, according to the task migration request that requires changing the current deployment location of each of the multiple tasks.

6. The method as described in claim 5, characterized in that, The step of controlling the coordinator responsible for task migration to migrate the multiple tasks sequentially in a serial order, based on the task migration request and the need to change the current deployment location of each of the multiple tasks, includes: The task graph of the stream processing job is topologically sorted, and each task in the multiple tasks is determined to have a migration requirement according to the topological sort. If a task requires migration, the coordinator responsible for task migration will be controlled to migrate the task based on the task's current deployment location.

7. A scheduling device for stream processing tasks, characterized in that, The device includes: The determining unit is used to receive a task migration request initiated by a user, and determine the task migration triggering time and task migration strategy according to the task migration request. The task migration request carries the task to be migrated that meets the scheduling triggering conditions. The migration unit is used to acquire each computing resource node involved in the task migration and control the coordinator responsible for task migration to schedule each computing resource node according to the task migration trigger time and task migration strategy to execute the task migration. The deployment unit is used to inject a special delimiter event into the data stream of the streaming job during the task migration process, use the special delimiter event to disconnect the migrated task from all its upstream and downstream tasks to confirm that the migrated task is in a migrateable state, and deploy the migrated task to the computing resource node where the task needs to be migrated. The determining unit includes: The monitoring module is used to monitor the operation indicators of the stream processing job according to the task migration request. When abnormal indicators are detected in the stream processing job, the target stream processing task corresponding to the abnormal indicators is extracted. The determination module is used to determine if the target stream processing task meets the triggering conditions for dynamic scheduling, and then trigger a task migration instruction. Based on the impact of the target stream processing task on the operation of the stream processing job before and after migration, the module determines the time for triggering the task migration and the task migration strategy. The determining module is specifically used to select the peak period of task traffic to collect runtime data based on the changes in task traffic experienced during the operation of the stream processing job, and select the trough period of task traffic as the time for triggering task migration. The determining module is further configured to calculate the migration cost of tasks on computing resource nodes with CPU utilization higher than a preset value based on the impact of the target stream processing task on the data transmission cost between tasks before and after migration, and select the task with the lowest migration cost on the computing resource node for migration. The deployment unit includes: The injection module is used to inject special delimiter events into the data stream of the streaming job during the task migration process, and to notify all upstream task computing resource nodes of the migrated task to send the special delimiter events to the computing resource nodes of the migrated task, and at the same time broadcast the special delimiter events to all upstream task computing resource nodes of the migrated task. The deployment module is used to confirm that the migrated task is in a migrateable state when the migrated task receives the special delimiter event sent by all upstream tasks, and to deploy a new task on the computer node to which the task is migrated. Then, it broadcasts the special delimiter event to all downstream tasks of the migrated task, notifying all upstream and downstream computing resource nodes of the migrated task to establish a connection with the new task so as to send the data cached in the migrated task to the new task.

Images