Task migration method, scheduler, electronic device, and readable storage medium

Abstract

Embodiments of the present application provide a task migration method, a scheduler, an electronic device and a readable storage medium, and relate to the technical field of distributed real-time stream processing. The method comprises: when it is determined that a first scheduling unit finishes preheating a target task, sending a task interruption instruction and a first request to a second scheduling unit; the task interruption instruction is used to instruct the second scheduling unit to interrupt execution of the target task, and the first request is used to obtain a first save point; the first save point represents a last piece of data processed by the second scheduling unit up to a task interruption time; a task start instruction is sent to the first scheduling unit, and the task start instruction is used to instruct the first scheduling unit to collect and process target data, the target data being data in the target task whose processing sequence is after the first save point. Embodiments of the present application reduce the delay generated in the preheating stage, avoid processing the backlog data in the preheating stage, and reduce the delay generated in the processing stage.

Description

Technical Field

[0001] This application relates to the field of distributed real-time stream processing technology, and more specifically, to a method for task migration, a scheduler, an electronic device, and a readable storage medium. Background Technology

[0002] As the amount of data increases, the necessity of distributed real-time stream processing becomes stronger. In order to improve throughput, distributed real-time stream processing tasks are often run on multiple scheduling units, that is, multiple scheduling units jointly execute the task. The scheduling unit is a process or thread. Each scheduling unit is constantly acquiring the data required to execute the task and processing the acquired data during the execution of the task.

[0003] Task scheduling refers to migrating a task from an original scheduling unit to a new scheduling unit. Existing solutions require interrupting the execution of the task by the original scheduling unit before starting a new scheduling unit to continue executing the task.

[0004] The new scheduling unit's task execution involves two phases: a warm-up phase and a processing phase. During the warm-up phase, the new scheduling unit needs to determine the savepoint corresponding to the interruption time when the original scheduling unit stopped executing the task, and retrieve the data after that savepoint from the data source. Since the new scheduling unit can only retrieve data from the data source during the warm-up phase and cannot process it, this introduces a delay. In the processing phase, the new scheduling unit begins processing data, but it needs to process the data accumulated during the warm-up phase before processing the real-time data, which also introduces a delay. The total delay of the warm-up and running phases is quite long, potentially reaching the minute level, which does not meet the delay requirements set for task migration (typically, the 99th percentile of the delay from data acquisition to data processing should be controlled within 10 seconds). Summary of the Invention

[0005] This application provides a method, scheduler, electronic device, computer-readable storage medium, and computer program product for task migration, which can solve the problem of long latency in existing solutions for task migration. The technical solution is as follows:

[0006] According to a first aspect of the embodiments of this application, a task migration method is provided, applied to a scheduler, the scheduler being configured to migrate a target task from a second scheduling unit to a first scheduling unit; the method includes:

[0007] When it is determined that the first scheduling unit has completed the preheating of the target task, a task interruption instruction and a first request are sent to the second scheduling unit. The task interruption instruction is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption.

[0008] In response to receiving the first save point sent by the second scheduling unit, a task start instruction is sent to the first scheduling unit. The task start instruction is used to instruct the first scheduling unit to collect and process target data, which is the data in the target task whose processing order is after the first save point.

[0009] In one possible implementation, before determining that the first scheduling unit has finished warming up the target task, the following steps are also included:

[0010] A second request is sent to the second scheduling unit. The second request is used to obtain the second save point. The second save point represents the last data that the second scheduling unit has completed processing up to the first moment. The first moment is the moment when the second request is received.

[0011] In response to receiving the second save point sent by the second scheduling unit, a first preheating instruction is sent to the first scheduling unit; the first preheating instruction is used to instruct the first scheduling unit to start preheating the target task, collect data in the processing order after the second save point during the preheating process, and send a preheating completion message after the preheating is completed; the preheating completion message is used to instruct the first scheduling unit to complete the preheating.

[0012] In one possible implementation, in response to receiving the warm-up complete message from the first scheduling unit, the following steps are included beforehand:

[0013] If a fault is detected in the first scheduling unit during the preheating process, a preheating cancellation command is sent to the first scheduling unit. The preheating cancellation command is used to cancel the task preheating performed by the first scheduling unit.

[0014] In one possible implementation, the method also includes:

[0015] If the target task being migrated is determined to be faulty at the time of the fault, and after the fault is repaired, if it is determined that the target task has not been completed at the current time and has not sent a task start instruction to the first scheduling unit before the fault, then a third request is sent to the first scheduling unit; the third request is used to obtain the warm-up status of the first scheduling unit.

[0016] If a preheating completion message is received from the first scheduling unit in response to the third request, it is determined that the first scheduling unit has completed preheating in the event of its own failure, and a task interruption instruction and the first request are sent to the second scheduling unit.

[0017] In one possible implementation, a third request is sent to the first scheduling unit, followed by:

[0018] If a fourth message is received from the first scheduling unit in response to the third request, and the fourth message indicates that the first scheduling unit is warming up, then it is determined that the first scheduling unit has not completed the warm-up process due to its own failure.

[0019] A second preheating instruction is sent to the first scheduling unit, which instructs the first scheduling unit to continue task preheating.

[0020] In one possible implementation, the second scheduling unit collects and processes data during the execution of the target task; the first scheduling unit collects data during the warm-up process but does not process the data.

[0021] In one possible implementation, the first scheduling unit is a process or thread; the second scheduling unit is a process or thread.

[0022] According to a second aspect of the embodiments of this application, a scheduler is provided, the scheduler being used to migrate a target task from a second scheduling unit to a first scheduling unit; the scheduler includes:

[0023] The interrupt control module is used to send a task interruption command and a first request to the second scheduling unit when it is determined that the first scheduling unit has completed the preheating of the target task. The task interruption command is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption.

[0024] The migration module is used to send a task start instruction to the first scheduling unit in response to receiving the first save point sent by the second scheduling unit. The task start instruction is used to instruct the first scheduling unit to collect and process target data, which is the data in the target task whose processing order is after the first save point.

[0025] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, wherein the processor executes the program to implement the steps of the method provided in the first aspect.

[0026] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method provided in the first aspect.

[0027] According to a fifth aspect of the present application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein when a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform steps implementing the method provided in the first aspect.

[0028] The beneficial effects of the technical solutions provided in this application are:

[0029] In this embodiment of the application, during migration scheduling, the first scheduling unit preheats the target task before the second scheduling unit interrupts the execution of the target task, reducing the latency generated during the preheating phase. Furthermore, after the first scheduling unit completes the preheating, it does not process the data collected during the preheating process, but directly collects and processes the target data whose processing order is after the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption, avoiding the processing of data accumulated during the preheating phase, reducing the latency generated during the processing phase, and thus significantly reducing the total latency of the preheating and processing phases. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0031] Figure 1 This is a schematic diagram of the system architecture for implementing the task migration method provided in the embodiments of this application;

[0032] Figure 2 A flowchart illustrating a task migration method provided in an embodiment of this application;

[0033] Figure 3 This is a schematic diagram illustrating an application scenario of a task migration method provided in an embodiment of this application;

[0034] Figure 4 A schematic diagram of the structure of a scheduler provided in an embodiment of this application;

[0035] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0037] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0039] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0040] Figure 1 This is a schematic diagram of the system architecture for implementing a task migration method provided in an embodiment of this application, which includes a first scheduling unit 110, a scheduler 130, and a second scheduling unit 130. Both the first scheduling unit 110 and the second scheduling unit 130 can be processes or threads, and both can execute a target task. The scheduler 130 is used to migrate the target task from the second scheduling unit 130 to the first scheduling unit 110.

[0041] This application provides a method for task migration, such as... Figure 2 As shown, this method is applied to a scheduler, which is used to migrate a target task from a second scheduling unit to a first scheduling unit; the method includes:

[0042] Step S201: When it is determined that the first scheduling unit has completed the preheating of the target task, a task interruption instruction and a first request are sent to the second scheduling unit. The task interruption instruction is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption.

[0043] The scheduling unit can execute tasks. Considering the lifecycle of the task execution by the scheduling unit, the entire lifecycle includes a warm-up phase and a data processing phase. During the warm-up phase, the scheduling unit warms up the task and can collect the data required for task execution, but it cannot process the collected data. When the scheduling unit is started or restarted after being interrupted, the scheduling unit needs to warm up the task.

[0044] The scheduling unit can be a thread or process or other unit that can perform resource scheduling or resource allocation. The scheduling unit in this application embodiment includes a first scheduling unit and a second scheduling unit. The task that the scheduler is migrating is called the target task. The scheduler is used to migrate the target task from the second scheduling unit to the first scheduling unit. That is, the first part of the target task can be executed by the second scheduling unit, and the second part can be executed by the second scheduling unit.

[0045] In this embodiment of the application, before the second scheduling unit is interrupted, the first scheduling unit can preheat the target task, and after the first scheduling unit has finished preheating, the interrupted second scheduling unit executes the target task.

[0046] In this embodiment of the application, after determining that the first scheduling unit has completed the preheating of the target task, a task interruption instruction and a first request are generated. The task interruption instruction is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain a first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption.

[0047] In this embodiment of the application, after generating the task interruption instruction and the first request, the task interruption instruction and the first request are sent to the second scheduling unit by calling the RPC (Remote Procedure Call Protocol) module. The RPC module is used to receive and send messages or commands exchanged between the scheduler and the scheduling unit.

[0048] During the execution of the target task, the second scheduling unit continuously collects and processes data. When the second scheduling unit receives a task interruption command from the scheduler, it responds to the command, interrupts the execution of the target task, and generates a first savepoint. This first savepoint represents the last data processed by the second scheduling unit up to the time of task interruption. The first savepoint can be data itself or a data identifier (the data identifier uniquely identifies the data).

[0049] It is understandable that the scheduling unit processes the collected data in a certain order, and the scheduling unit processes the collected data according to the order in which each piece of data is processed.

[0050] In this embodiment of the application, after generating the first savepoint, the second scheduling unit, in response to the first request sent by the scheduler, sends the first savepoint it generated to the scheduler.

[0051] Step S202: In response to receiving the first save point sent by the second scheduling unit, a task start instruction is sent to the first scheduling unit. The task start instruction is used to instruct the first scheduling unit to collect and process target data. The target data is the data in the target task whose processing order is after the first save point.

[0052] In this embodiment, after receiving the first savepoint sent by the second scheduling unit, the scheduler sends a task start instruction to the first scheduling unit in response to receiving the first savepoint. The task start instruction instructs the first scheduling unit to collect and process target data, which is the data that the target task needs to process and whose processing order is after the first savepoint.

[0053] After receiving the task start instruction, the first scheduling unit will not process the data collected during the preheating phase, but will instead start collecting and processing the target data that is in the processing order after the first save point, and process the target data. That is, there is no overlap between the data processed by the first scheduling unit and the data processed by the second unit.

[0054] In this embodiment of the application, during task scheduling, the first scheduling unit preheats the target task before the second scheduling unit interrupts the execution of the target task, reducing the latency generated during the preheating phase. Furthermore, after the first scheduling unit completes the preheating, it does not process the data collected during the preheating process, but directly collects and processes the target data whose processing order is after the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption, avoiding the processing of data accumulated during the preheating phase, reducing the latency generated during the processing phase, and thus significantly reducing the total latency of the preheating and processing phases.

[0055] This application embodiment provides a possible implementation method, which includes, before determining that the first scheduling unit has completed preheating of the target task, the following steps are also included:

[0056] A second request is sent to the second scheduling unit. The second request is used to obtain the second save point. The second save point represents the last data that the second scheduling unit has completed processing up to the first moment. The first moment is the moment when the second request is received.

[0057] In response to receiving the second save point sent by the second scheduling unit, a first preheating instruction is sent to the first scheduling unit; the first preheating instruction is used to instruct the first scheduling unit to start preheating the target task, collect data in the processing order after the second save point during the preheating process, and send a preheating completion message after the preheating is completed; the preheating completion message is used to instruct the first scheduling unit to complete the preheating.

[0058] In this embodiment, the preheating of the first scheduling unit occurs during the execution of the target task by the second scheduling unit. That is, from the start of preheating of the first scheduling unit to its completion, the second scheduling unit is not interrupted. The preheating of the first scheduling unit specifically includes:

[0059] During the execution of the target task by the second scheduling unit, a second request is sent to the second scheduling unit. The second request is used to obtain a second savepoint. The second savepoint represents the last data that the second scheduling unit has completed processing up to the first moment. The first moment is the moment when the second request is received. That is, after receiving the second request sent by the scheduler, the second scheduling unit determines the last data that has been processed at the time of receiving the second request and determines the second savepoint based on the data. The data can be directly used as the second savepoint or the data identifier of the data can be used as the second savepoint. This application embodiment does not limit this.

[0060] After generating the second savepoint, the second scheduling unit sends the second savepoint to the scheduler via the RPC module. The scheduler receives and responds to the second savepoint sent by the second scheduling unit, generates a first preheating instruction, and sends the first preheating instruction to the first scheduling unit via the RPC module. The first preheating instruction is used to instruct the first scheduling unit to start preheating the target task. During the preheating process, data with a processing order after the second savepoint is collected and processed. After the preheating is completed, a preheating completion message is sent. The preheating completion message is used to instruct the first scheduling unit to complete the preheating.

[0061] After receiving the first preheating instruction sent by the scheduler, the first scheduling unit begins to preheat the target task. During the preheating process, the first scheduling unit will collect data after the second save point in terms of processing order, but will not process the data collected during the preheating process.

[0062] After the first scheduling unit collects a preset amount of data and completes the preheating, it sends a preheating completion message to the scheduler via the RPC module. This preheating completion message indicates that the first scheduling unit has completed the preheating process.

[0063] This application embodiment provides a possible implementation, which, in response to receiving the preheating completion message sent by the first scheduling unit, further includes:

[0064] If a fault is detected in the first scheduling unit during the preheating process, a preheating cancellation command is sent to the first scheduling unit. The preheating cancellation command is used to cancel the task preheating performed by the first scheduling unit.

[0065] In the existing solution, when migrating a task, the execution of the target task by the second scheduling unit is interrupted first. After the second scheduling unit is interrupted, the first scheduling unit begins to warm up. If a fault is detected in the first scheduling unit during the warm-up process, the target task needs to be re-executed by the second scheduling unit. The second scheduling unit needs to warm up the target task again, which will also cause some delay.

[0066] In this embodiment, the preheating of the first scheduling unit begins during the execution of the target task by the second scheduling unit. If a fault is detected in the first scheduling unit during the preheating process, the preheating of the first scheduling unit needs to be interrupted. This can be achieved by sending a preheating cancellation command to the first scheduling unit. The preheating cancellation command cancels the task preheating performed by the first scheduling unit. Upon receiving the preheating cancellation command, the first scheduling unit will cancel the preheating of the target task. This avoids migrating the target task to the faulty scheduling unit. Furthermore, if the first scheduling unit fails during the preheating phase, since the preheating of the target task by the first scheduling unit has been interrupted, the target task will continue to be executed by the second scheduling unit without the need for the second scheduling unit to re-perform the preheating, effectively reducing latency.

[0067] This application provides a possible implementation method, which further includes:

[0068] If the target task being migrated is determined to be faulty at the time of the fault, and after the fault is repaired, if it is determined that the target task has not been completed at the current time and has not sent a task start instruction to the first scheduling unit before the fault, then a third request is sent to the first scheduling unit; the third request is used to obtain the warm-up status of the first scheduling unit.

[0069] If a preheating completion message is received from the first scheduling unit in response to the third request, it is determined that the first scheduling unit has completed preheating in the event of its own failure, and a task interruption instruction and the first request are sent to the second scheduling unit.

[0070] When migrating tasks, the scheduler itself may encounter failures. When the scheduler fails, it cannot migrate the target task. After the scheduler recovers from the failure, it will identify the target task that was being migrated when it failed and determine whether the target task has been completed. If the target task has been completed, there is no need to migrate the target task.

[0071] Specifically, for example, if the scheduler fails before migrating the target task (i.e. the target task is executed by the second scheduling unit), and it is determined after the failure that the target task has been completed by the second scheduling unit, then there is no need to migrate the target task from the second scheduling unit to the first scheduling unit.

[0072] For example, if the scheduler fails after migrating the target task to the first scheduling unit, and the first scheduling unit has already completed the target task after the failure is recovered, then there is no need to migrate the target task again.

[0073] If it is determined that the target task has not been completed at the current moment, and the task start instruction has been sent to the first scheduling unit before the fault occurred, then whether the physical first scheduling unit has completed the target task or not, there is no need to migrate the target task.

[0074] If it is determined that the target task has not been completed at the current time, and no task start instruction was sent to the first scheduling unit before the fault, it means that the first scheduling unit was in the warm-up process before the fault. Since it was in the fault process, it could not determine whether it received the warm-up completion message sent by the first scheduling unit. Therefore, it could not determine whether the first scheduling unit had completed the warm-up. So it sent a third request to the first scheduling unit. The third request was used to obtain the warm-up status of the first scheduling unit.

[0075] If a preheating completion message is received from the first scheduling unit in response to the third request, it is determined that the first scheduling unit has completed preheating during its own failure, meaning that the amount of data in the first scheduling unit at the current moment is sufficient. Therefore, the target task can be directly migrated from the second scheduling unit to the first scheduling unit. Specifically, a task interruption instruction and a first request are sent to the second scheduling unit to resume the migration of the target task. The task interruption instruction is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain the first savepoint. The first savepoint represents the last data processed by the second scheduling unit up to the time of task interruption.

[0076] This application embodiment provides a possible implementation, which involves sending a third request to a first scheduling unit, and then further includes:

[0077] If a fourth message is received from the first scheduling unit in response to the third request, and the fourth message indicates that the first scheduling unit is warming up, then it is determined that the first scheduling unit has not completed the warm-up process due to its own failure.

[0078] A second preheating instruction is sent to the first scheduling unit, which instructs the first scheduling unit to continue task preheating.

[0079] In this embodiment of the application, if the first scheduling unit has not completed preheating during the scheduler failure phase, the first scheduler will respond with a fourth message to the third request sent by the scheduler after the failure is repaired. The fourth message indicates that the first scheduling unit is preheating. After receiving the fourth message, the scheduler determines that the first scheduling unit has not completed preheating during its own failure.

[0080] After determining that the first scheduling unit has not completed preheating, the scheduler may not respond at all, but may directly wait for the first scheduling unit to complete preheating. After the first scheduling unit completes preheating, the scheduler may receive a preheating completion message sent by the first scheduling unit, or send a second preheating instruction to the first scheduling unit. The second preheating instruction is used to instruct the first scheduling unit to continue task preheating.

[0081] This application provides a possible implementation method in which the second scheduling unit collects and processes data during the execution of the target task; while the first scheduling unit collects data but does not process the data during the preheating process.

[0082] This application provides a possible implementation method in which the first scheduling unit is a process or a thread; the second scheduling unit is a process or a thread, both of which can perform resource scheduling or resource allocation.

[0083] Figure 2 This is a schematic diagram illustrating an application scenario of a task migration method provided in an embodiment of this application. Both the first scheduling unit and the second scheduling unit are threads. The first scheduling unit is referred to as the target thread, and the second scheduling unit is referred to as the original thread. The schematic diagram of the interaction flow between the original thread, the scheduler, and the target thread includes:

[0084] Step S301: The scheduler sends an initial savepoint acquisition request (the initial savepoint acquisition request is the second request mentioned above) to the original thread. The initial savepoint acquisition request is used to acquire the initial savepoint (the initial savepoint is the second savepoint mentioned above). The initial savepoint represents the last data processed by the initial thread up to the first moment. The first moment is the moment when the initial thread receives the initial savepoint acquisition request.

[0085] Step S302: The original thread generates an initial savepoint and sends the initial savepoint to the scheduler;

[0086] In step S303, the scheduler receives the initial save point sent by the original thread and sends a preheating instruction to the target thread. The preheating instruction instructs the target thread to start preheating the target task. During the preheating process, data that is processed after the initial save point is collected and a preheating completion message is sent after the preheating is completed.

[0087] Step S304: After receiving the preheating instruction, the target thread begins to preheat the target task. During the preheating process, data that is processed after the initial save point is collected and processed.

[0088] Step S305: After the target thread has finished warming up, it sends a warm-up completion message to the scheduler. The warm-up completion message is used to indicate that the target thread has finished warming up.

[0089] In step S306, after receiving the preheating completion message from the target thread, the scheduler sends a task interruption instruction and a target savepoint acquisition request (the target savepoint acquisition request is the first request mentioned above) to the original thread. The task interruption instruction is used to instruct the original thread to interrupt the execution of the target task, and the target savepoint acquisition request is used to acquire the target savepoint (the target savepoint is the first savepoint mentioned above). The target savepoint acquisition request represents the last data processed by the original thread up to the time of task interruption.

[0090] Step S307: After receiving the task interruption instruction, the original thread interrupts the execution of the target task and obtains the target save point;

[0091] Step S308: The original thread sends the target savepoint to the scheduler;

[0092] In step S309, the scheduler receives the target save point sent by the original thread and sends a task start instruction to the target thread. The task start instruction is used to instruct the target thread to collect and process the target data, which is the data in the target task that is processed after the target save point.

[0093] S310: The target thread receives the task start instruction, executes the target task, directly collects and processes the target data that is processed after the target save point.

[0094] Steps S301 to S305 above resolve the delay caused by the preheating stage, and steps S306 to S310 resolve the delay caused by the backlog of data during the preheating stage.

[0095] The specific implementation methods of steps S301-S310 are the same as those in the previous embodiments, and will not be described in detail here.

[0096] This application provides a scheduler 40, such as... Figure 4As shown, scheduler 40 is used to migrate the target task from the second scheduling unit to the first scheduling unit; scheduler 40 includes:

[0097] The interrupt control module 410 is used to send a task interruption instruction and a first request to the second scheduling unit when it is determined that the first scheduling unit has completed the preheating of the target task; the task interruption instruction is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain a first save point; the first save point represents the last data processed by the second scheduling unit up to the time of task interruption.

[0098] The migration module 420 is used to send a task start instruction to the first scheduling unit in response to receiving the first save point sent by the second scheduling unit. The task start instruction is used to instruct the first scheduling unit to collect and process target data, which is the data in the target task whose processing order is after the first save point.

[0099] In this embodiment of the application, during migration scheduling, the first scheduling unit preheats the target task before the second scheduling unit interrupts the execution of the target task, reducing the latency generated during the preheating phase. Furthermore, after the first scheduling unit completes the preheating, it does not process the data collected during the preheating process, but directly collects and processes the target data whose processing order is after the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption, avoiding the processing of data accumulated during the preheating phase, reducing the latency generated during the processing phase, and thus significantly reducing the total latency of the preheating and processing phases.

[0100] This application provides a possible implementation, and the scheduler further includes:

[0101] The second savepoint acquisition module sends a second request to the second scheduling unit. The second request is used to acquire the second savepoint. The second savepoint represents the last data processed by the second scheduling unit up to the first moment, which is the moment when the second request is received.

[0102] The preheating instruction generation module is used to send a first preheating instruction to the first scheduling unit in response to receiving the second save point sent by the second scheduling unit. The first preheating instruction is used to instruct the first scheduling unit to start preheating the target task, collect data in the processing order after the second save point during the preheating process, and send a preheating completion message after the preheating is completed. The preheating completion message is used to instruct the first scheduling unit to complete the preheating.

[0103] This application provides a possible implementation, and the scheduler further includes:

[0104] The preheating cancellation module is used to send a preheating cancellation command to the first scheduling unit if a fault is detected during the preheating process. The preheating cancellation command is used to cancel the task preheating performed by the first scheduling unit.

[0105] This application provides a possible implementation, and the device further includes:

[0106] The migration recovery module is used to determine the target task being migrated when it fails. After the failure is repaired, if it is determined that the target task has not been completed at the current time and no task start instruction was sent to the first scheduling unit before the failure, a third request is sent to the first scheduling unit. The third request is used to obtain the warm-up status of the first scheduling unit. If a warm-up completion message is received from the first scheduling unit in response to the third request, it is determined that the first scheduling unit has completed the warm-up when it failed, and a task interruption instruction and the first request are sent to the second scheduling unit.

[0107] This application provides a possible implementation method. The migration recovery module is further configured to, if it receives a fourth message sent by the first scheduling unit in response to the third request, the fourth message indicating that the first scheduling unit is performing preheating, determine that the first scheduling unit has not completed preheating when it fails; and send a second preheating instruction to the first scheduling unit, the second preheating instruction being used to instruct the first scheduling unit to continue task preheating.

[0108] This application provides a possible implementation method in which the second scheduling unit collects and processes data during the execution of the target task; the first scheduling unit collects data but does not process the data during the preheating process.

[0109] This application provides a possible implementation method in which the first scheduling unit is a process or thread, and the second scheduling unit is a process or thread.

[0110] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0111] This application provides an electronic device including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a task migration method. Compared with related technologies, this application can achieve the following: When performing migration scheduling, the first scheduling unit preheats the target task before the second scheduling unit interrupts the execution of the target task, reducing the latency generated during the preheating stage; and after the first scheduling unit finishes preheating, it does not process the data collected during the preheating process, but directly collects and processes the target data whose processing order is after the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption, avoiding the processing of data accumulated during the preheating stage, reducing the latency generated during the processing stage, and significantly reducing the total latency of the preheating stage and the processing stage.

[0112] In one alternative embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The illustrated electronic device 5000 includes a processor 5001 and a memory 5003. The processor 5001 and the memory 5003 are connected, for example, via a bus 5002. Optionally, the electronic device 5000 may further include a transceiver 5004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 5004 is not limited to one type, and the structure of the electronic device 5000 does not constitute a limitation on the embodiments of this application.

[0113] Processor 5001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 5001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0114] Bus 5002 may include a path for transmitting information between the aforementioned components. Bus 5002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 5002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0115] The memory 5003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0116] The memory 5003 is used to store computer programs that execute the embodiments of this application, and its execution is controlled by the processor 5001. The processor 5001 is used to execute the computer programs stored in the memory 5003 to implement the steps shown in the foregoing method embodiments.

[0117] The electronic device package may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0118] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, this application achieves the following: During migration scheduling, the first scheduling unit preheats the target task before the second scheduling unit interrupts the execution of the target task, reducing the latency generated during the preheating phase. Furthermore, after the first scheduling unit completes the preheating, the data collected during the preheating process is not processed. Instead, the target data whose processing order is after the first save point is directly collected and processed. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption. This avoids processing the data accumulated during the preheating phase, reducing the latency generated during the processing phase, and significantly reducing the total latency of the preheating and processing phases.

[0119] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0120] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, this application embodiment achieves the following: During migration scheduling, before the second scheduling unit interrupts the execution of the target task, the first scheduling unit preheats the target task, reducing the latency generated during the preheating phase; and after the first scheduling unit completes the preheating, it does not process the data collected during the preheating process, but directly collects and processes the target data whose processing order is after the first save point. The first save point represents the last data processed by the second scheduling unit up to the time of task interruption, avoiding the processing of data accumulated during the preheating phase, reducing the latency generated during the processing phase, and thus significantly reducing the total latency of the preheating and processing phases.

[0121] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0122] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0123] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application, without departing from the technical concept of this application, also fall within the protection scope of the embodiments of this application.

Claims

1. A method for task migration, characterized in that, Applied to a scheduler, the scheduler is used to migrate a target task from a second scheduling unit to a first scheduling unit; the method includes: When it is determined that the first scheduling unit has completed the preheating of the target task, a task interruption instruction and a first request are sent to the second scheduling unit; the task interruption instruction is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain a first save point; the first save point represents the last data processed by the second scheduling unit up to the time of task interruption. In response to receiving the first save point sent by the second scheduling unit, a task start instruction is sent to the first scheduling unit. The task start instruction is used to instruct the first scheduling unit to collect and process target data, which is the data in the target task whose processing order is after the first save point. Before receiving the task interruption instruction, the second scheduling unit continuously collects and processes data during the execution of the target task; the first scheduling unit collects data but does not process the data during the warm-up process.

2. The method according to claim 1, characterized in that, Before determining that the first scheduling unit has completed preheating the target task, the process also includes: A second request is sent to the second scheduling unit. The second request is used to obtain a second save point. The second save point represents the last data that the second scheduling unit has completed processing up to the first time point, where the first time point is the time when the second request is received. In response to receiving the second save point sent by the second scheduling unit, a first preheating instruction is sent to the first scheduling unit; the first preheating instruction is used to instruct the first scheduling unit to start preheating the target task, collect and process data after the second save point during the preheating process, and send a preheating completion message after the preheating is completed; the preheating completion message is used to instruct the first scheduling unit to complete the preheating.

3. The method according to claim 2, characterized in that, The response to receiving the preheating completion message sent by the first scheduling unit further includes, before: If a fault is detected in the first scheduling unit during the preheating process, a preheating cancellation command is sent to the first scheduling unit. The preheating cancellation command is used to cancel the task preheating performed by the first scheduling unit.

4. The method according to claim 2, characterized in that, The method further includes: If the target task being migrated at the time of the fault is determined, and after the fault is repaired, if it is determined that the target task has not been completed at the current time and has not sent a task start instruction to the first scheduling unit before the fault, then a third request is sent to the first scheduling unit; the third request is used to obtain the warm-up status of the first scheduling unit. If a preheating completion message is received from the first scheduling unit in response to the third request, it is determined that the first scheduling unit has completed preheating in the event of its own failure, and the task interruption instruction and the first request are sent to the second scheduling unit.

5. The method according to claim 4, characterized in that, The step of sending the third request to the first scheduling unit further includes: If a fourth message is received from the first scheduling unit in response to the third request, and the fourth message indicates that the first scheduling unit is performing preheating, then it is determined that the first scheduling unit has not completed preheating due to its own failure. A second preheating instruction is sent to the first scheduling unit, which instructs the first scheduling unit to continue task preheating.

6. The method according to any one of claims 1-5, characterized in that, The first scheduling unit is a process or a thread; the second scheduling unit is a process or a thread.

7. A scheduler, characterized in that, The scheduler is used to migrate the target task from the second scheduling unit to the first scheduling unit; the scheduler includes: The interrupt control module is used to send a task interruption instruction and a first request to the second scheduling unit when it is determined that the first scheduling unit has completed the preheating of the target task; the task interruption instruction is used to instruct the second scheduling unit to interrupt the execution of the target task, and the first request is used to obtain a first save point; the first save point represents the last data processed by the second scheduling unit up to the time of task interruption. The migration module is used to send a task start instruction to the first scheduling unit in response to receiving a first save point sent by the second scheduling unit. The task start instruction is used to instruct the first scheduling unit to collect and process target data, which is data in the target task whose processing order is after the first save point. Before receiving the task interruption instruction, the second scheduling unit continuously collects and processes data during the execution of the target task; the first scheduling unit collects data but does not process the data during the warm-up process.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images