Task scheduling method and device

Abstract

The embodiment of the application provides a task scheduling method and device, comprising: in the case that the first demand resource quantity of a service task is greater than or equal to a preset resource quantity threshold, splitting the service task into a plurality of split tasks, and determining the split tasks as to-be-scheduled tasks; and scheduling the to-be-scheduled tasks to resource nodes for execution. The application splits a larger service task, schedules the split tasks, thereby reducing the difficulty of subsequent scheduling tasks, reducing the probability of resource quantity overflow during subsequent task execution, and improving the execution success rate of the service task.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a task scheduling method, apparatus, electronic device, and machine-readable medium. Background Technology

[0002] Cloud editing involves editing multimedia materials in the cloud, including decoding, compositing, special effects processing, and encoding.

[0003] During cloud editing, service tasks are scheduled to resource nodes for processing. However, each stage of cloud editing requires significant resources, which often leads to Out of Memory (OOM) errors when resource nodes execute service tasks. An OOM error occurs when the memory required by a task exceeds the memory available for allocation on the resource node. Related technologies address this by halting the execution of some tasks on the resource node when an OOM error occurs, thus ensuring the resource node's system continues to provide normal service.

[0004] However, the inventors discovered through research that the current solution, which stops tasks that encounter OOM (Out of Memory) during execution, causes a large number of task execution failures, thus reducing the user experience. Summary of the Invention

[0005] This application provides a task scheduling method to solve the problem of a large number of task execution failures and reduced user experience in related technologies.

[0006] Accordingly, embodiments of this application also provide a task scheduling device, an electronic device, and a storage medium to ensure the implementation and application of the above methods.

[0007] To address the aforementioned problems, this application discloses a task scheduling method, the method comprising:

[0008] Obtain service tasks;

[0009] If the first resource requirement of the service task is greater than or equal to a preset resource threshold, the service task is divided into multiple fragmented tasks, and the fragmented tasks are identified as tasks to be scheduled.

[0010] The task to be scheduled is scheduled to the resource node for execution.

[0011] This application discloses a task scheduling device, the device comprising:

[0012] The acquisition module is used to acquire service tasks;

[0013] The segmentation module is used to segment the service task into multiple segmented tasks when the first required resource amount of the service task is greater than or equal to a preset resource amount threshold, and to determine the segmented tasks as tasks to be scheduled.

[0014] The execution module is used to schedule the task to be scheduled to the resource node for execution.

[0015] This application also discloses an electronic device, including: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform one or more methods as described in this application.

[0016] This application also discloses one or more machine-readable media storing executable code thereon, which, when executed, causes a processor to perform one or more of the methods described in this application.

[0017] Compared with related technologies, the embodiments of this application have the following advantages:

[0018] In this embodiment, by estimating and judging the resource requirements of a service task, it can be determined whether the service task is difficult to schedule due to excessive resource requirements. Service tasks with excessive resource requirements can be divided into multiple fragmented tasks before subsequent scheduling, thereby reducing the difficulty of subsequent scheduling. Scheduling smaller fragmented tasks to resource nodes can reduce the probability of OOM and improve the success rate of service task execution. Attached Figure Description

[0019] Figure 1 This is a system architecture diagram of an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of an editing task processing scenario provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a cloud computing task scenario according to an embodiment of this application;

[0022] Figure 4 This is a flowchart illustrating the steps of a task scheduling method according to an embodiment of this application;

[0023] Figure 5 This is a flowchart illustrating the specific steps of a task scheduling method according to an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of a cloud-based service task processing procedure according to an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of a task slicing process according to an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of a scheduling process according to an embodiment of this application;

[0027] Figure 9 This is a schematic diagram of a resource quantity estimation process according to an embodiment of this application;

[0028] Figure 10 This is a schematic diagram of the overall process of a task scheduling method according to an embodiment of this application;

[0029] Figure 11 This is a block diagram of a task scheduling device according to an embodiment of this application;

[0030] Figure 12 This is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation

[0031] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] To enable those skilled in the art to better understand this application, the concepts involved in this application are explained below:

[0033] Service task: A task that is processed using computing resources. In a cloud computing scenario, it can be a cloud service task.

[0034] Editing task: A service task that performs editing processing on multimedia materials. Editing involves decoding, special effects processing, encoding and other steps of multimedia materials. The data corresponding to the editing task can include the editing material, output duration, the time period of the editing material in the output duration, special effects processing performed on the editing material, etc. Among them, the output duration defines the total duration of the multimedia content output by the editing task.

[0035] Out of Memory (OOM): Also known as resource overflow, it refers to a situation where, during the processing of a task, the memory space required by the task exceeds the memory space that the resource node can allocate. In other words, there is unrecoverable memory or excessive memory usage in the resource node's system, which ultimately causes the memory required for task execution to exceed the maximum memory that the system can provide, resulting in the task failing to execute.

[0036] Resource requirements: The amount of resources required to complete the service task, specifically an estimated amount.

[0037] Sharded tasks: Tasks obtained by slicing a complete service task. Sharded tasks are part of a service task. They are smaller in size and easier to schedule later.

[0038] Resource node: A carrier with the ability to compute and process tasks; it can be a physical machine or a virtual machine.

[0039] The task scheduling method in this embodiment can be applied to scenarios involving the processing of service tasks submitted by user devices. These scenarios may include video editing tasks and cloud computing tasks. Taking video editing tasks as an example, the task scheduling method in this embodiment allows the user device to initiate an editing task to a cloud server. After receiving the editing task, the cloud server can schedule it to a resource node in the resource cluster for execution. The completed editing result is then sent to the cloud server, which forwards it to the user device, thereby enabling fast and efficient video editing using cloud processing capabilities.

[0040] Reference Figure 1 This document illustrates a system architecture diagram of a task scheduling method provided in an embodiment of this application, including: a user device, a cloud server, and a resource cluster. The user device sends service tasks to the cloud server. The cloud server and the resource cluster together constitute a cloud environment, providing cloud processing capabilities. Specifically, after obtaining a service task, the cloud server schedules the task to resource nodes in the resource cluster. The resource nodes execute the service task, obtain the execution result, and send it to the cloud server. Finally, the cloud server feeds back the execution result to the user device, thus completing the processing of the service task.

[0041] Specifically, a cloud server comprises a service layer, a scheduling layer, and an engine layer. The service layer interacts with user devices and performs initial resource estimation and limit checks on service tasks sent by user devices. If the limit check determines that the resource requirement of a service task is too large, the task is sliced ​​into multiple sub-tasks, which are then designated as tasks to be scheduled. If the limit check determines that the resource requirement of a service task is normal, the task is designated as a task to be scheduled. Task slicing breaks large tasks into smaller ones, and subsequent scheduling of these smaller tasks with lower resource requirements is easier and can minimize the occurrence of OutOfMemoryError (OOM).

[0042] The scheduling layer is used to schedule tasks to resource nodes for execution. In this embodiment, specifically, the scheduling layer can further accurately estimate and determine the resource requirements of the tasks to be scheduled. If the estimated resource requirements are determined and a suitable target resource node exists in the resource cluster (the remaining resources of the target resource node are greater than or equal to the resource requirements of the tasks to be scheduled), the tasks to be scheduled can be scheduled to the target resource node for execution. If the estimated resource requirements are determined and a suitable target resource node does not exist in the resource cluster, new resource nodes with higher specifications can be added to the resource cluster, and the tasks to be scheduled can be scheduled to the new resource nodes for execution.

[0043] The execution layer is used to execute service tasks scheduled to resource nodes within the resource cluster. However, during the execution of service tasks on resource nodes, resource overflow (OOM) issues may occur for various reasons, leading to insufficient resources on the resource node to continue executing the service task. For example, if there are accuracy issues in the estimation of the resource requirements of service tasks in the service and scheduling layers of the cloud server, there is a chance of incorrect estimation (the estimated resource requirement is smaller than the actual resource requirement). This significantly increases the likelihood of resource overflow issues when resource nodes run service tasks.

[0044] In this embodiment, the execution layer can, in the event of resource overflow during the execution of scheduled tasks on a resource node, stop at least a portion of the scheduled tasks executing on that resource node and reschedule those tasks to other resource nodes for re-execution. First, stopping the execution of at least a portion of the scheduled tasks ensures the normal operation of the resource node system, preventing it from crashing and improving service stability. Second, rescheduling the stopped tasks to other resource nodes maximizes the completion rate of service tasks, thus improving the success rate of service task execution.

[0045] This application's embodiments can determine whether a service task is difficult to schedule due to excessive resource requirements by estimating and scheduling its resource needs. Service tasks with excessive resource requirements are then segmented into multiple fragmented tasks before subsequent scheduling, thereby reducing the difficulty of subsequent scheduling. Scheduling smaller fragmented tasks to resource nodes reduces the probability of OutOfMemoryError (OOM). Furthermore, if resource overflow occurs during the execution of scheduled tasks on a resource node, this application stops the execution of at least some of the scheduled tasks on that resource node, ensuring the normal operation of the resource node, preventing it from crashing, and improving service stability. Additionally, rescheduling the stopped scheduled tasks to other resource nodes for re-execution maximizes the completion rate of service tasks, improving the success rate of service task execution.

[0046] It should be noted that the task scheduling method provided based on the embodiments of this application can be implemented in several specific scenarios as follows:

[0047] In one implementation, refer to Figure 2 The illustration shows a schematic diagram of an editing task processing scenario provided by an embodiment of this application, including: a user device, a cloud editing server, and a resource cluster. The user device can initiate an editing task to the cloud editing server. The editing task is used to edit multimedia materials, including decoding, compositing, special effects processing, encoding, etc. After receiving the editing task, the cloud editing server can schedule the editing task to a resource node in the resource cluster for execution. The completed editing result is sent to the cloud editing server, which then forwards it to the user device, thereby realizing fast and efficient editing processing using cloud processing capabilities.

[0048] In another implementation, refer to Figure 3 The illustration shows a schematic diagram of a cloud computing task scenario provided by an embodiment of this application, including: a user device, a cloud computing server, and a resource cluster. The user device can initiate a cloud computing task to the cloud computing server. The cloud computing task is used to perform high-performance computing on data, such as computing time-series data in the logistics process, statistical computing of data in the e-commerce sales process, and training of deep learning models. After the cloud computing server receives the cloud computing task, it can schedule the cloud computing task to a resource node in the resource cluster for execution. The completed computing result is sent to the cloud computing server, and then forwarded to the user device by the cloud computing server, thereby realizing fast and efficient computing processing by utilizing cloud processing capabilities.

[0049] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0050] Reference Figure 4 It illustrates a flowchart of the steps of a task scheduling method provided in an embodiment of this application, including:

[0051] Step 101: Obtain service tasks.

[0052] In this application embodiment, the service task is a task used to process using cloud processing capabilities. For example, a service task in a video editing scenario can be an editing task, which is a task to edit multimedia materials. Cloud editing involves decoding, compositing, special effects processing, encoding and other steps of multimedia materials.

[0053] Specifically, refer to Figure 1 In the scenario of service task execution, the user device generates service tasks according to actual needs and sends the service tasks to the cloud server. After the cloud server receives the service task, it can schedule the service task to the resource node in the resource cluster for execution. The result of the execution is sent to the cloud server, and then forwarded to the user device by the cloud server, thereby realizing fast and efficient service task processing by utilizing cloud processing capabilities.

[0054] Step 102: If the first resource requirement of the service task is greater than or equal to a preset resource threshold, the service task is divided into multiple fragmented tasks, and the fragmented tasks are identified as tasks to be scheduled.

[0055] In this embodiment, after the cloud server receives the service task, it can first perform a preliminary resource quantity estimation and limit judgment on the service task sent by the user device. The resource quantity estimation is based on the content of the service task to estimate the first resource requirement of the service task. The limit judgment is to compare the first resource requirement with a preset resource quantity threshold to determine whether the service task has an excessively large resource requirement. If the first resource requirement of the service task is greater than or equal to the preset resource quantity threshold, the service task is considered to be a large-scale task. Direct scheduling would likely cause an OutOfMemoryError (OOM) when subsequent resource nodes run the service task. If the first resource requirement of the service task is less than the preset resource quantity threshold, the service task is considered to be a normal-scale task and can be directly scheduled as a task to be scheduled.

[0056] Furthermore, if the cloud server determines through an over-limit check that the service task is a large-scale task, this embodiment of the application can slice the service task, dividing it into multiple fragmented tasks, and identifying these fragmented tasks as tasks to be scheduled. If the over-limit check determines that the resource requirements of the service task are normal, the service task is then designated as a task to be scheduled. Task slicing can break down large tasks into smaller tasks, and subsequent scheduling of fragmented tasks with smaller resource requirements is easier, reducing the likelihood of OutOfMemoryError (OOM).

[0057] Step 103: Schedule the task to be scheduled to the resource node for execution.

[0058] In this embodiment of the application, the cloud server schedules the task to be scheduled to the resource node for execution. Specifically, it can first accurately estimate the resource requirement of the task to be scheduled and collect the remaining resources of each resource node in the resource cluster. If the remaining resources of a resource node are greater than or equal to the resource requirement of the task to be scheduled, the task to be scheduled can be scheduled to that resource node for execution.

[0059] In summary, the embodiments of this application can determine whether a service task is difficult to schedule due to excessive resource requirements by estimating and scheduling the resource requirements of the service task. Service tasks with excessive resource requirements can be divided into multiple fragmented tasks before subsequent scheduling, thereby reducing the difficulty of subsequent scheduling. Scheduling smaller fragmented tasks to resource nodes can reduce the probability of OOM and improve the success rate of service task execution.

[0060] Reference Figure 5 It illustrates a flowchart of the specific steps of a task scheduling method provided in an embodiment of this application, including:

[0061] Step 201: Obtain service tasks.

[0062] For details of this step, please refer to step 101 above; it will not be repeated here.

[0063] Step 202: If the service task is an editing task and the first required resource amount of the editing task is greater than or equal to a preset resource amount threshold, the editing task is divided into multiple segmented tasks according to the editing materials, output duration, first required resource amount, and time period of the editing materials in the output duration contained in the data corresponding to the editing task.

[0064] The service task is an editing task, and the data corresponding to the editing task includes: editing materials, output duration, and the time period in which the editing materials are located within the output duration.

[0065] Reference Figure 6 It illustrates a schematic diagram of the cloud-based service task processing process. First, the service layer of the cloud server is used to interact with the user device and obtain the service tasks sent by the user device. The scheduling layer then schedules the tasks to be scheduled to the resource nodes running Pods in the engine layer for execution.

[0066] In a container scheduling environment, a Pod can be viewed as a collection of container objects. These containers can be deployed and run on resource nodes, which can be virtual machine nodes or physical machine nodes. Specifically, a pod is the smallest unit that can be deployed and managed in container scheduling. A pod can be understood as a resource object, and it can consist of multiple containers, specifically a root container plus one or more user-defined containers. Multiple containers within a pod share the pod's Internet Protocol (IP) address and data volume. Therefore, in a container scheduling environment, the pod is the carrier of containers; all containers are managed within pods. Placing one or more containers within a pod as a unit facilitates management and achieves tight coupling of services.

[0067] In this embodiment of the application, a specific description of the task scheduling implementation is provided for a scenario where the service task is an editing task. After the cloud server obtains the editing task, it can first perform a preliminary resource quantity estimation and limit judgment on the editing task sent by the user device. If the limit judgment determines that the first required resource quantity of the editing task is greater than or equal to the preset resource quantity threshold, it can be determined that the editing task is a large-scale task. In this embodiment of the application, the editing task can be sliced ​​according to the editing material, output duration, first required resource quantity, and the time period in which the editing material is located in the output duration, based on the data corresponding to the editing task. The editing task is divided into multiple segmented tasks, and the segmented tasks are determined as tasks to be scheduled for subsequent scheduling.

[0068] Specifically, the data corresponding to the editing task includes: editing materials, output duration, and the time period in which the editing materials are located within the output duration. The editing materials can be multimedia materials, such as audio files, video files, images, etc. The output duration defines the total duration of the multimedia content output after the editing task is completed. The time period in which the editing materials are located within the output duration reflects the time period in which each editing material is located within the entire duration of the multimedia content after final editing.

[0069] For example, for an editing task, the goal is to edit three clips, a (3 minutes), b (4 minutes), and c (5 minutes), into a final clip with an output length of 12 minutes. Specifically, clip a is edited into the first 3 minutes of the final clip (i.e., clip a appears in the first 3 minutes of the output length), clip b is edited into the 3rd to 7th minutes of the final clip (i.e., clip b appears in the 3rd to 7th minutes of the output length), and clip c is edited into the last 4 minutes of the final clip (i.e., clip c appears in the last 4 minutes of the output length).

[0070] Optionally, the resource quantity threshold is: the configured resource quantity of the resource node with the largest configured resource quantity in the cluster to which the resource node belongs.

[0071] In this embodiment of the application, after the cloud server obtains the editing task, the resource threshold used in the over-limit judgment of the editing task can be a fixed value. This resource threshold is not related to the remaining resource amount of the resource node, but is directly related to the hardware configuration of the resource node. Specifically, it can be the configured resource amount of the resource node with the largest configured resource amount in the cluster to which the resource node belongs. For example, if the resource node is a physical machine, and the resource node with the highest configuration in the cluster is a c7.16xlarge model resource node with a configured memory (configured resource amount) size of 128G, then the resource threshold can be 128G.

[0072] Optionally, step 202 may specifically include:

[0073] Sub-step 2021: Determine the number of tasks in the sharding task based on the first required resource amount and the preset upper limit of the resource amount of the sharding task.

[0074] Sub-step 2022: Determine the segment duration corresponding to the segmented task based on the output duration and the number of tasks.

[0075] Sub-step 2023: Based on the final video length and the time period in which the edited material is located within the output length, the edited material is segmented, thereby dividing the editing task into the number of segmented tasks.

[0076] In this embodiment of the application, for sub-steps 2021-2023, when performing task slicing of the service task, the number of tasks in the slicing task can be determined first based on the ratio of the first required resource amount of the service task to the preset upper limit of the resource amount of the slicing task. Then, based on the ratio of the output duration to the number of tasks, the final duration corresponding to the slicing task is determined. Finally, the editing material is sliced ​​according to the final duration and the time period in which the editing material is located within the output duration, thereby dividing the editing task into the number of slicing tasks. Task slicing can divide a large task into smaller tasks. Subsequently, the smaller tasks with smaller required resource amounts are scheduled, which is less difficult and can minimize the occurrence of OOM. Finally, the results of the multiple smaller tasks after slicing are merged to obtain the processing result of the large task.

[0077] Specifically, refer to Figure 7 The diagram illustrates a task slicing process. Assuming an editing task has an output duration of 100 minutes, the estimated initial resource requirement is 200GB of memory. According to the principle that the initial resource requirement is proportional to the output duration, editing a 100-minute output requires 200GB of memory, and processing a 1-minute finished piece requires 2GB of memory.

[0078] Furthermore, based on actual needs, the maximum resource limit for each segment task is set to no more than 5GB of memory. Therefore, the editing task needs to be divided into 200 / 5 = 40 segments (number of tasks). The output duration of each segment task is 100 minutes / 40 = 2.5 minutes. The editing material is then divided according to the time period in the output duration. Each segment task retains its own time period for editing operations, thus dividing the editing task into 40 segment tasks. Then, each of the 40 segment tasks is scheduled separately and each segment task is scheduled to the corresponding Pod in the resource node for execution.

[0079] Specifically, assuming the editing task corresponds to data including footage a (50 minutes) and footage b (50 minutes), where footage a occupies the first 50 minutes of the 100-minute output duration, and footage b occupies the last 50 minutes, then for... Figure 7 The above-mentioned segmentation method, which divides the editing material into 40 segment tasks according to the time period in the output duration, specifically includes:

[0080] The first 20 of the 40 final segment tasks are obtained by segmenting source material 'a' and the editing operations performed on it. Since each segment task is 2.5 minutes long, the data for the first segment task includes the first 2.5 minutes of source material 'a' and the editing operations performed on that content; the data for the second segment task includes the content from the 2.5th to the 5th minute of source material 'a' and the editing operations performed on that content; and so on, until the twentieth segment task includes the last 2.5 minutes of source material 'a' and the editing operations performed on that content. The last 20 of the 40 final segment tasks are obtained by segmenting source material 'b' and the editing operations performed on it, with the segmentation process being similar.

[0081] It should be noted that the above-mentioned segmentation method is to segment the complete editing task according to a preset and fixed upper limit of the resource amount of the segmented task, so as to obtain segmented tasks that do not exceed the upper limit of the resource amount. The segmentation method of this application embodiment is not limited to this, and other segmentation methods can also be used. This application embodiment does not limit this.

[0082] Step 203: Schedule the task to be scheduled to the resource node for execution.

[0083] For details of this step, please refer to step 102 above, which will not be repeated here.

[0084] Optionally, step 203 may specifically include:

[0085] Sub-step 2031: Obtain the second required resource amount of the task to be scheduled.

[0086] Sub-step 2032: If there is a resource node with remaining resources greater than or equal to the second required resource amount, the task to be scheduled is scheduled to be executed on the resource node.

[0087] Sub-step 2033: If there is no resource node with remaining resources greater than or equal to the second required resource amount, add a new resource node to the cluster to which the resource node belongs, and schedule the task to be scheduled to the new resource node for execution, wherein the remaining resources of the new resource node are greater than or equal to the second required resource amount.

[0088] In this embodiment, regarding sub-steps 2031-2033, the specific scheduling and execution of the task to be scheduled by the cloud server can first be achieved by accurately estimating the required resource quantity, estimating the second required resource quantity of the task to be scheduled, and collecting the remaining resource quantity of each resource node in the resource cluster. Then, based on the estimated second required resource quantity, a scheduling judgment is made. If it is determined that there is a suitable target resource node in the resource cluster (the remaining resource quantity of the target resource node is greater than or equal to the second required resource quantity of the task to be scheduled), the task to be scheduled can be scheduled to the target resource node for execution. If, based on the estimated second required resource quantity, a scheduling judgment is made and it is determined that there is no suitable target resource node in the resource cluster, a new resource node with a higher configuration can be added to the resource cluster, and the task to be scheduled can be scheduled to the new resource node for execution, wherein the remaining resource quantity of the new resource node is greater than or equal to the second required resource quantity.

[0089] Specifically, refer to Figure 8 The diagram illustrates a scheduling process. Assume a resource node has a total resource capacity of 16GB of memory. The resource cluster currently contains resource nodes A, B, and C. Node A has 10GB of remaining memory, node B has 6GB, and node C has 13GB. The second resource requirement for task 1 is 10.5GB. By assessing the remaining resources of nodes A, B, and C, it can be determined that node C has more remaining resources than task 1, and therefore can be used to execute task 1. Thus, the cloud server can schedule task 1 to resource node C for execution.

[0090] Furthermore, assuming that the second resource requirement of the task to be scheduled 2 is 18G of memory, after analysis, resource nodes A, B, and C cannot meet its requirements. Therefore, in this embodiment, a new resource node D with a higher configuration (total resource amount of 32G of memory) can be added to the resource cluster, and then the task to be scheduled 2 can be scheduled to resource node D for execution.

[0091] Optionally, the service task is an editing task, and the data corresponding to the task to be scheduled includes: editing materials and special effects processing performed on the editing materials; sub-step 2031 may specifically include:

[0092] Sub-step 20311: Obtain the third resource requirement for decoding the clip material, the fourth resource requirement for performing the special effects processing on the clip material, and the fifth resource requirement for encoding the clip material.

[0093] Sub-step 20312: Sum the third resource requirement, the fourth resource requirement, and the fifth resource requirement to obtain the second resource requirement of the task to be scheduled.

[0094] In this embodiment of the application, for sub-steps 20311-20312, when the service task is an editing task, the data corresponding to the task to be scheduled includes: editing materials and special effects processing performed on the editing materials. Cloud editing includes decoding, special effects processing, and encoding of the editing materials. These processes all require resource processing. In the process of accurately estimating the second resource requirement of the task to be scheduled, the third resource requirement for decoding the editing materials, the fourth resource requirement for performing special effects processing on the editing materials, and the fifth resource requirement for encoding the editing materials can be obtained. Finally, the third, fourth, and fifth resource requirements are summed to obtain the second resource requirement of the task to be scheduled.

[0095] For example, refer to Figure 9 The diagram illustrates a resource estimation process. Assume a scheduled task contains clip 1, clip 2, and clip 3. Decoding clip 1 requires 1GB of memory, processing clip 1's effects requires 0.2GB, decoding clip 2 requires 0.8GB, processing clip 2's effects requires 0.5GB, decoding clip 3 requires 2GB, processing clip 3's effects requires 4GB, and encoding clip 1, clip 2, and clip 3 as a whole requires 4GB. Therefore, the estimated second resource requirement for the scheduled task = 1 + 0.2 + 0.8 + 0.5 + 2 + 4 + 4 = 12.5GB.

[0096] It should be noted that in the above embodiments, the first resource requirement of the service task estimated by the service layer of the cloud server can be a rough estimate, while the second resource requirement of the task to be scheduled estimated by the scheduling layer of the cloud server can be a precise estimate. The difference between the two is that when estimating the resource requirement, some prior coefficients, weights and other auxiliary parameters can be introduced to participate in the estimation calculation. The update and optimization frequency of the auxiliary parameters used when estimating the second resource requirement is much greater than the update and optimization frequency of the auxiliary parameters used when estimating the first resource requirement. This makes the estimation of the second resource requirement relatively more accurate.

[0097] Step 204: If a resource overflow occurs during the execution of the scheduled task on the resource node, at least some of the scheduled tasks executed on the resource node are stopped, and the at least some of the scheduled tasks are rescheduled to other resource nodes for re-execution.

[0098] In this embodiment, during the execution of service tasks by resource nodes, resource overflow (OOM) may occur for various reasons, leading to insufficient resources on the resource nodes to continue executing the service tasks. For example, if there are accuracy issues in the estimation of the resource requirements of service tasks at the service and scheduling layers of the cloud server, there is a chance of incorrect estimation of the resource requirements (the estimated resource requirements are smaller than the actual resource requirements). This significantly increases the likelihood of resource overflow when resource nodes run service tasks.

[0099] The cloud server in this embodiment can, in the event of resource overflow during the execution of scheduled tasks on a resource node, stop at least a portion of the scheduled tasks executing on that resource node and reschedule those tasks to other resource nodes for re-execution. Firstly, stopping the execution of at least a portion of the scheduled tasks ensures the normal operation of the resource node system, preventing it from crashing and improving service stability. Secondly, rescheduling the stopped tasks to other resource nodes maximizes the completion rate of service tasks, thus improving the success rate of service task execution.

[0100] Optionally, step 204 may specifically include:

[0101] Sub-step 2041: If a resource overflow occurs during the execution of the scheduled task on the resource node, stop the target scheduled task on the resource node where at least some of the resources have overflowed, and reschedule the target scheduled task to other resource nodes for re-execution.

[0102] In this embodiment, if a resource overflow occurs during the execution of the scheduled task by a resource node, it is preferable to stop at least part of the target scheduled task in the resource node that has experienced a resource overflow, and then reschedule the target scheduled task to other resource nodes for re-execution. During the task processing of the resource node, the target scheduled task that experiences a resource overflow is due to OOM (Out of Memory) and other tasks that cannot continue to run. Therefore, stopping its execution can ensure the system stability of the resource node, thereby enabling the normal execution of other tasks in the resource node. Subsequently, rescheduling the target scheduled task to other resource nodes for re-execution maximizes the completion rate of the service task and improves the success rate of the service task execution.

[0103] Optionally, sub-step 2041 may specifically include:

[0104] Sub-step 20411: When there are multiple target tasks to be scheduled, obtain the stop priority of the target tasks to be scheduled; the stop priority of the target tasks to be scheduled is positively correlated with the second resource requirement of the target tasks to be scheduled.

[0105] Sub-step 20412: Stop the target scheduled task whose stop priority is greater than the preset priority threshold, and reschedule the target scheduled task whose stop priority is greater than the preset priority threshold to other resource nodes for re-execution.

[0106] In this embodiment of the application, regarding sub-steps 20411-20412, when the number of scheduled tasks processed in the resource node is large, multiple target scheduled tasks with resource overflow may occur simultaneously. In this case, a corresponding stop priority can be set for the target scheduled tasks based on the amount of resources used by the target scheduled tasks, and the stop priority is positively correlated with the second required resource amount. Then, the target scheduled tasks with a stop priority greater than the preset priority threshold are stopped, and the target scheduled tasks with a stop priority greater than the preset priority threshold are scheduled to other resource nodes for re-execution. That is, when multiple target scheduled tasks with resource overflow occur, the target scheduled task with the largest second required resource amount is stopped first to ensure that other target scheduled tasks can resume operation as much as possible after the target scheduled task with the largest second required resource amount is stopped. If the target scheduled task with the largest second required resource amount is stopped first, and the remaining target scheduled tasks still cannot resume operation, then the target scheduled task with the largest second required resource amount among the remaining target scheduled tasks continues to be stopped until the OOM phenomenon of the resource node disappears.

[0107] Optionally, sub-step 20412 may specifically include:

[0108] Sub-step 204121: Obtain the current processing progress and used resources of the target scheduled task whose stop priority is greater than the preset priority threshold.

[0109] Sub-step 204122: Based on the current processing progress and the amount of resources used, calculate the target resource requirement of the target scheduled task whose stop priority is greater than the preset priority threshold.

[0110] Sub-step 204123: Based on the target resource demand, schedule the target scheduled task with a stop priority greater than the preset priority threshold to other resource nodes for re-execution.

[0111] In this embodiment of the application, for sub-steps 204121-204123, the stopped target scheduled tasks are rescheduled. Specifically, based on the information that the target scheduled task has been running before OOM, the target resource requirement of the stopped target scheduled task is determined, and according to the target resource requirement and the remaining resources of other resource nodes, the target scheduled task with a stop priority greater than a preset priority threshold is rescheduled to other resource nodes for re-execution.

[0112] Specifically, the target resource requirement of the target task to be scheduled can be obtained from the processing progress and the amount of resources used when the target task to be scheduled is in OOM. That is, according to the percentage of processing progress and the amount of resources used corresponding to that processing progress, the amount of resources needed when the processing progress is 100% is calculated as the target resource requirement of the target task to be scheduled.

[0113] For example, if a target task to be scheduled is currently 50% complete and has used 6GB of memory, the target resource requirement for that task can be 12GB of memory.

[0114] Optionally, the method further includes:

[0115] Step 205: If the first resource requirement of the service task is less than a preset resource threshold, the service task is determined as a task to be scheduled.

[0116] In this embodiment of the application, if the service layer of the cloud server determines through the limit judgment that the first required resource amount of the service task is less than the preset resource amount threshold, then the service task is considered to be relatively small and easier to schedule in the future. The cloud server can directly determine the service task as a task to be scheduled for subsequent scheduling.

[0117] In summary, considering the overall plan and referring to... Figure 10The diagram illustrates the overall process of a task scheduling method, which includes: S1, user equipment submits a service task; the service layer of the cloud server executes S2-S4, where S2 is the service layer's estimation of the resource requirements for the service task; S3 is to determine whether the estimated resource quantity exceeds the limit; if S3 determines that the limit is exceeded, then S4 is executed, which involves slicing the service task into multiple slices, and then S5 is executed after the slices are identified as tasks to be scheduled; if S3 determines that the limit is not exceeded, then the service task is directly designated as a task to be scheduled and then S5 is executed. The cloud server's scheduling layer executes S5-S8. S5 is for issuing and processing the schedule, and S6 is for the scheduling layer to estimate the resource requirements of the tasks to be scheduled. S7 is for the tasks to be scheduled to determine whether there are any schedulable resource nodes in the cluster. If S7 determines that there are no available resource nodes, then S8 is executed to add new resource nodes. Then S9 is executed to schedule the tasks to be scheduled to the new resource nodes, and the new resource nodes execute the tasks to be scheduled. The cloud server's engine layer executes S9-S11. If S7 determines that there are available resource nodes, then S9 is executed to schedule the tasks to be scheduled to the resource nodes, and the resource nodes execute the tasks to be scheduled. S10 is for stopping tasks according to priority when a resource node experiences an OutOfMemoryError (OOM). S11 is for retrying the stopped tasks and then entering S5 to reschedule them to other resource nodes for execution.

[0118] In summary, this application's embodiments can determine whether a service task is difficult to schedule due to excessive resource requirements by estimating and scheduling its resource needs. Service tasks with excessive resource requirements are then segmented into multiple fragmented tasks before subsequent scheduling, thereby reducing the difficulty of subsequent scheduling. Scheduling smaller fragmented tasks to resource nodes reduces the probability of OutOfMemoryError (OOM). Furthermore, if resource overflow occurs during the execution of scheduled tasks on a resource node, this application stops the execution of at least some of the scheduled tasks on that resource node, ensuring the normal operation of the resource node, preventing it from crashing, and improving service stability. Additionally, rescheduling the stopped scheduled tasks to other resource nodes for re-execution maximizes the completion rate of service tasks, improving the success rate of service task execution.

[0119] Reference Figure 11 It shows a block diagram of a task scheduling device provided in an embodiment of this application, including:

[0120] Module 301 is used to obtain service tasks;

[0121] The segmentation module 302 is used to segment the service task into multiple segmented tasks when the first required resource amount of the service task is greater than or equal to a preset resource amount threshold, and to determine the segmented tasks as tasks to be scheduled.

[0122] Execution module 303 is used to schedule the task to be scheduled to the resource node for execution;

[0123] Optionally, the device further includes:

[0124] The scheduling module is used to stop at least some of the scheduled tasks being executed on the resource node when a resource overflow occurs during the execution of the scheduled tasks on the resource node, and to schedule the at least some of the scheduled tasks to other resource nodes for re-execution.

[0125] Optionally, the device further includes:

[0126] The judgment module is used to determine the service task as a task to be scheduled when the first required resource amount of the service task is less than a preset resource amount threshold.

[0127] Optionally, the execution module 303 includes:

[0128] The acquisition submodule is used to acquire the second required resource quantity of the task to be scheduled;

[0129] The first judgment submodule is used to schedule the task to be scheduled to the resource node for execution when there is a resource node with a remaining resource amount greater than or equal to the second required resource amount;

[0130] The second judgment submodule is used to add a new resource node to the cluster to which the resource node belongs when there is no resource node with remaining resources greater than or equal to the second required resource amount, and to schedule the task to be scheduled to the new resource node for execution, wherein the remaining resources of the new resource node are greater than or equal to the second required resource amount.

[0131] Optionally, the service task is an editing task, and the data corresponding to the editing task includes: editing materials, output duration, and the time period in which the editing materials are located within the output duration;

[0132] The segmentation module 302 includes:

[0133] The segmentation module is used to segment the editing task into multiple segment tasks based on the editing material, the output duration, the first required resource amount, and the time period in which the editing material is located within the output duration.

[0134] Optionally, the molecular cutting module includes:

[0135] The quantity determination unit is used to determine the number of tasks of the sharding task based on the first required resource quantity and the preset upper limit of the resource quantity of the sharding task.

[0136] The duration unit is used to determine the segment duration corresponding to the segmented task based on the output duration and the number of tasks;

[0137] The segmentation unit is used to segment the editing material according to the finished video length and the time period in the output length, thereby segmenting the editing task into the number of segment tasks.

[0138] Optionally, the service task is an editing task, and the data corresponding to the task to be scheduled includes: editing materials and special effects processing performed on the editing materials;

[0139] The acquisition submodule includes:

[0140] The acquisition unit is used to acquire the third resource requirement for decoding the clip material, the fourth resource requirement for performing the special effects processing on the clip material, and the fifth resource requirement for encoding the clip material.

[0141] The summing unit is used to sum the third resource requirement, the fourth resource requirement, and the fifth resource requirement to obtain the second resource requirement of the task to be scheduled.

[0142] Optionally, the scheduling module includes:

[0143] The scheduling submodule is used to, in the event of a resource overflow during the execution of the scheduled task on the resource node, stop the target scheduled task on the resource node where at least part of the resource overflow has occurred, and reschedule the target scheduled task to other resource nodes for re-execution.

[0144] Optionally, the scheduling submodule includes:

[0145] A priority unit is used to obtain the stop priority of the target tasks to be scheduled when there are multiple target tasks to be scheduled; the stop priority of the target tasks to be scheduled is positively correlated with the second resource requirement of the target tasks to be scheduled.

[0146] The stop execution unit is used to stop the target scheduled task whose stop priority is greater than a preset priority threshold, and to schedule the target scheduled task whose stop priority is greater than the preset priority threshold to other resource nodes for re-execution.

[0147] Optionally, the stop execution unit includes:

[0148] The acquisition subunit is used to acquire the current processing progress and the amount of resources used for the target scheduled task whose stop priority is greater than a preset priority threshold;

[0149] The demand calculation subunit is used to calculate the target resource demand of the target scheduled task whose stop priority is greater than a preset priority threshold based on the current processing progress and the amount of resources used.

[0150] The rescheduling subunit is used to reschedule the target scheduled task with a stop priority greater than a preset priority threshold to other resource nodes for re-execution based on the target resource demand.

[0151] Optionally, the resource quantity threshold is the configured resource quantity of the resource node with the largest configured resource quantity in the cluster to which the resource node belongs.

[0152] In summary, the embodiments of this application can determine whether a service task is difficult to schedule due to excessive resource requirements by estimating and scheduling the resource requirements of the service task. Service tasks with excessive resource requirements can be divided into multiple fragmented tasks before subsequent scheduling, thereby reducing the difficulty of subsequent scheduling. Scheduling smaller fragmented tasks to resource nodes can reduce the probability of OOM and improve the success rate of service task execution.

[0153] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0154] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In this application, the electronic device includes various types of devices such as terminal devices and server (cluster) devices.

[0155] The embodiments of this disclosure can be implemented as an apparatus configured as desired using any suitable hardware, firmware, software, or any combination thereof, including terminal devices, server (cluster) and other electronic devices. Figure 12 An exemplary apparatus 1000 is schematically shown that can be used to implement the various embodiments described in the embodiments of this application.

[0156] In one embodiment, Figure 12An exemplary device 1000 is shown, which includes one or more processors 1002, a control module (chipset) 1004 coupled to at least one of the processors 1002, a memory 1006 coupled to the control module 1004, a non-volatile memory (NVM) / storage device 1008 coupled to the control module 1004, one or more input / output devices 1010 coupled to the control module 1004, and a network interface 1012 coupled to the control module 1004.

[0157] Processor 1002 may include one or more single-core or multi-core processors, and processor 1002 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, device 1000 can serve as a terminal device, server (cluster), or other device as described in the embodiments of this application.

[0158] In some embodiments, apparatus 1000 may include one or more computer-readable media (e.g., memory 1006 or NVM / storage device 1008) having instructions 1014 and one or more processors 1002 that are combined with the one or more computer-readable media and configured to execute instructions 1014 to implement modules and thereby perform the actions described in this disclosure.

[0159] In one embodiment, the control module 1004 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1002 and / or any suitable device or component communicating with the control module 1004.

[0160] The control module 1004 may include a memory controller module to provide an interface to the memory 1006. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0161] Memory 1006 may be used, for example, to load and store data and / or instructions 1014 for device 1000. In one embodiment, memory 1006 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 1006 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

[0162] In one embodiment, the control module 1004 may include one or more input / output controllers to provide interfaces to the NVM / storage device 1008 and (one or more) input / output devices 1010.

[0163] For example, NVM / storage device 1008 may be used to store data and / or instructions 1014. NVM / storage device 1008 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0164] NVM / storage device 1008 may include storage resources that are physically part of a device on which device 1000 is mounted, or that are accessible to the device but do not necessarily have to be part of the device. For example, NVM / storage device 1008 may be accessed via a network via one or more input / output devices 1010.

[0165] One or more input / output devices 1010 may provide an interface for device 1000 to communicate with any other suitable device. Input / output devices 1010 may include communication components, audio components, sensor components, etc. Network interface 1012 may provide an interface for device 1000 to communicate via one or more networks. Device 1000 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, etc., or combinations thereof.

[0166] In one embodiment, at least one of the processors 1002 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 1004. In one embodiment, at least one of the processors 1002 may be logically packaged with one or more controllers of the control module 1004 to form a system-in-package (SiP). In one embodiment, at least one of the processors 1002 may be integrated with the logic of one or more controllers of the control module 1004 on the same die. In one embodiment, at least one of the processors 1002 may be integrated with the logic of one or more controllers of the control module 1004 on the same die to form a system-on-a-chip (SoC).

[0167] In various embodiments, device 1000 may be, but is not limited to, a server, a desktop computing device, or a mobile computing device (e.g., a laptop, a handheld computing device, a tablet, a netbook, etc.). In various embodiments, device 1000 may have more or fewer components and / or different architectures. For example, in some embodiments, device 1000 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0168] The detection device can use a main control chip as a processor or control module, and sensor data, position information, etc. can be stored in a memory or NVM / storage device. The sensor group can be used as an input / output device, and the communication interface can include a network interface.

[0169] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0170] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0171] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal equipment, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0174] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0175] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0176] The above provides a detailed description of the task scheduling method, apparatus, electronic device, and machine-readable medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. Under the same circumstances, those skilled in the art will find that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A task scheduling method, characterized in that, include: Obtain service tasks; If the first resource requirement of the service task is greater than or equal to a preset resource threshold, the service task is divided into multiple fragmented tasks, and the fragmented tasks are identified as tasks to be scheduled. The task to be scheduled is scheduled to a resource node for execution; If a resource overflow occurs during the execution of the scheduled task on the resource node, at least some of the scheduled tasks executed on the resource node shall be stopped, and the at least some of the scheduled tasks shall be rescheduled to other resource nodes for re-execution. The resource quantity threshold is the configured resource quantity of the resource node with the largest configured resource quantity in the cluster to which the resource node belongs; The step of scheduling the task to be scheduled to the resource node for execution includes: Obtain the second resource requirement of the task to be scheduled; If there is a resource node with remaining resources greater than or equal to the second required resource amount, the task to be scheduled will be scheduled to that resource node for execution. If no resource node has remaining resources greater than or equal to the second required resource quantity, a new resource node is added to the cluster to which the resource node belongs, and the task to be scheduled is scheduled to be executed on the new resource node, wherein the remaining resources of the new resource node are greater than or equal to the second required resource quantity; wherein the update and optimization frequency of the auxiliary parameters used to estimate the second required resource quantity is greater than the update and optimization frequency of the auxiliary parameters used to estimate the first required resource quantity; the first required resource quantity is estimated by the service layer, and the second required resource quantity is estimated by the scheduling layer.

2. The method according to claim 1, characterized in that, The method further includes: If the first resource requirement of the service task is less than a preset resource threshold, the service task is determined as a task to be scheduled.

3. The method according to claim 1 or 2, characterized in that, The service task is an editing task, and the data corresponding to the editing task includes: editing materials, output duration, and the time period in which the editing materials are located within the output duration; The process of dividing the service task into multiple fragmented tasks includes: Based on the edited material, the output duration, the first required resource quantity, and the time period in which the edited material is located within the output duration, the editing task is divided into multiple segmented tasks.

4. The method according to claim 3, characterized in that, The step of dividing the editing task into multiple segmented tasks based on the edited material, the output duration, the first required resource quantity, and the time period in which the edited material is located within the output duration includes: The number of tasks in the sharding task is determined based on the first required resource quantity and the preset upper limit of the resource quantity of the sharding task. Based on the output duration and the number of tasks, determine the segment duration corresponding to the segmented task; Based on the final video length and the time period in which the edited material appears within the output length, the edited material is segmented, thereby dividing the editing task into the number of segmented tasks.

5. The method according to claim 1, characterized in that, The service task is an editing task, and the data corresponding to the task to be scheduled includes: editing materials and special effects processing performed on the editing materials; The step of obtaining the second resource requirement of the task to be scheduled includes: Obtain the third resource requirement for decoding the clip material, the fourth resource requirement for performing the special effects processing on the clip material, and the fifth resource requirement for encoding the clip material; The third, fourth, and fifth resource requirements are summed to obtain the second resource requirement of the task to be scheduled.

6. The method according to claim 1, characterized in that, If a resource overflow occurs during the execution of the scheduled tasks on the resource node, at least some of the scheduled tasks executing on the resource node are stopped, and the at least some of the scheduled tasks are rescheduled to other resource nodes for re-execution, including: If a resource overflow occurs during the execution of the scheduled task on the resource node, the target scheduled task that has at least partially experienced a resource overflow on the resource node shall be stopped, and the target scheduled task shall be rescheduled to other resource nodes for re-execution.

7. The method according to claim 6, characterized in that, The step of stopping at least some of the target scheduled tasks in the resource nodes that have experienced resource overflow, and rescheduling the target scheduled tasks to other resource nodes for re-execution, includes: When there are multiple target tasks to be scheduled, the stop priority of each target task to be scheduled is obtained; the stop priority of each target task to be scheduled is positively correlated with the second resource requirement of each target task to be scheduled. Stop the target scheduled task whose stop priority is greater than the preset priority threshold, and reschedule the target scheduled task whose stop priority is greater than the preset priority threshold to other resource nodes for re-execution.

8. The method according to claim 7, characterized in that, The step of scheduling the target task with a stop priority greater than a preset priority threshold to other resource nodes for execution includes: Obtain the current processing progress and used resources of the target scheduled task whose stop priority is greater than a preset priority threshold; Based on the current processing progress and the amount of resources used, calculate the target resource requirement of the target scheduled task whose stop priority is greater than a preset priority threshold; Based on the target resource demand, the target scheduled tasks with a stop priority greater than a preset priority threshold are scheduled to other resource nodes for re-execution.

9. A task scheduling device, characterized in that, include: The acquisition module is used to acquire service tasks; The segmentation module is used to segment the service task into multiple segmented tasks when the first required resource amount of the service task is greater than or equal to a preset resource amount threshold, and to determine the segmented tasks as tasks to be scheduled. An execution module is used to schedule the task to be scheduled to the resource node for execution; specifically, the execution module is used to obtain the second resource requirement of the task to be scheduled. If there is a resource node with remaining resources greater than or equal to the second required resource amount, the task to be scheduled will be scheduled to that resource node for execution. If there is no resource node with remaining resources greater than or equal to the second required resource amount, a new resource node is added to the cluster to which the resource node belongs, and the task to be scheduled is scheduled to be executed on the new resource node, wherein the remaining resources of the new resource node are greater than or equal to the second required resource amount; wherein the update and optimization frequency of the auxiliary parameters used to estimate the second required resource amount is greater than the update and optimization frequency of the auxiliary parameters used to estimate the first required resource amount. The scheduling submodule is used to stop the target task that has at least partially overflowed in the resource node when a resource overflow occurs during the execution of the scheduled task on the resource node, and to reschedule the target task to other resource nodes for re-execution; the resource threshold is the configured resource amount of the resource node with the largest configured resource amount in the cluster to which the resource node belongs.

10. An electronic device, characterized in that, include: processor; and a memory having executable code stored thereon, which, when executed, causes the processor to perform the method as described in any one of claims 1 to 8.

11. One or more machine-readable media having executable code stored thereon, which, when executed, causes a processor to perform the method as claimed in any one of claims 1 to 8.

Images